New issue
Advanced search Search tips

Issue 731285 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Add support for applying private patches to kernel builds

Project Member Reported by groeck@chromium.org, Jun 8 2017

Issue description

Under some circumstances it may be necessary to apply private patches to kernel builds.

 

Comment 1 by vapier@chromium.org, Oct 16 2017

Cc: vapier@chromium.org
Labels: OS-Chrome
the context for this is  issue 724093 .  comment #17 pointed out:
"14:41:18: ERROR: We do not yet support local patching for projects that are checked out to multiple directories". If you know a way around that please let me know.

imo we should be fixing that instead of adding project/ebuild-specific hacks that are hard to maintain.  the kernel isn't the only project to take advantage of shared repo checkouts.

Comment 2 by groeck@chromium.org, Oct 16 2017

Labels: -Pri-3 M-63 Pri-1
#1: I'll be happy to work with you on finding a better and more generic solution, but please note that this isn't about applying local patches (as mentioned in the other bug). We need to be able to apply private patches to release builds. Being able to apply local patches to a trybot run would be nice, but doesn't solve that problem.
That reminds me to update the priority for this bug.

Comment 3 by vapier@chromium.org, Oct 16 2017

that scenario should be addressed by forking ebuilds, not by applying out of band patches which the bots don't know about.

i.e. you can already satisfy this by coping the relevant kernel ebuild to chromeos-overlay.
Project Member

Comment 4 by bugdroid1@chromium.org, Oct 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/642e80548f5153ad054fa5a110f91d16cacbb6b4

commit 642e80548f5153ad054fa5a110f91d16cacbb6b4
Author: Guenter Roeck <groeck@chromium.org>
Date: Tue Oct 17 09:37:31 2017

cros-kernel2: Add support for private patches

Under some circumstances it may be necessary to apply a series of
private patches to the kernel. Add support for it.

With this patch, the kernel build system will look for private patches
in directory "${ROOT}/usr/share/chromeos-kernel-patches". The matching
chromeos-kernel-patches-<version> ebuilds are expected to drop
applicable patches in that location.

Patches are only applied if the kernel has been not been checked out
using cros_workon. If the kernel has been checked out, patches will
only be applied if the apply_patches use flag is set.

BUG= chromium:731285 
TEST=Build image with patches in private overlays

Change-Id: Ic0f98db893936c9e6868a8a8894465b8acaa6e31
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/528472
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>

[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-3_8/chromeos-kernel-3_8-9999.ebuild
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-4_4/chromeos-kernel-4_4-9999.ebuild
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-3_18/chromeos-kernel-3_18-9999.ebuild
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/eclass/cros-kernel2.eclass
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-3_14/chromeos-kernel-3_14-9999.ebuild
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-4_12/chromeos-kernel-4_12-9999.ebuild
[modify] https://crrev.com/642e80548f5153ad054fa5a110f91d16cacbb6b4/sys-kernel/chromeos-kernel-3_10/chromeos-kernel-3_10-9999.ebuild

Project Member

Comment 5 by bugdroid1@chromium.org, Oct 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/19ff4a6e85980279738de9c92d9792eb99c02c6b

commit 19ff4a6e85980279738de9c92d9792eb99c02c6b
Author: Guenter Roeck <groeck@chromium.org>
Date: Tue Oct 17 09:37:30 2017

sys-kernel: Add dummy ebuilds for kernel patch files

The ebuilds defined here will be overwritten by private overlays if needed.

BUG= chromium:731285 
TEST=Build image with patches in private overlays

Change-Id: I640d0c01f51e9a829dc1095725535f2e2efd5292
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/531424

[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-3_14/chromeos-kernel-patches-3_14-0.0.1-r0.ebuild
[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-3_18/chromeos-kernel-patches-3_18-0.0.1-r0.ebuild
[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-4_12/chromeos-kernel-patches-4_12-0.0.1-r0.ebuild
[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-4_4/chromeos-kernel-patches-4_4-0.0.1-r0.ebuild
[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-3_10/chromeos-kernel-patches-3_10-0.0.1-r0.ebuild
[add] https://crrev.com/19ff4a6e85980279738de9c92d9792eb99c02c6b/sys-kernel/chromeos-kernel-patches-3_8/chromeos-kernel-patches-3_8-0.0.1-r0.ebuild

Comment 6 by groeck@chromium.org, Oct 17 2017

#3: You mean the complete contents of, say, sys-kernel/chromeos-kernel-4_4 ?

I found one example with gcc-libs-9999.ebuild, which would suggest that the -9999.ebuild file is the same and that the release file needs to have a release number higher than the one in the public repository. For example, I would copy the entire content of sys-kernel/chromeos-kernel-4_4 and then replace chromeos-kernel-4_4-4.4.86-r1139.ebuild with, say, chromeos-kernel-4_4-4.4.86-r2000.ebuild.
Would that work, and would it be preferred ? Does this approach replace the public -9999.ebuild with the private version, even if the version is the same ?
If so, I'll be happy to give it a try.

Comment 7 by groeck@chromium.org, Oct 17 2017

Cc: chirantan@chromium.org

Comment 8 by vapier@chromium.org, Oct 17 2017

yes, copy the 9999 ebuild to the chromeos-overlay.  you can change $PV so the private/public stay unique, and you know they never go higher.
  public: chromeos-kernel-4_12-4.12.14-r38.ebuild
  private: chromeos-kernel-4_12-4.12.14_p123-r38.ebuild
which is trivial to accomplish via files/chromeos-version.sh

with the patches approach, you have no way of knowing what a $PVR represents or to guarantee that every running that $PVR includes your patches.

for one-off testing, fix the -p handling in chromite instead of copying & pasting lots of random code for every package.  it simply doesn't scale and is way more of a pita to for people to test.

for actual releases, we should be copying the ebuild.  this isn't really any more work than these random custom patches packages, and it's a lot more sane.  it's also what we've already been doing with past projects (dnsmasq, nss, nspr, wpa_supplicant, etc...).

Comment 9 by groeck@chromium.org, Oct 18 2017

"which is trivial to accomplish via files/chromeos-version.sh"

Ah yes, for you. For me it is going to take a bit longer. The basic approach is working, I'll just have to figure out the file naming. Once I do, I'll submit patches to revert the original changes and use the new approach.

Adding to #9: I could just change the private version of chromeos-version.sh to always add "_p1" or something to the version. Problem with that would be though that this only really works if we always keep the private directories around, even if there are no pending patches. This would effectively make the public version obsolete. Is that what we want ?
I was looking for a means for chromeos-version.sh to detect if it is running from the private ebuild. Is there a clean way to do that ? Or do you have something else in mind ?

Project Member

Comment 11 by bugdroid1@chromium.org, Oct 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/96c2b8e890796110c5774648a871fac8ba1622bd

commit 96c2b8e890796110c5774648a871fac8ba1622bd
Author: Guenter Roeck <groeck@chromium.org>
Date: Sat Oct 28 14:14:49 2017

cros-kernel2: Apply local patches from FILESDIR

Instead of picking patches from a non-standard location,
pick them up directly from FILESDIR.

This patch partially reverts commit 642e80548f5 ("cros-kernel2: Add
support for private patches"); we no longer pick up patches from
a private path, and the build no longer depends on
sys-kernel/chromeos-kernel-patches-<version>.

BUG= chromium:731285 
TEST=Build image with patches in FILESDIR
CQ-DEPEND=CL:742425

Change-Id: I1457645182dc85e1811d58300cf7475fb70456f2
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/724264
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/96c2b8e890796110c5774648a871fac8ba1622bd/eclass/cros-kernel2.eclass

Project Member

Comment 13 by bugdroid1@chromium.org, Oct 28 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/chromeos-overlay/+/24ed990175d34714305b2cc5247cf5370e8cb1e0

commit 24ed990175d34714305b2cc5247cf5370e8cb1e0
Author: Guenter Roeck <groeck@chromium.org>
Date: Sat Oct 28 17:53:38 2017

Project Member

Comment 14 by bugdroid1@chromium.org, Oct 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/10ebd438682f874440bda64824ff0672ae48b5ac

commit 10ebd438682f874440bda64824ff0672ae48b5ac
Author: Guenter Roeck <groeck@chromium.org>
Date: Sat Oct 28 17:53:37 2017

Revert "sys-kernel: Add dummy ebuilds for kernel patch files"

This reverts commit 19ff4a6e85980279738de9c92d9792eb99c02c6b.

We are going to pick up patches directly from ${FILESDIR}. If patches
are needed, the affected kernel ebuilds will be replicated in private
or board overlays. Separate dummy packages just for kernel patches are
not necessary.

BUG= chromium:731285 
TEST=None

Change-Id: I81df106edbe4ecdff53e8d45fca31654af9afca5
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/737543
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-3_14/chromeos-kernel-patches-3_14-0.0.1-r0.ebuild
[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-3_18/chromeos-kernel-patches-3_18-0.0.1-r0.ebuild
[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-4_12/chromeos-kernel-patches-4_12-0.0.1-r0.ebuild
[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-4_4/chromeos-kernel-patches-4_4-0.0.1-r0.ebuild
[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-3_10/chromeos-kernel-patches-3_10-0.0.1-r0.ebuild
[delete] https://crrev.com/607a9680eba57c90e88eb606639f0bfae3708b58/sys-kernel/chromeos-kernel-patches-3_8/chromeos-kernel-patches-3_8-0.0.1-r0.ebuild

Project Member

Comment 15 by bugdroid1@chromium.org, Oct 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d361d157b2daad1afb29f1979ba75a0129757c9b

commit d361d157b2daad1afb29f1979ba75a0129757c9b
Author: Guenter Roeck <groeck@chromium.org>
Date: Sat Oct 28 17:53:37 2017

sys-kernel: Update kernel version scripts

Append _p1 to the kernel version if the version script runs from
a board overlay directory. This ensures that the generated kernel
version takes precedence over kernel versions generated by scripts
in chromiumos-overlay.

With this change, the version scripts can be used in both places
and don't have to be modified to run from board overlay directories.

BUG= chromium:731285 
TEST=Run version script from board or private overlays

Change-Id: I8b669f51ab6b98ce6f5fd12138a553a6ac0befd3
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/737544
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-3_8/files/chromeos-version.sh
[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-3_18/files/chromeos-version.sh
[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-3_14/files/chromeos-version.sh
[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-4_4/files/chromeos-version.sh
[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-3_10/files/chromeos-version.sh
[modify] https://crrev.com/d361d157b2daad1afb29f1979ba75a0129757c9b/sys-kernel/chromeos-kernel-4_12/files/chromeos-version.sh

Status: Fixed (was: Started)
thanks for sticking it out
... and thanks a lot for your patience.

Project Member

Comment 19 by bugdroid1@chromium.org, Dec 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/48e75fcc0f86f8737d0f69ea0a2803676506441d

commit 48e75fcc0f86f8737d0f69ea0a2803676506441d
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Dec 21 04:21:05 2018

cros-kernel2.eclass: move `use` out of global scope

The ebuild spec says `use` is invalid in global scope, so move it to
the src_unpack phase before the cros-workon.eclass needs it.

BUG= chromium:731285 ,  chromium:874977 
TEST=precq passes

Change-Id: I25d190b6f4932b0e735298d6ab3a890fad597ac3
Reviewed-on: https://chromium-review.googlesource.com/c/1316147
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/48e75fcc0f86f8737d0f69ea0a2803676506441d/eclass/cros-kernel2.eclass

Sign in to add a comment