I've already started on a fix for this; should be in the tree in a day or two.

As a reference from crrev.com/i/527947:

I'd recommend removing BCS URLs from the bsp config, to prevent unnecessary cherry-picks when people want to sync the config between factory branches and release or ToT, because we usually don't want to include or change the firmware image in factory branches.

Since BCS is really a build time thing, maybe leave chromeos-firmware-* ebuild simple and standalone so it's isolated from system config stuff.

Actually, that kind of move would not be sufficient to avoid cherry-picks: the configuration also includes:
* the filenames that drive the pack_firmware script (not just the download)
* the Signer instructions needed to perform signing of firmware artifacts on the branch
* the HW identity used to select the correct model when running updater4.sh

That said, we could potentially explore segmentation of firmware information into a different config file that is joined with the main config. such that only cherry-picks on that file would be needed into factory branches--that would be relatively easy at the cost of making it harder to update the configuration.

WRT to the issue of uprev not working reliably, I believe that Simon and I have found a good, final solution to the problem. I'm sending these two CL's through the CQ over the next day to prove that it works: https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-reef-private/+/529819

I think this solution may still cause problems. We have public/private repos, and this is sill counter to portage's ebuild separation assumptions. 

Can we just stop programmatically generating SRC_URI, and keep changes within their own ebuilds?

Any scheme that involves working against portage is going to be an ongoing maintenance and flake burden.

I'm thinking a different way, for instance in each firmware ebuild, we use BASH array to configure the mapping. i.e.,

CROS_FIRMWARE_IMAGES=(
 [model1]="bcs://Model1.1923.tbz"
 [model2]="bcs://Model2.1923.tbz"
 [*]="bcs://GenericModel.1923.tbz"
)

And similar settings for other variables.

Re: #5, we looked at that too today: we couldn't find any example of a firmware ebuild that wasn't also collocated with the bsp ebuild.

Re; #6, yes, that was the solution I was working on this morning. If it the desire for BCS/firmware repo split arise, that will work but that's less elegant: we can do programmatic generation of the SRC_URI's (and other variables) from the config that is written into the firmware ebuild's files/ directory. In that solution, the config would still be the source of truth and have to be updated at the same time as the firmware ebuild but at least that implementation could span public/private repositories.

re: #5
We haven't done any public / private repos yet because we haven't launched anything w/ uni yet. It's a pending bug.

reef-uni runs into the BCS permission issue as well: 
Likely we'll need to put each firmware and config in it's own ebuild if we plan to be able to unibuild anything with proprietary partner config.

I think I had long discussion with Vapier for if we can put the mapping of SRC into files/ directory, and the conclusion is that's hard. Vapier was considering some hacks for future but it is better to not doing that. SRC_URI should be easily parsed by reading ebuild & eclass files, not from external source.

> we couldn't find any example of a firmware ebuild that wasn't also collocated with the bsp ebuild.
Maybe I misread this, but most firmware ebuilds not using unibuild do not  depend on bsp ebuild.

> We haven't done any public / private repos yet because we haven't launched anything w/ uni yet. It's a pending bug.

> Maybe I misread this, but most firmware ebuilds not using unibuild do not  depend on bsp ebuild.

I'm talking about https://cs.corp.google.com/search/?q=f:bsp-%5Cw%2B/$&m=25&sq=package:%5Echromeos&type=cs which is a legacy analogue to the role that unibuild configuration plays today. This is the data from which we drove our conclusion.

> Likely we'll need to put each firmware and config in it's own ebuild if we plan to be able to unibuild anything with proprietary partner config.

We ran this down a few weeks ago with various folks are were unable to come up with any historical example of a case where a partner understanding wouldn't wave this away: after all, all of the contents of the images are public as soon as we publish a built image to real users.

> I think I had long discussion with Vapier for if we can put the mapping of SRC into files/ directory, and the conclusion is that's hard. Vapier was considering some hacks for future but it is better to not doing that. SRC_URI should be easily parsed by reading ebuild & eclass files, not from external source.

I ran some experiments today and I was able to get it to work within http://cs/chromeos_public/src/third_party/chromiumos-overlay/eclass/cros-firmware.eclass?l=453&rcl=981223ed74490389f6ce07ec5060513834404ac1 . This would be combined with CROS_WORKON_SUBDIRS_TO_REV to get it to work both locally and on the builders.

On Thu, Dec 14, 2017, 8:36 PM jclinton via monorail <
monorail+v2.3511897545@chromium.org> wrote:.

Unclear who you discussed this with, maybe only for migrations? But the
consensus in partner eng is that this is a nonstarter for new projects with
significant hardware differentiation.

Anyways, this fix is probably fine for coral but should be considered a
workaround rather than a final fix.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-reef-private/+/1571b9bc78ca6fe487c3f3176abdcd48ae43070a

commit 1571b9bc78ca6fe487c3f3176abdcd48ae43070a
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Dec 15 05:25:10 2017

> I'm talking about https://cs.corp.google.com/search/?q=f:bsp-%5Cw%2B/$&m=25&sq=package:%5Echromeos&type=cs

Oh, so you are saying "every overlays do have bsp, so all firmware ebuilds will collocate with bsp".

Well, bsp is the virtual package people created for every projects so it's always there, and I'd say firmware ebuilds usually do no have "direct" dependency on bsp packages, and that's a big difference.

But anyway, I agree with Nick on #12. It's fine to have your fix for Coral landed .

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-reef-private/+/a6185ea9c59bb4a9c1053b7b13765a4d61740a2a

commit a6185ea9c59bb4a9c1053b7b13765a4d61740a2a
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Dec 15 21:23:45 2017

I submitted https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-reef-private/+/529878 as an experiment to prove that this works but it failed to uprev: https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Fmaster-paladin%2F17209%2F%2B%2Frecipes%2Fsteps%2FUprev%2F0%2Fstdout.

There is a bug somewhere in the https://cs.corp.google.com/chromeos_public/chromite/lib/portage_util.py but it's very difficult to debug because we can't easily recreate the environment on the CQ. Continuing to look at it. One potential explanation is CROS_WORKON_LOCALNAME.

That said, I'm increasingly leaning toward my original plan which is to just simply have SRC_URI stored in the firmware ebuilds but derived from the configuration at the same time the person doing the uprev generates the new Manifest.

Proposed fix in 2 CL's:
https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/831236
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-reef-private/+/530641

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/e5eca43273093d85f324c3d403f4742d0cc2a3e2

commit e5eca43273093d85f324c3d403f4742d0cc2a3e2
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 19 01:54:57 2017

cros-firmware.eclass: Add feature to get URI's from files/srcuris

This allows a firmware ebuild to have a file "files/srcuris" that
declares the SRC_URI's used for this ebuild. UniBuild configuration is
used to generate this file (see next CL for example). Like this:

  export BOARD=reef
  cros_workon --board=${BOARD?} start chromeos-config-bsp \
    chromeos-firmware-${BOARD?}
  emerge-${BOARD?} chromeos-config-bsp chromeos-config
  cros_config_host -c \
    /build/${BOARD?}/usr/share/chromeos-config/config.dtb \
    get-firmware-uris > files/srcuris
  ebuild-${BOARD?} chromeos-firmware-${BOARD?}-9999.ebuild manifest
  emerge-${BOARD?} chromeos-firmware-${BOARD?}

This should solve the CQ uprev problem because there is now a file
changed within the ebuild (as opposed to outside of it).

BUG=chromium:794694
TEST=followed instructions above (with modified reef ebuild in next CL)

Change-Id: Ia41dc9cbbe3ae866de9719d292cbca324c2fe042
Reviewed-on: https://chromium-review.googlesource.com/831236
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: C Shapiro <shapiroc@google.com>

[modify] https://crrev.com/e5eca43273093d85f324c3d403f4742d0cc2a3e2/eclass/cros-firmware.eclass

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/28b7cdb0442bab721e364bc9c384dcd53e5ba93e

commit 28b7cdb0442bab721e364bc9c384dcd53e5ba93e
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Wed Dec 20 06:28:45 2017

cros_update_firmware: Transition to using srcuris

This will make updating the ebuild instructions simpler.

BUG=chromium:794694
TEST=run script after update; confirm firmware updater built

Change-Id: I120c0737a9e6e547e563841a9efefad2157c6e7c
Reviewed-on: https://chromium-review.googlesource.com/832825
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Patrick Berny <pberny@chromium.org>
Reviewed-by: C Shapiro <shapiroc@google.com>

[modify] https://crrev.com/28b7cdb0442bab721e364bc9c384dcd53e5ba93e/cros_update_firmware

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-kahlee-private/+/0a33002d54c0538b5bdf39c27a13bf62f5208584

commit 0a33002d54c0538b5bdf39c27a13bf62f5208584
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Dec 21 02:32:02 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/2f76e29788f390b75deab7bfa230919a44612ea1

commit 2f76e29788f390b75deab7bfa230919a44612ea1
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Dec 21 09:04:36 2017

cros_update_firmware: Fix use of BOARD env var

This was using the environment variable, not the local.

BUG=chromium:794694
TEST=./cros_update_firwmare --board=kahlee

Change-Id: I0fe2818308df0504e0f37ec98d2d8c16d126223a
Reviewed-on: https://chromium-review.googlesource.com/837422
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: C Shapiro <shapiroc@google.com>

[modify] https://crrev.com/2f76e29788f390b75deab7bfa230919a44612ea1/cros_update_firmware

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/a1e068f470d34117256094b5c13fa2b4f8b91474

commit a1e068f470d34117256094b5c13fa2b4f8b91474
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Dec 21 16:37:55 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/3e3f6fc78b48932e5ddbc99e715ab2781cf716a1

commit 3e3f6fc78b48932e5ddbc99e715ab2781cf716a1
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Dec 21 16:37:56 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-rainier-private/+/1b01946a14308a200ae4ad6139f1a294b61cf366

commit 1b01946a14308a200ae4ad6139f1a294b61cf366
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Dec 21 16:37:55 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/9791d19504db80aaff71fc59e249264f3042dd68

commit 9791d19504db80aaff71fc59e249264f3042dd68
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Dec 22 02:08:55 2017

cros-firmware.eclass: Recycle FILESDIR replacement for early SRC_URI

BUG=chromium:794694
TEST=emerge-rainier chromeos-firmware-rainier

Change-Id: I14bed24117adabfdee1650509fb909aad886a725

[modify] https://crrev.com/9791d19504db80aaff71fc59e249264f3042dd68/eclass/cros-firmware.eclass

There was no way to be certain that things were working until someone tried to update a firmware version. One such CL is in flight now (https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-fizz-private/+/534338) and the pre-CQ logs show that it didn't work: https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromiumos.tryserver%2Fno_vmtest_pre_cq%2F198049%2F%2B%2Frecipes%2Fsteps%2FUprev%2F0%2Fstdout .

The reason, though, isn't related to the second solving strategy: it's that all of the firmware ebuilds have incorrect CROS_WORKON_PROJECT's defined. I might as well fix this while I'm here so we aren't blamed for that. They are set to:

  CROS_WORKON_PROJECT="chromiumos/platform/firmware"

when they should be:

  CROS_WORKON_PROJECT="chromeos/overlays/overlay-fizz-private"

However, this may also indicate that the first strategy in #c4 (use CROS_WORKON_SUBDIRS_TO_REV to point to the BSP ebuild) may have failed (#c16) for the same reason. That said, the second strategy (srcuris in files/) support spanning across public/private repos.

I'll submit CL's in all private repos to set CROS_WORKON_PROJECT to the correct value.

Fixes out for review:
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-reef-private/+/535840
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-fizz-private/+/535841
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-coral-private/+/535842
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-rainier-private/+/535843

Those failed on trybots for 3 reasons:

1. There's a series of hacks in portage_util.py to try to infer the project in platform/. These hacks can be overridden with:

-CROS_WORKON_LOCALNAME="firmware"                                  
 CROS_WORKON_PROJECT="chromeos/overlays/overlay-rainier-private"   
+CROS_WORKON_SRCPATH="src/private-overlays/overlay-rainier-private"

Once we do that, though, we see:

2. cros-firmware.eclass assumes that platform/firmware is the directory from which src_compile and src_install are executed. It's easy enough change it to allow pack_firmware.py run outside of that context. 

Once we fix that, we see:

3. We also install a handful of scripts from platform/firmware/sbin instead of simply having them be installed by a normal ebuild. The historical reason for this is support for per-updater.sh specializations of these scripts. E.g. there's a firmware-boot-update and firmware-boot-update.3 which are conditionally installed on the target root depending on the value of CROS_FIRMWARE_SCRIPT. This prevents of from creating and using a simple ebuild for these scripts (or, at least, not without submitting a CL to every private repo depending on the right simple ebuild).

The only way that I can come up with that will clean up this mess is to either:
* add support to portage_util.py to implicitly uprev the ebuild if anything in the ebuild files/ is modified.
* add support for multiple projects concurrently with CROS_WORKON_SUBDIRS_TO_REV. This would be extremely invasive, though.

I'm going to implicitly uprev if files/ is modified.

Fix out for review: https://chromium-review.googlesource.com/844159

4 CL's above set to depend on it.

Sent to trybot to confirm that it works: https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/pre_cq/builds/74511

Tryjob passed Uprev: https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/pre_cq/builds/74511

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/d3924c7d7ee19967ea382c616c6b5c6928d1a4db

commit d3924c7d7ee19967ea382c616c6b5c6928d1a4db
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Mon Dec 25 03:08:34 2017

Putting overlay into CROS_WORKON_PROJECT sounds like getting more hacks, since overlays are not supposed to be projects...

So you really don't want to consider moving firmware stuff out of config-bsp dtsi?

I feel that original firmware ebuilds are much easier to understand and maintain.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/9b4f8892bfa0a9329b9ecc8199076784007e87d0

commit 9b4f8892bfa0a9329b9ecc8199076784007e87d0
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 00:31:23 2017

cros-firmware.eclass: Add feature to get URI's from files/srcuris

This allows a firmware ebuild to have a file "files/srcuris" that
declares the SRC_URI's used for this ebuild. UniBuild configuration is
used to generate this file (see next CL for example). Like this:

  export BOARD=reef
  cros_workon --board=${BOARD?} start chromeos-config-bsp \
    chromeos-firmware-${BOARD?}
  emerge-${BOARD?} chromeos-config-bsp chromeos-config
  cros_config_host -c \
    /build/${BOARD?}/usr/share/chromeos-config/config.dtb \
    get-firmware-uris > files/srcuris
  ebuild-${BOARD?} chromeos-firmware-${BOARD?}-9999.ebuild manifest
  emerge-${BOARD?} chromeos-firmware-${BOARD?}

This should solve the CQ uprev problem because there is now a file
changed within the ebuild (as opposed to outside of it).

BUG=chromium:794694
TEST=followed instructions above (with modified reef ebuild in next CL)

Change-Id: Ia41dc9cbbe3ae866de9719d292cbca324c2fe042
Reviewed-on: https://chromium-review.googlesource.com/831236
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: C Shapiro <shapiroc@google.com>
Reviewed-on: https://chromium-review.googlesource.com/844040
Commit-Queue: David Wu <david_wu@quanta.corp-partner.google.com>
Tested-by: David Wu <david_wu@quanta.corp-partner.google.com>

[modify] https://crrev.com/9b4f8892bfa0a9329b9ecc8199076784007e87d0/eclass/cros-firmware.eclass

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/1d65c266d1b119063cb5779a7878b525e33327cc

commit 1d65c266d1b119063cb5779a7878b525e33327cc
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 03:14:12 2017

> Putting overlay into CROS_WORKON_PROJECT sounds like getting more hacks, since overlays are not supposed to be projects...

After #c29, we're no longer going to change CROS_WORKON_PROJECT. It wouldn't work without ~100 CL's because there are too many old hacks in place.

> So you really don't want to consider moving firmware stuff out of config-bsp dtsi?

This is one step forward in the much larger effort to unify build, runtime, test, factory, and release configuration across all of CrOS. To not use configuration for firmware would make achieving other parts of the infrastructure goals unobtainable.

> I feel that original firmware ebuilds are much easier to understand and maintain.

Yes, that true. Though, I would argue that the original firmware ebuilds were also not optimal: a few of us of have been working around this area for months and are are still discovering new facets to the complexity in the old setup (i.e. #29 is an example of the complexity that already existed).

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d9d9f41b65d98c0e24b319f7c6bbabfe41e07704

commit d9d9f41b65d98c0e24b319f7c6bbabfe41e07704
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 03:19:18 2017

cros-firmware.eclass: Recycle FILESDIR replacement for early SRC_URI

BUG=chromium:794694
TEST=emerge-rainier chromeos-firmware-rainier

Change-Id: I14bed24117adabfdee1650509fb909aad886a725
Reviewed-on: https://chromium-review.googlesource.com/844041
Reviewed-by: Jason Clinton <jclinton@chromium.org>
Commit-Queue: David Wu <david_wu@quanta.corp-partner.google.com>
Tested-by: David Wu <david_wu@quanta.corp-partner.google.com>

[modify] https://crrev.com/d9d9f41b65d98c0e24b319f7c6bbabfe41e07704/eclass/cros-firmware.eclass

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/4f88f77f4f72f87b5c1ad2759270f4c34c5f032f

commit 4f88f77f4f72f87b5c1ad2759270f4c34c5f032f
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 05:45:11 2017

RevWorkOnEBuild: Remove enforce_subdir_rev flag

It looks like this was added to flag control the rollout of
CROS_WORKON_SUBDIRS_TO_REV but this was a long time ago and this is
always enabled, now.

BUG=chromium:794694
TEST=cbuildbot/run_tests

Change-Id: Ic13526316d2cab671df64379ebd2e6ba061c0d1a
Reviewed-on: https://chromium-review.googlesource.com/844158
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

[modify] https://crrev.com/4f88f77f4f72f87b5c1ad2759270f4c34c5f032f/lib/portage_util_unittest.py
[modify] https://crrev.com/4f88f77f4f72f87b5c1ad2759270f4c34c5f032f/scripts/cros_mark_as_stable.py
[modify] https://crrev.com/4f88f77f4f72f87b5c1ad2759270f4c34c5f032f/lib/portage_util.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/516066795790200e45985587c1429b27cfe88ee6

commit 516066795790200e45985587c1429b27cfe88ee6
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 08:04:35 2017

portage_util: Always uprev if files/ is modified

See bug for discussion.

BUG=chromium:794694
TEST=cbuildbot/run_tests

Change-Id: Id41a05db04cb4f0376dc5a96daaddff5595c4875

[modify] https://crrev.com/516066795790200e45985587c1429b27cfe88ee6/lib/portage_util_unittest.py
[modify] https://crrev.com/516066795790200e45985587c1429b27cfe88ee6/lib/portage_util.py

Uprev fixes are in. Just need +2 on the board-specific final fixes out for review here (apparently internal Gerrit email wasn't working over the weekend):
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-reef-private/+/535840
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-fizz-private/+/535841
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-coral-private/+/535842
https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-rainier-private/+/535843

Hung-Te, re #33 I feel that this is a very easy problem that any normal build systems would happily support. Portage is making this very hard.

The old ebuild shell variables don't scale to the case where we have a dozen different models and want to have some of them use different firmware.

Any change to the firmware ebuild (e.g. 'version=1, version=2') would be enough to trigger an uprev. This is what coreboot does, for example.

Once Jason is done on this effort we'll take a look at what we can to do make portage more friendly.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/12cacef290b83bacfe4074ca8317d190889f608b

commit 12cacef290b83bacfe4074ca8317d190889f608b
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 23:57:05 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-reef-private/+/b75d66623c6f377d92bb5ad04011f1dbd6ff0cbb

commit b75d66623c6f377d92bb5ad04011f1dbd6ff0cbb
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 23:57:05 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-rainier-private/+/30eaf0af836d3a5f3f26999cce9d9cbf52bf03b1

commit 30eaf0af836d3a5f3f26999cce9d9cbf52bf03b1
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Dec 26 23:57:05 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/8c873df9abaa7ccde359f025d0c5c87246c21688

commit 8c873df9abaa7ccde359f025d0c5c87246c21688
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Wed Dec 27 02:53:27 2017

All done.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/810602ae62dd4e22dab0a5e93a766685999d36bd

commit 810602ae62dd4e22dab0a5e93a766685999d36bd
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Jan 18 22:44:47 2018

Stop using this silly _Print method

This is preventing us from seeing useful debug messages.

BUG=chromium:794694
TEST=cbuildbot/run_tests

Change-Id: I5ce268541b4c4fd3b99e2a7b349976a6dd986072
Reviewed-on: https://chromium-review.googlesource.com/865441
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Brian Norris <briannorris@chromium.org>

[modify] https://crrev.com/810602ae62dd4e22dab0a5e93a766685999d36bd/lib/portage_util.py

We're having an issue where the uprev logic successfully identifies that there are changes to files/ contents, decides to uprev, and then, later, when we go to build, the ebuild being built is the old version. The first report of this was on https://chrome-internal-review.googlesource.com/c/chromeos/overlays/overlay-coral-private/+/550820 two weeks ago and I should have reopened this then.

I'm going to try to fix the immediate issue of files/ uprev not actually working and then start work on transitioning BCS download SRC_URI's to chromeos-config ebuild as discussed with the SIE team (also tracked in https://buganizer.corp.google.com/issues/72809378).

In the SRC_URI in chromeos-config approach, the BCS tarballs are downloaded to the DUT build root during chromeos-config install and then later consumed by ebuild that needs them. For example, Reef's chromeos-config will download Reef.9042.97.1.tbz2 to some location in the build root and then chromeos-firmware-reef will locate it there. Later, when we go to build the final DUT filesystem, we will remove the tarballs so that they don't take up extra space on the DUT root FS.

FYI

in #c29:
> The only way that I can come up with that will clean up this mess is to either:
> * add support to portage_util.py to implicitly uprev the ebuild if anything in the ebuild files/ is modified.
> * add support for multiple projects concurrently with CROS_WORKON_SUBDIRS_TO_REV. This would be extremely invasive, though.

The second option will be supported very soon with CROS_WORKON_SUBTREE (go/cros-workon-subtree, crbug.com/791888). It's essentially CROS_WORKON_SUBDIRS_TO_REV with multiple project support and sandbox safety.
There are two CLs to land to introduce the new variable. One is already landed, and the other (crrev.com/c/886302) is in CQ.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/4ef251e35691db06623ec1a3343950655be514d5

commit 4ef251e35691db06623ec1a3343950655be514d5
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 07:42:00 2018

lib/portage_util.py: Log when we skip uprev due to no changes

This can happen if a WORKON ebuild doesn't have the CROS_WORKON_COMMIT
and CROS_WORKON_TREE variables defined.

BUG=chromium:794694,b:72809378
TEST=cbuildbot/run_tests

Change-Id: I0276aceda8a6dad1fa2b9a2231b648e3bd236e01
Reviewed-on: https://chromium-review.googlesource.com/907733
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/4ef251e35691db06623ec1a3343950655be514d5/lib/portage_util.py

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/5060528bf1da7314c6faf7043e54e1be24e380ba

commit 5060528bf1da7314c6faf7043e54e1be24e380ba
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 10:19:58 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/963e6e0a0cc08669bbb6a4dbcd5bec42bd023b92

commit 963e6e0a0cc08669bbb6a4dbcd5bec42bd023b92
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 10:20:06 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-reef-private/+/fb473fc2c4d2a8bf0f88ac7d973c2821f7991d8b

commit fb473fc2c4d2a8bf0f88ac7d973c2821f7991d8b
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 10:20:01 2018

 Issue 782211  has been merged into this issue.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-kahlee-private/+/c5d24f611ee700ce43f635813643598e330958ab

commit c5d24f611ee700ce43f635813643598e330958ab
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Mon Feb 12 22:06:25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/f1bbad93368ba33bfdde5b5852b333d2f5547fa5

commit f1bbad93368ba33bfdde5b5852b333d2f5547fa5
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Feb 13 21:40:00 2018

portage_util.py: Don't skip uprev if files/ was changed

Also added a unit test to catch this error. This is a bug fix for the
change that landed in CL:844159.

BUG=chromium:794694,b:72809378
TEST=./cbuildbot/run_tests

Change-Id: Ibb034bb14a08c3cbf3ace41ce65f2a6ab9442e75
Reviewed-on: https://chromium-review.googlesource.com/912467
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>

[modify] https://crrev.com/f1bbad93368ba33bfdde5b5852b333d2f5547fa5/lib/portage_util_unittest.py
[modify] https://crrev.com/f1bbad93368ba33bfdde5b5852b333d2f5547fa5/lib/portage_util.py

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/5a8a1b3b4119555aa4bb1895c3000896a42945d5

commit 5a8a1b3b4119555aa4bb1895c3000896a42945d5
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Mar 09 22:30:28 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/5cf2bbb0e090ad907cff40db71a3fb046137ae76

commit 5cf2bbb0e090ad907cff40db71a3fb046137ae76
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Tue Mar 13 22:52:49 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/d4e04d94255504249fe8e5e7b3df6d8aed0361aa

commit d4e04d94255504249fe8e5e7b3df6d8aed0361aa
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Mar 16 02:37:50 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/7d668fdd27fddfd713cd205821f09acd294b7be5

commit 7d668fdd27fddfd713cd205821f09acd294b7be5
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Mar 16 02:37:52 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/6cf72f3ca6d8db3b2f28a545bc6219b931d34f6b

commit 6cf72f3ca6d8db3b2f28a545bc6219b931d34f6b
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Mar 16 02:40:54 2018