New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 711615 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

factory: A better and unified factory resource management system.

Project Member Reported by hungte@chromium.org, Apr 14 2017

Issue description

Currently we have the factory bundle stored in several different ways:
 - Mini-omaha uses a 'memento image' format
   - files are stored as gzipped
   - rootfs & kernel are packed together
   - other partitions are gzipped
 - RMA store images as-is using partitions on USB image
 - Copy machine images use the installed layout
 - Mutliple-RMA uses "compressed files on stateful partition"
 - Umpire used a format close to mini-omaha with different meta data

This is making installer hard to maintain, packaing script (make_factory_package) complicated, also makes it difficult for pytests to download individual resource to update (for example, we can't update release rootfs & kernel).

The goal is to create an unified "cros payload" system that can unify the way we store data with a simplified installed that both pytests, RMA, and factory installer (even netboot) can share.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Apr 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/a326f97446481e42ffe1f600ba5674f45041f8f9

commit a326f97446481e42ffe1f600ba5674f45041f8f9
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Apr 21 09:34:43 2017

init: Revise factory_utils.

Clean up factory_utils lint warnings.
Also updated is_factory_installer_mode by checking kernel boot
command line, which is the right way to detect a factory installer
image.

BUG= chromium:711615 
TEST=build_image test; manual install toolkit and boots properly.
     build_image factory_install; manually boots successfully.

Change-Id: I78396a5a5b3d303205bca3771ef51f344a0d19af
Reviewed-on: https://chromium-review.googlesource.com/479974
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/a326f97446481e42ffe1f600ba5674f45041f8f9/init/factory_utils.sh

Project Member

Comment 2 by bugdroid1@chromium.org, Apr 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/f143b6d3bb22977d27036da97dce4b290b3d7ce2

commit f143b6d3bb22977d27036da97dce4b290b3d7ce2
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Apr 21 22:45:39 2017

init: factory-init: Support new cros_payloads system.

To unify factory installation on different scenario (RMA, Shim+Network,
Netboot, Copy Machine, ...), we want to make sure the provisioned
resources will be installed and updated in same way.

The new cros_payloads provides us a way to install resources, even
before toolkit is installed. And the installation must be executed
before checking factory mode because we may install toolkit (which will
enable factory mode then) inside cros_payloads.

BUG= chromium:711615 
TEST=build_image test;
     manually create /mnt/stateful_partition/cros_payloads/install/test.sh
     and see it executed by /mnt/stateful_partition/cros_payloads/install/log.

Change-Id: Idc32a2efc8cfe3b5215febe8cdf43e5cc3c881e2
Reviewed-on: https://chromium-review.googlesource.com/479975
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/f143b6d3bb22977d27036da97dce4b290b3d7ce2/init/upstart/test-init/factory-init.conf
[modify] https://crrev.com/f143b6d3bb22977d27036da97dce4b290b3d7ce2/init/factory_utils.sh

Project Member

Comment 3 by bugdroid1@chromium.org, Apr 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/c772e1aea534532c1cb07330ee0ce6259d89e247

commit c772e1aea534532c1cb07330ee0ce6259d89e247
Author: Hung-Te Lin <hungte@chromium.org>
Date: Mon Apr 24 15:17:54 2017

setup: Add 'pygpt.py' utility.

To simplify RMA image creation in future, we want to store resources on
its stateful partition and the most simple (and correct) way is to allow
enlarging RMA USB image.

To do that, we need an utility to correct GPT after an image file is
resized. Unfortunately, there's no existing widely available tool can do
that (cgpt is very close, but it has some bug and hard to bundle).

As a result, here's the new tool made from scratch and dedicated for
factory flow to use.

BUG= chromium:711615 
TEST=truncate -s $((3*1048576*1024)) factory_install.bin
     cgpt show factory_install.bin
     pygpt repair --expand factory_install.bin
     pyget show factory_install.bin
     cgpt show factory_install.bin
     ./pygpt_unittest.py

Change-Id: I1d50def371404c4badf32b6f35e3947bf9b307a7
Reviewed-on: https://chromium-review.googlesource.com/477814
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[add] https://crrev.com/c772e1aea534532c1cb07330ee0ce6259d89e247/setup/pygpt
[add] https://crrev.com/c772e1aea534532c1cb07330ee0ce6259d89e247/py/tools/pygpt_unittest.py
[add] https://crrev.com/c772e1aea534532c1cb07330ee0ce6259d89e247/py/tools/pygpt.py
[add] https://crrev.com/c772e1aea534532c1cb07330ee0ce6259d89e247/bin/pygpt

Project Member

Comment 4 by bugdroid1@chromium.org, Apr 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/1dc33c87de16ca9f8760c76968285858cf869254

commit 1dc33c87de16ca9f8760c76968285858cf869254
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Apr 27 07:18:18 2017

sh: Add a new "cros_payload" to manage resources for installers.

For better managing how factory resource will be installed, we need an
unified system for different installation scenario (RMA-USB,
Shim/Network(Omaha), Netboot, Copy-Machine, ...).

The new 'cros_payload' defined a new way to keep storage for
installation of disk and file resources.

To use this:

 1. Create a JSON config file with only '{}' in resources folder.
 2. cros_payload add board.json firmware ~/Downloads/chromeos-firmwareupdate
 3. cros_payload add board.json release_image ~/Downloads/*recovery*.bin
 4. cros_payload add board.json test_image ~/Downloads/*test_image*.bin

Check board.json which should have reference to gzipped contents.

 5. Get a empty USB stick, say /dev/sde, and format properly:
 6. cros_payload install board.json /dev/sde release_image
 7. cros_payload install board.json /dev/sde test_image

Performance (tested on z840):

time ./cros_payload add src/t.json release_image \
  ~/Downloads/chromeos_8198.0.0_chell_recovery_dev-channel_mp.bin

INFO: Adding component release_image part 1 (18M)...
INFO: Adding component release_image part 2 (16M)...
INFO: Adding component release_image part 3 (1290M)...
...

real    0m7.262s
user    1m38.181s
sys     0m12.961s

time ./cros_payload add src/t.json test_image \
  ~/Downloads/chromiumos_test_image.bin

INFO: Adding component test_image part 1 (1216M)...
INFO: Adding component test_image part 2 (16M)...
INFO: Adding component test_image part 3 (2000M)...
...

real    0m16.571s
user    3m25.217s
sys     0m31.313s

BUG= chromium:711615 
TEST=None

Change-Id: Iaf8b401eec6f27c21abb7306ff6e28e292cf6b06
Reviewed-on: https://chromium-review.googlesource.com/463127
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[add] https://crrev.com/1dc33c87de16ca9f8760c76968285858cf869254/setup/cros_payload
[add] https://crrev.com/1dc33c87de16ca9f8760c76968285858cf869254/sh/cros_payload.sh
[add] https://crrev.com/1dc33c87de16ca9f8760c76968285858cf869254/bin/cros_payload

Project Member

Comment 5 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/15da66227063a3a3f986c40effbaa4a722198cf2

commit 15da66227063a3a3f986c40effbaa4a722198cf2
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 04 08:41:24 2017

sh: cros_payload: Fix install_file output folder.

When creating installation stub, the file path should be based on
out_dir instead of output (which is the path for JSON file).

BUG= chromium:711615 
TEST=cros_payload install <BLOCK_DEVICE> hwid toolkit

Change-Id: Id2ac392e236764d38b86429622be401f0c78f3ed
Reviewed-on: https://chromium-review.googlesource.com/495026
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>
Commit-Queue: Hung-Te Lin <hungte@chromium.org>
Trybot-Ready: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/15da66227063a3a3f986c40effbaa4a722198cf2/sh/cros_payload.sh

Project Member

Comment 6 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/a0c1b9e40e14ad156caba186988619b2e2f1979e

commit a0c1b9e40e14ad156caba186988619b2e2f1979e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 04 13:07:02 2017

sh: cros_payload: Do not preserve mtime when creating gzipped payloads.

'gzip' by default stores both original file name and mtime, which will generate
different gzipped payload files when we re-process same files.
So we should add '-n' to ignore original file name and mtime.

'pigz' has different arguments. Its -n only controls file name so we have to
add -T for mtime.

BUG= chromium:711615 
TEST=cros_payload add release_image IMAGE;
     cros_payload add release_image IMAGE;
     # No extra files generated.

Change-Id: I4c4b5849f378499ed90c14bb219319906bad14d6
Reviewed-on: https://chromium-review.googlesource.com/495986
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/a0c1b9e40e14ad156caba186988619b2e2f1979e/sh/cros_payload.sh

Project Member

Comment 7 by bugdroid1@chromium.org, May 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/7b8d62abbdd82b3f53e17fc30bda445b37f605e9

commit 7b8d62abbdd82b3f53e17fc30bda445b37f605e9
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 05 12:41:37 2017

Makefile: Create a 'installer.tar' resource to share cros_payload and cutoff.

To make it easier sharing files with factory_installer, we want to
prepare the resource archive in standard build procedure.

BUG= chromium:711615 
TEST=make resource; tar tvf build/resource/installer.tar

Change-Id: Ic82c20fcffbf124c0e344a70870eeaaa645d9d47
Reviewed-on: https://chromium-review.googlesource.com/496227
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/7b8d62abbdd82b3f53e17fc30bda445b37f605e9/Makefile

Project Member

Comment 8 by bugdroid1@chromium.org, May 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/51408b1072be37fd0299d21691c509431c5f5e18

commit 51408b1072be37fd0299d21691c509431c5f5e18
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 05 14:54:21 2017

factory_installer: Use installer resource from chromeos-base/factory.

The cros_payload also has to be shared between factory installer and
factory toolkit, so we have to put it into a shared resource for
factory_installer. The resource is now renamed from "cutoff" to
"installer" because it will contain more non-cutoff resource in future.

BUG= chromium:711615 
TEST=emerge-eve factory factory_installer
CQ-DEPEND=CL:496227

Change-Id: I03f4d31d7a9a96af4080b2fb966d296673dffb56
Reviewed-on: https://chromium-review.googlesource.com/495948
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/51408b1072be37fd0299d21691c509431c5f5e18/chromeos-base/factory_installer/factory_installer-9999.ebuild
[modify] https://crrev.com/51408b1072be37fd0299d21691c509431c5f5e18/chromeos-base/factory/factory-9999.ebuild

Project Member

Comment 9 by bugdroid1@chromium.org, May 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/f6831ea389f73bf99b90ecf545f2eec8117f4a43

commit f6831ea389f73bf99b90ecf545f2eec8117f4a43
Author: Hung-Te Lin <hungte@chromium.org>
Date: Mon May 08 17:14:10 2017

sh: cros_payloads: Fix execution failure by factory_install.sh.

This is a collection of fixes for real factory_install.sh:

- Add 'complete' component for Mini-omaha and Umpire to execute after
  installation complete.

- Fix toolkit installer stub (need -- --yes)

- 'gzip -d FILE' should do add '-f' because we may re-install payloads.

- 'umount' in cleanup should never stop on failure because that will
  change exit value.

- 'rm' in cleanup should do 'rm -rf' in so both directory and file
  objects won't fail.

BUG= chromium:711615 
TEST=cros_payload add complete FILE
     cros_payload install URL DRIVE toolkit

Change-Id: I82e9ffa2b213dd6c655c647685fcdb14816bccf5
Reviewed-on: https://chromium-review.googlesource.com/497238
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/f6831ea389f73bf99b90ecf545f2eec8117f4a43/sh/cros_payload.sh

Project Member

Comment 10 by bugdroid1@chromium.org, May 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/a984175250889b1b9484c92b7bedab23b5259b56

commit a984175250889b1b9484c92b7bedab23b5259b56
Author: Hung-Te Lin <hungte@chromium.org>
Date: Mon May 08 19:40:59 2017

setup: make_factory_package: Add --cros_payload to support the new format.

The new --cros_payload will create static folder using cros_payload.
This change is the first step and also a demonstration for how easy the
resources for download can be prepared using the new cros_payload.

BUG= chromium:711615 
TEST=./make_factory_package --cros_payload --release RELEASE --test TEST \
      --factory_toolkit TOOLKIT --hwid HWID
     ./make_factory_package --cros_payload --release RELEASE --test TEST \
      --factory_toolkit TOOLKIT --hwid HWID --diskimg diskimg.bin
     ./make_factory_package --cros_payload --release RELEASE --test TEST \
      --factory_toolkit TOOLKIT --hwid HWID --install INSTALL_SHIM \
      --usbimg usbimg.bin

Change-Id: I16e43d51eab814abbe0d670f8125efdf63fa3e8f
Reviewed-on: https://chromium-review.googlesource.com/476950
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/a984175250889b1b9484c92b7bedab23b5259b56/setup/make_factory_package.sh
[modify] https://crrev.com/a984175250889b1b9484c92b7bedab23b5259b56/setup/lib/cros_image_common.sh

Project Member

Comment 11 by bugdroid1@chromium.org, May 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/aefbfe95c97a9033bc6a77d3ffe1fdd4ba76135b

commit aefbfe95c97a9033bc6a77d3ffe1fdd4ba76135b
Author: Hung-Te Lin <hungte@chromium.org>
Date: Mon May 08 19:41:02 2017

chromeos-base/factory_installer: Add app-misc/jq.

The new cros_payload needs jq to process payload configuration when
Python is not available.

BUG= chromium:711615 
TEST=build_image factory_install

Change-Id: I9aedec01588b922814e65fa9d16ab2aae3ef7d79
Reviewed-on: https://chromium-review.googlesource.com/497154
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/aefbfe95c97a9033bc6a77d3ffe1fdd4ba76135b/chromeos-base/factory_installer/factory_installer-9999.ebuild

Project Member

Comment 12 by bugdroid1@chromium.org, May 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/d09436e936666ac3ada9710c5b5f9c0c77cf60e3

commit d09436e936666ac3ada9710c5b5f9c0c77cf60e3
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue May 09 21:58:30 2017

setup: make_factory_package: Revise and remove deprecated params.

To align with bundle folder and component names,

- The --factory should be deprecated by --test_image and --toolkit.
- The --release and --test should be renamed to --release_image and
  --test_image.
- The --firmware_updater and --hwid_updater should be renamed to
  --firmware and --hwid.
- The --factory_toolkit should be renamed to --toolkit.

BUG= chromium:711615 
TEST=./make_factory_package.sh --test TEST --toolkit TOOLKIT --release RELEASE \
     --hwid none --firmware none

Change-Id: I17c7ee29239df140d7e88cfe802f3650aaf7c0f7
Reviewed-on: https://chromium-review.googlesource.com/477772
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/d09436e936666ac3ada9710c5b5f9c0c77cf60e3/setup/make_factory_package.sh
[modify] https://crrev.com/d09436e936666ac3ada9710c5b5f9c0c77cf60e3/setup/get_recovery_image.py
[modify] https://crrev.com/d09436e936666ac3ada9710c5b5f9c0c77cf60e3/py/factory_flow/usb_install.py

Project Member

Comment 13 by bugdroid1@chromium.org, May 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/0a8d601c19c9306adb46a448d8017b0a8aacbabc

commit 0a8d601c19c9306adb46a448d8017b0a8aacbabc
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue May 09 21:58:29 2017

setup: make_factory_package: Add USE_CROS_PAYLOAD for RMA images (--usbimg).

The factory_installer package needs to know if the RMA was built with
--cros_payload by checking USE_CROS_PAYLOAD in lsb-factory file.

BUG= chromium:711615 
TEST=make_factory_package --cros_payload --release RELEASE --test TEST \
      --hwid HWID --install INSTALL --usbimg rma.bin

Change-Id: Ic172fda12062cebdd148abc50d15b2f80824b1e8
Reviewed-on: https://chromium-review.googlesource.com/499810
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/0a8d601c19c9306adb46a448d8017b0a8aacbabc/setup/make_factory_package.sh

Project Member

Comment 14 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/0704357f48ee307bd94a7a783759556fee835f41

commit 0704357f48ee307bd94a7a783759556fee835f41
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 09:08:28 2017

cros_payload: Add 'download' command for special resources.

To preserve content from release image stateful partition (for example,
pre-cached extensions) we want a new command to download arbitrary
resources (for instance, release_image.part1).

This is different from 'install' command because even if dest is a block
device, we still have to save it as file instead of overwriting the
block device.

BUG= chromium:711615 
TEST=cros_payload download URL . release_image.part1

Change-Id: I1f2f40ea44b0e5191d92c598f6f40bddd608c6c6
Reviewed-on: https://chromium-review.googlesource.com/499811
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/0704357f48ee307bd94a7a783759556fee835f41/sh/cros_payload.sh

Project Member

Comment 15 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/cc2473120aeaf613d8200adea88114ad815cabe0

commit cc2473120aeaf613d8200adea88114ad815cabe0
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 09:08:28 2017

cros_payload: Change file components to store an object with file attribute.

We will want to store more attributes, for example version number, to
all components. As a result, file components should also store the files
like {"file": FILE} instead of pure string.

For backward compatibility and to reduce processing effort, the trick is
done in json_get_file_value, that a descriptor pointing to a dict/object
will be resolved as descriptor.file.

BUG= chromium:711615 
TEST=cros_payload add JSON toolkit TOOLKIT

Change-Id: I16d86b1c48842942c8b6ae61521dca3e3196ee07
Reviewed-on: https://chromium-review.googlesource.com/499832
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>
Reviewed-by: Ting Shen <phoenixshen@chromium.org>

[modify] https://crrev.com/cc2473120aeaf613d8200adea88114ad815cabe0/sh/cros_payload.sh

Project Member

Comment 16 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/initramfs/+/b85c6992c6e3b182fa9d632587332999e9ec25a5

commit b85c6992c6e3b182fa9d632587332999e9ec25a5
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 14:08:28 2017

factory_netboot: Add cros_payload and new factory_installer files.

The factory installer has been updated with new factory_* scripts and
using cros_payload (need jq, curl), so we do want to include them in
netboot shim.

BUG= chromium:711615 
TEST=./make_netboot.sh --board eve

Change-Id: I7f9c0f75598b9d76126d67c259e6658eb44f13eb
Reviewed-on: https://chromium-review.googlesource.com/501687
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/b85c6992c6e3b182fa9d632587332999e9ec25a5/factory_netboot/Makefile
[modify] https://crrev.com/b85c6992c6e3b182fa9d632587332999e9ec25a5/factory_netboot/init

Project Member

Comment 17 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/initramfs/+/363f58be64023da920f1aaac40fc552f8a46107f

commit 363f58be64023da920f1aaac40fc552f8a46107f
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 14:08:27 2017

factory_shim: Change log file name.

The factory_shim.log is taken by factory_install/factory_shim_service.sh,
which will be a problem in netboot environment that /var/log is a
symlink to /log.

BUG= chromium:711615 
TEST=build_image factory_install

Change-Id: I2d5f62ccb0c0f3e4a0b6ac73fe9e38cc761c2def
Reviewed-on: https://chromium-review.googlesource.com/501747
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/363f58be64023da920f1aaac40fc552f8a46107f/factory_shim/bootstrap.sh

Project Member

Comment 18 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/initramfs/+/363f58be64023da920f1aaac40fc552f8a46107f

commit 363f58be64023da920f1aaac40fc552f8a46107f
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 14:08:27 2017

factory_shim: Change log file name.

The factory_shim.log is taken by factory_install/factory_shim_service.sh,
which will be a problem in netboot environment that /var/log is a
symlink to /log.

BUG= chromium:711615 
TEST=build_image factory_install

Change-Id: I2d5f62ccb0c0f3e4a0b6ac73fe9e38cc761c2def
Reviewed-on: https://chromium-review.googlesource.com/501747
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/363f58be64023da920f1aaac40fc552f8a46107f/factory_shim/bootstrap.sh

Project Member

Comment 19 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/c0682734df8945b1667a56b585fd522cf87604ab

commit c0682734df8945b1667a56b585fd522cf87604ab
Author: You-Cheng Syu <youcheng@chromium.org>
Date: Wed May 10 14:08:29 2017

installer: put toolkit version into toolkit installer lsm file

This change make users able to get the content of TOOLKIT_VERSION by
running `./install_factory_toolkit.run --lsm`, which is much faster and
easier than decompressing the whole archive.

BUG= chromium:711615 
TEST=1. make toolkit BOARD=xxx
     2. cd /path/to/factory/build/
     3. ./install_factory_toolkit.run --lsm
     4. ./install_factory_toolkit.run --noexec --target z
     5. ./install_factory_toolkit.run -- --repack z \
                                         --pack-into a.run
     6. ./a.run --lsm

Change-Id: I293f2ce470d6d8b2bc923bfcb5e8dd32dce320db
Reviewed-on: https://chromium-review.googlesource.com/501667
Commit-Ready: Youcheng Syu <youcheng@google.com>
Tested-by: Youcheng Syu <youcheng@google.com>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/c0682734df8945b1667a56b585fd522cf87604ab/py/toolkit/installer.py

Project Member

Comment 20 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/a33adac60b3f5d70a0af3af266264c42db4f7213

commit a33adac60b3f5d70a0af3af266264c42db4f7213
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 18:58:12 2017

Change factory_install{.conf,_service.sh} to factory_shim*.

The factory_install is actually a combination of "installer, reset shim
and RMA stub". To make it structured better (and revise the installer
later), we want to change the upstart related file names to
"factory_shim" instead of "factory_install".

BUG= chromium:711615 
TEST=build_image factory_install
CQ-DEPEND=CL:497112

Change-Id: Id57a4d722b5e0175df1b827343859e1938ed7c9a
Reviewed-on: https://chromium-review.googlesource.com/497369
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[rename] https://crrev.com/a33adac60b3f5d70a0af3af266264c42db4f7213/factory_shim_service.sh
[rename] https://crrev.com/a33adac60b3f5d70a0af3af266264c42db4f7213/init/factory_shim.conf

Project Member

Comment 21 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/aa9dc87d34b27d7dc0c7031f7dc19037d8517704

commit aa9dc87d34b27d7dc0c7031f7dc19037d8517704
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 18:58:12 2017

chromeos-base/factory_installer: Install upstart conf from init/ folder.

The upstart files should have its own folder and also prevent hard-coded
file names.

BUG= chromium:711615 
TEST=build_image factory_install
CQ-DEPEND=CL:497369,CL:501687

Change-Id: I619ddf29b7f661818ab1436e74083e7a6520937a
Reviewed-on: https://chromium-review.googlesource.com/497112
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/aa9dc87d34b27d7dc0c7031f7dc19037d8517704/chromeos-base/factory_installer/factory_installer-9999.ebuild

Project Member

Comment 22 by bugdroid1@chromium.org, May 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/26766827482516328ce23029771c8cfb8e653609

commit 26766827482516328ce23029771c8cfb8e653609
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 10 18:58:12 2017

chromeos-base/factory_installer, factory-deps: Add pigz into RDEPEND.

The new cros_payload can download and run faster if pigz is available.

BUG= chromium:711615 
TEST=build_image factory_install

Change-Id: I16e85721a57ad85670232952c0105c3ef0985aa4
Reviewed-on: https://chromium-review.googlesource.com/501548
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/26766827482516328ce23029771c8cfb8e653609/chromeos-base/factory-deps/factory-deps-1.ebuild
[rename] https://crrev.com/26766827482516328ce23029771c8cfb8e653609/chromeos-base/factory-deps/factory-deps-1-r3.ebuild
[modify] https://crrev.com/26766827482516328ce23029771c8cfb8e653609/chromeos-base/factory_installer/factory_installer-9999.ebuild

Project Member

Comment 23 by bugdroid1@chromium.org, May 11 2017

Project Member

Comment 24 by bugdroid1@chromium.org, May 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/a8ae38c41ef88a2d04c90883e4fdb08d1328aeb5

commit a8ae38c41ef88a2d04c90883e4fdb08d1328aeb5
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 11 06:17:45 2017

factory_install: Move legacy Omaha and USB install to a standalone file.

Before the transition to new cros_payload, we'd like to move old Omaha
and USB installation logic into a standalone file.

BUG= chromium:711615 
TEST=build_image factory_install; boot

Change-Id: Idea0286366b5265183e1a87e09a96eb01a810976
Reviewed-on: https://chromium-review.googlesource.com/498948
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[add] https://crrev.com/a8ae38c41ef88a2d04c90883e4fdb08d1328aeb5/factory_install_legacy.sh
[modify] https://crrev.com/a8ae38c41ef88a2d04c90883e4fdb08d1328aeb5/factory_install.sh

Project Member

Comment 25 by bugdroid1@chromium.org, May 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/c1ae92538a9c04966b19e00218eb202af36ad65e

commit c1ae92538a9c04966b19e00218eb202af36ad65e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 11 16:40:25 2017

lbzip2: upgraded package to upstream

Upgraded app-arch/lbzip2 to version 2.5 on amd64

The ChromeOS factory software imaging process is looking for a new
compression tool for storing and transfer big payloads. lbzip2 is known
to run faster with better compression ratio that pbzip2, less dependency
(no libstdc++), and is more widely available on some Linux distros.
lbzip2 is also directly supported by GNU tar when bzip2 is not found.

As a result, we should add lbzip2 into overlay.

BUG= chromium:711615 
TEST=sudo emerge lbzip2

Change-Id: I237f51b9323c3db7f5fd78e24a7f13874847420c
Reviewed-on: https://chromium-review.googlesource.com/501729
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/c1ae92538a9c04966b19e00218eb202af36ad65e/metadata/md5-cache/app-arch/lbzip2-2.5
[add] https://crrev.com/c1ae92538a9c04966b19e00218eb202af36ad65e/app-arch/lbzip2/Manifest
[add] https://crrev.com/c1ae92538a9c04966b19e00218eb202af36ad65e/app-arch/lbzip2/files/lbzip2-2.3-s_isreg.patch
[add] https://crrev.com/c1ae92538a9c04966b19e00218eb202af36ad65e/app-arch/lbzip2/lbzip2-2.5.ebuild
[add] https://crrev.com/c1ae92538a9c04966b19e00218eb202af36ad65e/app-arch/lbzip2/metadata.xml

Project Member

Comment 27 by bugdroid1@chromium.org, May 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/7af267ca7438053b0fbb5ae9efbbf3d2069a34b1

commit 7af267ca7438053b0fbb5ae9efbbf3d2069a34b1
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 11 20:19:39 2017

cros_payload: Support additional compression formats.

'gzip' is known to be fastest with the cost of larger output size. For
some environment, people may prefer other compression formats like bzip2
for less network transfer cost.

The new CROS_PAYLOAD_FORMAT allows overriding the compression format to be
used when adding new component.  The decompression will always select
right format according to the name stored in JSON config.

The 'install' and 'download' commands have been changed to that install
will always decompress, while download will keep files compressed.

BUG= chromium:711615 
TEST= CROS_PAYLOAD_FORMAT=bz2 ./make_factory_package.sh ...

Change-Id: I405c51f39c892cf924c4af623cff45e68e392959
Reviewed-on: https://chromium-review.googlesource.com/502127
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/7af267ca7438053b0fbb5ae9efbbf3d2069a34b1/sh/cros_payload.sh

Project Member

Comment 28 by bugdroid1@chromium.org, May 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/b2c399582dcae8af95504bc4fb1df13d712beb48

commit b2c399582dcae8af95504bc4fb1df13d712beb48
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu May 11 20:19:39 2017

cros_payload: Revise and style nits.

Revised few comments.
Changed file name pattern from 'a_b' to 'a.b' by review feedback.
Removed unused return value.

BUG= chromium:711615 
TEST=cros_payload add JSON COMPONENT FILE

Change-Id: Ib687956696e7de470d561f536734a75d6b9334a5
Reviewed-on: https://chromium-review.googlesource.com/502132
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/b2c399582dcae8af95504bc4fb1df13d712beb48/sh/cros_payload.sh

Project Member

Comment 29 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/initramfs/+/a878c6922db5fc5c2565b4058f1f4713ee5115aa

commit a878c6922db5fc5c2565b4058f1f4713ee5115aa
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 05:27:46 2017

factory_netboot: Add pigz and lbzip2.

The factory netboot installer needs to download and extract files in gz
or bzip2 format. Adding pigz and lbzip2 so we can utilize CPU cores for
better imaging speed.

BUG= chromium:711615 
TEST=src/scripts/make_netboot.sh --board eve

Change-Id: Icafbfe10a009f4dd84edcd675e9b2482c7cc5dc9
Reviewed-on: https://chromium-review.googlesource.com/502767
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/a878c6922db5fc5c2565b4058f1f4713ee5115aa/factory_netboot/Makefile

Project Member

Comment 30 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/3dc351457b0a34c8d80dd86d4310671c7444e46f

commit 3dc351457b0a34c8d80dd86d4310671c7444e46f
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 10:25:30 2017

setup: make_factory_package: Deprecate legacy style RMA and disk image.

The factory_installer now supports cros_payload style USB installation
if USE_CROS_PAYLOAD is set. We are ready to deprecate memento-update
style images.

BUG= chromium:711615 
TEST=./make_factory_package.sh --release RELEASE --test TEST --hwid HWID \
     --usbimg usb.bin

Change-Id: I8fe7b26e057665989d6d2be824f931cbaa4bb359
Reviewed-on: https://chromium-review.googlesource.com/499812
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/3dc351457b0a34c8d80dd86d4310671c7444e46f/setup/make_factory_package.sh
[modify] https://crrev.com/3dc351457b0a34c8d80dd86d4310671c7444e46f/setup/lib/cros_image_common.sh

Project Member

Comment 31 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/902eef7ce010fcb89eec27011d218e9780c5189e

commit 902eef7ce010fcb89eec27011d218e9780c5189e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 10:25:30 2017

setup: Change merge_factory_usbimg to work with cros_payload.

Now RMA is always using cros_payload so we should be able to migrate the
multi-RMA image.

BUG= chromium:711615 
TEST=merge_factory_usbimg.sh out.bin PATH_TO_USBIMG1 PATH_TO_USBIMG2

Change-Id: I246f66212788dec9d5671a824c7c1f3c866e5807
Reviewed-on: https://chromium-review.googlesource.com/501693
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>

[modify] https://crrev.com/902eef7ce010fcb89eec27011d218e9780c5189e/setup/factory_common.sh
[delete] https://crrev.com/3dc351457b0a34c8d80dd86d4310671c7444e46f/setup/lib/compress_cros_image.sh
[delete] https://crrev.com/3dc351457b0a34c8d80dd86d4310671c7444e46f/setup/mk_memento_images.sh
[delete] https://crrev.com/3dc351457b0a34c8d80dd86d4310671c7444e46f/setup/make_image_resource.sh
[modify] https://crrev.com/902eef7ce010fcb89eec27011d218e9780c5189e/py/umpire/commands/update_unittest.py
[modify] https://crrev.com/902eef7ce010fcb89eec27011d218e9780c5189e/setup/merge_factory_usbimg.sh

Project Member

Comment 32 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/8c608c83f1f8fe9f27123f56ea1a8b083efe96a8

commit 8c608c83f1f8fe9f27123f56ea1a8b083efe96a8
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 10:25:38 2017

factory_install: Correct post install file processing.

The cros_payload command has been changed to support multiple
compression formats and defined 'download' is compressed while 'install'
is uncompressed.

So we have to follow same calling convention in factory installer.

- firmware and complete are already uncompressed in 'cros_payload install'.
- release_image.part1 is compressed so we have to decompress first.

BUG= chromium:711615 
TEST=Boots factory install shim and finished installation.

Change-Id: Icf322c64993cfadcb1f3c543be2376081110b409
Reviewed-on: https://chromium-review.googlesource.com/503128
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/8c608c83f1f8fe9f27123f56ea1a8b083efe96a8/factory_install.sh

Project Member

Comment 33 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/aa25bc424465d620ee29c2780aecee748430b77c

commit aa25bc424465d620ee29c2780aecee748430b77c
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 12:28:37 2017

cros_payload: Speed up partition dd execution time for 'add' commands.

Usually dd runs slow if bs is small, especially for large files. When
adding image components, we should try to increase bs if possible.
A boundary (32M) is set to prevent dd allocating too much memory buffer.

This has reduced a make_factory_package execution time from 30s to 22.8s
on z840.

BUG= chromium:711615 
TEST=time DEBUG=1 ./make_factory_package --release RELEASE --test TEST \
     --toolkit TOOLKIT --hwid none --cros_payload
     time DEBUG=1 dash ./cros_payload add static/eve.json test_image IMAGE

Change-Id: I20e7bab3fe781c544bdef6fe8dd6e228f4cd7f98
Reviewed-on: https://chromium-review.googlesource.com/503831
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/aa25bc424465d620ee29c2780aecee748430b77c/sh/cros_payload.sh

Project Member

Comment 34 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/45860a76b8dba7cab308b1d262526883b81c463e

commit 45860a76b8dba7cab308b1d262526883b81c463e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 12:28:36 2017

cros_payload: Support installing one partition.

The 'install' has some implicit rule for the type of source and
destination. However, sometimes we may want to do some explicit
installation, for example:

 - Only re-installing test image kernel to a kernel partition, or to a blob.
 - Only re-installing release image stateful partition, or to a blob.

As a result, we want install to accept component.partX for partition
installation.

Also revised help message and TODO list to reflect the new changes.

BUG= chromium:711615 
TEST=./cros_payload install eve.json /dev/mmcblk0p1 release_image.part1

Change-Id: I53b9df72796cb4b9e5de6d036157decdace915eb
Reviewed-on: https://chromium-review.googlesource.com/503908
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/45860a76b8dba7cab308b1d262526883b81c463e/sh/cros_payload.sh

Project Member

Comment 35 by bugdroid1@chromium.org, May 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/c988c18f4f53856b2fa3076dee668ae65e1e9dd9

commit c988c18f4f53856b2fa3076dee668ae65e1e9dd9
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri May 12 12:28:36 2017

cros_payload: New command 'add_meta' to include arbitrary properties.

To support resources that version can't be easily retrieved, we are
adding a new "add_meta" command so other scripts can use it to specify
component meta data without having that logic merged into cros_payload.

BUG= chromium:711615 
TEST=./cros_payload add_meta test.json test_image blah blah_value

Change-Id: I3c8ac95f802f933acc59089a673037771934cc41
Reviewed-on: https://chromium-review.googlesource.com/503909
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/c988c18f4f53856b2fa3076dee668ae65e1e9dd9/sh/cros_payload.sh

Status: Fixed (was: Started)
All changes merged.
Project Member

Comment 37 by bugdroid1@chromium.org, May 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/129aa51722ff77f984c19141c03f4b32c1676bc2

commit 129aa51722ff77f984c19141c03f4b32c1676bc2
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue May 23 17:24:04 2017

setup: Fix hang in make_factory_package --diskimg.

make_factory_package should free loop device when updating to
stateful partition was completed.

BUG= chromium:711615 
TEST=make_factory_package --diskimg [params]

Change-Id: I4190ece57449ab165809ea5179a8c6493dfdddd0
Reviewed-on: https://chromium-review.googlesource.com/511788
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/129aa51722ff77f984c19141c03f4b32c1676bc2/setup/make_factory_package.sh

Project Member

Comment 38 by bugdroid1@chromium.org, May 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/6556d6f7cd3f314afe792ff3e556c52ebca0f35f

commit 6556d6f7cd3f314afe792ff3e556c52ebca0f35f
Author: Hung-Te Lin <hungte@chromium.org>
Date: Wed May 31 09:15:26 2017

setup: Add add release_image.part1 in --diskimg.

To preserve contents from stateful partition of release image (usually
cache of Chrome extensions in unencrypted folder) we have to download
release_image.part1 in --diskimg mode.  (The payload will be downloaded
by factory_install when in --usbimg or network mode).

BUG= chromium:711615 
TEST=./make_factory_package.sh --diskimg [params]

Change-Id: I77faebacb43cde06e97b7415ef034fa4a2de4a90
Reviewed-on: https://chromium-review.googlesource.com/512424
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[modify] https://crrev.com/6556d6f7cd3f314afe792ff3e556c52ebca0f35f/setup/make_factory_package.sh

Project Member

Comment 39 by bugdroid1@chromium.org, Jun 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/052f5e81f410448a26aa9ca69606f14026a62fe9

commit 052f5e81f410448a26aa9ca69606f14026a62fe9
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Jun 01 04:14:38 2017

factory_install: Remove legacy Memento Updater implementation.

The factory setup program has been changed to new cros_payload so we
should deprecate memento_updater.

BUG= chromium:711615 
TEST=build_image factory_install; Boots and installs successfully.

Change-Id: I198200b3901e4569c29420b6a1f27a007458d80d
Reviewed-on: https://chromium-review.googlesource.com/518606
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>

[delete] https://crrev.com/39dc50c15c59193776fa3d9636b0af5755594349/factory_install_legacy.sh
[modify] https://crrev.com/052f5e81f410448a26aa9ca69606f14026a62fe9/factory_reset.sh
[modify] https://crrev.com/052f5e81f410448a26aa9ca69606f14026a62fe9/factory_install.sh

Project Member

Comment 40 by bugdroid1@chromium.org, Jun 2 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/567c53759b574d2ca291f269504c5f7af3e42fde

commit 567c53759b574d2ca291f269504c5f7af3e42fde
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 02 22:11:22 2017

Remove chromeos-base/memento_softwareupdate.

The memento_softwareupdate package has been deprecated by
new cros_payload in factory_install so we should not include it anymore.

BUG= chromium:711615 
TEST=build_packages --board eve;
     build_image --board eve test

Change-Id: I27c70dedadcf630f5fc7a8d78a353c578cf1500b
Reviewed-on: https://chromium-review.googlesource.com/518744
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[delete] https://crrev.com/7d29205c636ebbe53c8a6ffe02d12e9418075fd6/chromeos-base/memento_softwareupdate/memento_softwareupdate-9999.ebuild
[modify] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/chromeos-base/factory-deps/factory-deps-1.ebuild
[modify] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/chromeos-base/chromeos-initramfs/chromeos-initramfs-9999.ebuild
[rename] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/chromeos-base/factory-deps/factory-deps-1-r5.ebuild
[delete] https://crrev.com/7d29205c636ebbe53c8a6ffe02d12e9418075fd6/chromeos-base/memento_softwareupdate/memento_softwareupdate-0.0.1-r75.ebuild
[modify] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/chromeos-base/factory_installer/factory_installer-9999.ebuild
[modify] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/virtual/target-chromium-os-factory/target-chromium-os-factory-1.ebuild
[rename] https://crrev.com/567c53759b574d2ca291f269504c5f7af3e42fde/virtual/target-chromium-os-factory/target-chromium-os-factory-1-r4.ebuild

Project Member

Comment 41 by bugdroid1@chromium.org, Jun 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/44c7545fb169c07fe98483668b6864d2693f0b6b

commit 44c7545fb169c07fe98483668b6864d2693f0b6b
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Jun 29 08:16:16 2017

cros_payload: Support xz compression format.

Add xz compression format to support adding compressed resources in
future.

BUG= chromium:711615 
TEST=cros_payload add JSON hwid PATH_TO_HWID

Change-Id: I5aae3ca7b124cfbfaeb50c29c3b456c2d87681f6
Reviewed-on: https://chromium-review.googlesource.com/552418
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@chromium.org>

[modify] https://crrev.com/44c7545fb169c07fe98483668b6864d2693f0b6b/sh/cros_payload.sh

Project Member

Comment 42 by bugdroid1@chromium.org, Jun 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/a8848b5c9919a1e2298520c3f8062c9a63547909

commit a8848b5c9919a1e2298520c3f8062c9a63547909
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Jun 29 08:16:17 2017

cros_payload: Support compressed payload source.

Most images from GoldenEye or CPFE are compressed, so we should try to
decompress when the uploaded file for updating resource is in compressed
form.

BUG= chromium:711615 
TEST=cros_payload add JSON test_image PATH_TO/test_image.tar.xz

Change-Id: If2d87adeb041291ed63d6db138f9afc63cb6a963
Reviewed-on: https://chromium-review.googlesource.com/552419
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@chromium.org>

[modify] https://crrev.com/a8848b5c9919a1e2298520c3f8062c9a63547909/sh/cros_payload.sh

Project Member

Comment 43 by bugdroid1@chromium.org, Jun 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/e4e895e503eeeed43429b338088c2962c5bed54b

commit e4e895e503eeeed43429b338088c2962c5bed54b
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Jun 29 08:16:17 2017

cros_payload: Support tar compression format.

Test images from buildbot are usually released as tar+xz archive, so we
do want to support tar as well.

BUG= chromium:711615 
TEST=None

Change-Id: I5725d7a5f0eae101c4858b3ffc6aa0f8fc5650bd
Reviewed-on: https://chromium-review.googlesource.com/552478
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@chromium.org>

[modify] https://crrev.com/e4e895e503eeeed43429b338088c2962c5bed54b/sh/cros_payload.sh

Project Member

Comment 44 by bugdroid1@chromium.org, Jun 30 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/d6e010f65aa504181d6815a70a6b4f9a46c0519e

commit d6e010f65aa504181d6815a70a6b4f9a46c0519e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 30 06:41:09 2017

Project Member

Comment 45 by bugdroid1@chromium.org, Jun 30 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/manifest/+/e1870ef38309c5239f673cb576ee5c0de3dfc5f5

commit e1870ef38309c5239f673cb576ee5c0de3dfc5f5
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 30 06:41:09 2017

Remove memento_softwareupdate repo.

The memento_softwareupdate has been deprecated (AU uses update_engine
and factory uses its own installer today) and no more packages are using
it.

BUG= chromium:711615 
TEST=repo sync

Change-Id: I2adea5dff4b838f4d7caa9cf3c30f4c120298588
Reviewed-on: https://chromium-review.googlesource.com/554514
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Bernie Thompson <bhthompson@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/e1870ef38309c5239f673cb576ee5c0de3dfc5f5/full.xml

Project Member

Comment 46 by bugdroid1@chromium.org, Aug 1 2017

Labels: merge-merged-factory-gru-9017.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/abbe731308b65d32cad93cbf68c4ccc5ac452762

commit abbe731308b65d32cad93cbf68c4ccc5ac452762
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:43:44 2017

factory_install: Move legacy Omaha and USB install to a standalone file.

Before the transition to new cros_payload, we'd like to move old Omaha
and USB installation logic into a standalone file.

BUG= chromium:711615 
TEST=build_image factory_install; boot

Change-Id: Idea0286366b5265183e1a87e09a96eb01a810976
Reviewed-on: https://chromium-review.googlesource.com/498948
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
(cherry picked from commit a8ae38c41ef88a2d04c90883e4fdb08d1328aeb5)
Reviewed-on: https://chromium-review.googlesource.com/594694
Reviewed-by: Philip Chen <philipchen@chromium.org>
Commit-Queue: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[add] https://crrev.com/abbe731308b65d32cad93cbf68c4ccc5ac452762/factory_install_legacy.sh
[modify] https://crrev.com/abbe731308b65d32cad93cbf68c4ccc5ac452762/factory_install.sh

Project Member

Comment 47 by bugdroid1@chromium.org, Aug 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/2c4134b62184ef7f93cc3a13736cfc98b7bd7643

commit 2c4134b62184ef7f93cc3a13736cfc98b7bd7643
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:46:19 2017

factory_install: Change installation to support new cros_payload system.

The new cros_payload storage system allows factory install shim to unify
the installation logic, for both Mini-Omaha, RMA, and Multi-Boards RMA.

For USB installation, cros_payload is selected if USE_CROS_PAYLOAD is
set.

For network installation (netboot, mini-Omaha or Umpire), cros_payload
is selected if the ${BOARD}.json can be found in static path.

BUG= chromium:711615 
TEST=build_image factory_install
     Verified Mini-Omaha installation and RMA installation.

Change-Id: I6f44b5b0d8e9bcd223533b5e1849f4fc6db8f76d
Reviewed-on: https://chromium-review.googlesource.com/497240
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
(cherry picked from commit b5f115891b7a2240fd1a89d4e8bafcfe2934ebe8)
Reviewed-on: https://chromium-review.googlesource.com/594695
Reviewed-by: Philip Chen <philipchen@chromium.org>
Commit-Queue: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[modify] https://crrev.com/2c4134b62184ef7f93cc3a13736cfc98b7bd7643/factory_install_legacy.sh
[add] https://crrev.com/2c4134b62184ef7f93cc3a13736cfc98b7bd7643/factory_common.sh
[modify] https://crrev.com/2c4134b62184ef7f93cc3a13736cfc98b7bd7643/factory_install.sh

Project Member

Comment 48 by bugdroid1@chromium.org, Aug 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/03bdadc8ab52c94ada913da74c0fbcd9a2c952f4

commit 03bdadc8ab52c94ada913da74c0fbcd9a2c952f4
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:48:28 2017

factory_install: Correct post install file processing.

The cros_payload command has been changed to support multiple
compression formats and defined 'download' is compressed while 'install'
is uncompressed.

So we have to follow same calling convention in factory installer.

- firmware and complete are already uncompressed in 'cros_payload install'.
- release_image.part1 is compressed so we have to decompress first.

BUG= chromium:711615 
TEST=Boots factory install shim and finished installation.

Change-Id: Icf322c64993cfadcb1f3c543be2376081110b409
Reviewed-on: https://chromium-review.googlesource.com/503128
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>
(cherry picked from commit 8c608c83f1f8fe9f27123f56ea1a8b083efe96a8)
Reviewed-on: https://chromium-review.googlesource.com/594696
Reviewed-by: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[modify] https://crrev.com/03bdadc8ab52c94ada913da74c0fbcd9a2c952f4/factory_install.sh

Project Member

Comment 49 by bugdroid1@chromium.org, Aug 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/5fc380d88440b6cd11c3e26d4da8aba5db168d15

commit 5fc380d88440b6cd11c3e26d4da8aba5db168d15
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:49:31 2017

factory_install: Remove legacy Memento Updater implementation.

The factory setup program has been changed to new cros_payload so we
should deprecate memento_updater.

BUG= chromium:711615 
TEST=build_image factory_install; Boots and installs successfully.

Change-Id: I198200b3901e4569c29420b6a1f27a007458d80d
Reviewed-on: https://chromium-review.googlesource.com/518606
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Youcheng Syu <youcheng@google.com>
(cherry picked from commit 052f5e81f410448a26aa9ca69606f14026a62fe9)
Reviewed-on: https://chromium-review.googlesource.com/594698
Reviewed-by: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[delete] https://crrev.com/23db737fe1ff7c91cdcaaca4fe02b4c4d6059b32/factory_install_legacy.sh
[modify] https://crrev.com/5fc380d88440b6cd11c3e26d4da8aba5db168d15/factory_reset.sh
[modify] https://crrev.com/5fc380d88440b6cd11c3e26d4da8aba5db168d15/factory_install.sh

Project Member

Comment 50 by bugdroid1@chromium.org, Aug 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/d5f355c7218ed5ba56a668fbf51bcb4e542c6809

commit d5f355c7218ed5ba56a668fbf51bcb4e542c6809
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:51:00 2017

factory_install: Make 'complete' and 'firmware' components optional.

When importing bundle into Umpire, unlike make_factory_package, Umpire
will only create entries with real file in JSON config.

As a result, we should ignore optional components if they are not
specified in server JSON file.

BUG= chromium:711615 
TEST=Created a JSON config without complete & firmware, do factory_install.sh.

Change-Id: I9bcb4b57a88ea34429d166ab792eea5b646e50b2
Reviewed-on: https://chromium-review.googlesource.com/520867
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
(cherry picked from commit 4c05357cfa615d418097320df2f3c14ded5ddc2c)
Reviewed-on: https://chromium-review.googlesource.com/594700
Reviewed-by: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[modify] https://crrev.com/d5f355c7218ed5ba56a668fbf51bcb4e542c6809/factory_install.sh

Labels: VerifyIn-61

Comment 52 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment