New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 731520 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

factory: stateful partition is full after first factory installation

Project Member Reported by hungte@chromium.org, Jun 9 2017

Issue description

Found on ToT.

Due to Arc++, stateful partition is getting more and more full on test image. For example, a recent build has only 95M free.

When the toolkit is pretty large (for example contained a firmware updater, 30M), installation on first boot would need to extract the files, and since /usr/local/factory/enabled is not created at that time, system will try to do encrypted stateful partition, taking 10% of free stateful partition space. Together this makes installing factory toolkit not able to finish.

So there are two problems here:

1. We should either delete known huge unused folders (like telemetry) or increasing stateful partition when first installed.

2. We should figure out a way to let system first boot in unecrypted mode even if toolkit is not installed yet - or find a way so it can switch between modes without losing factory state data.
 
Currently a project can create a completion script, with
 resize2fs /dev/mmcblk0p1 2048M

to enlarge stateful partition, but we still need to figure out a "good" way.
Project Member

Comment 2 by bugdroid1@chromium.org, Jun 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/3285ffdd0924313a4f34f5f3ddce21b1ae53c452

commit 3285ffdd0924313a4f34f5f3ddce21b1ae53c452
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 09 14:10:41 2017

factory_install: Always try to get +1G free space for stateful partition.

To reduce installation issues, we want to always ensure the stateful
partition is large enough for installing toolkits.

The selected size is "+1G of current" because a stateful partition that
is too large may need very long time in finalization for wiping.
Chrome OS test images usually comes with 1.2G for stateful partition on
USB stick, and factory toolkit is usually 30~50M, so +1G should be large
enough for most cases.

BUG= chromium:731520 
TEST=build_image factory_install; tried RMA and success.

Change-Id: I373d910456f00a5415e0108d922edac680456f0c
Reviewed-on: https://chromium-review.googlesource.com/527865
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Mao Huang <littlecvr@chromium.org>

[modify] https://crrev.com/3285ffdd0924313a4f34f5f3ddce21b1ae53c452/factory_install.sh

Project Member

Comment 3 by bugdroid1@chromium.org, Jun 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/initramfs/+/06551d8f9821df12e02b101dc4a5f4f17d07d433

commit 06551d8f9821df12e02b101dc4a5f4f17d07d433
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 09 16:16:28 2017

factory_netboot: Add dumpe2fs and e2fsck in Makefile.

The factory installer is going to invoke resize2fs to enlarge stateful
partition in initial installation (CL:527865) so we have to include
the dependencies - dumpe2fs and e2fsck - in netboot environment.

BUG= chromium:731520 
TEST=make_netboot.sh

Change-Id: Ib33fb801fd56b6ff269e56e370ae3c095c9e0cb8
Reviewed-on: https://chromium-review.googlesource.com/527961
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/06551d8f9821df12e02b101dc4a5f4f17d07d433/factory_netboot/Makefile

Project Member

Comment 4 by bugdroid1@chromium.org, Jun 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/aebb3f03876ac0cc0a3a6e443c9c3f84510f576c

commit aebb3f03876ac0cc0a3a6e443c9c3f84510f576c
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 09 16:16:29 2017

init: Change factory data to bind on ${STATEFUL}/var/factory.

When factory toolkit is installed, newer test images will mount var and
/home/chronos as unencrypted. However, when installed from network or
USB, the toolkit (and tag of factory/enabled) will be installed after
chromeos_startup mounts stateful partition in encrypted state - and that
creates a transition between two reboots that /var/factory data will be
invisible on second boot.

To solve that, we want to make sure /var/factory is served from same
source location, no matter in encrypted or unencrypted mode.

Using ${STATEFUL}/var/factory is better (which is the default place that
unencrypted mount uses) than var_overlay (only used by encrypted).

BUG= chromium:731520 
TEST=make test; installed toolkit on DUT.

Change-Id: I87e0e5f082e59531afddac9561078155a8847822
Reviewed-on: https://chromium-review.googlesource.com/527545
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/aebb3f03876ac0cc0a3a6e443c9c3f84510f576c/init/common.d/unencrypted_factory_data.sh

Project Member

Comment 5 by bugdroid1@chromium.org, Jun 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/15f521768c081eed51402b084daa2809e95f9e26

commit 15f521768c081eed51402b084daa2809e95f9e26
Author: Hung-Te Lin <hungte@chromium.org>
Date: Sat Jun 10 06:25:28 2017

setup: Always +1G for diskimg (copy machine) in make_factory_package.

Factory installer now always appends 1G extra space in stateful
partition, and we have to apply same settings here because --diskimg
won't invoke factory_installer.

resize_image_fs.sh is also revised to allow specifying only "size to
add", no need to always calculate original file system size.

BUG= chromium:731520 
TEST=./make_factory_package.sh --diskimg FILE ....

Change-Id: I861c5fb1cf52916f279272ff6d745ae7a5d9b09d
Reviewed-on: https://chromium-review.googlesource.com/528047
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/15f521768c081eed51402b084daa2809e95f9e26/setup/make_factory_package.sh
[modify] https://crrev.com/15f521768c081eed51402b084daa2809e95f9e26/setup/resize_image_fs.sh

Project Member

Comment 6 by bugdroid1@chromium.org, Jun 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/5b96a2545ad5821ca3d0f3d363e16470d7341653

commit 5b96a2545ad5821ca3d0f3d363e16470d7341653
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Jun 16 18:24:28 2017

init: Install cros_payloads in a clean and safer way.

We found that in early, for example proto, builds, device may be in
unstable state and reboot unexpectedly during first boot. This implies
cros_payload installation may go wrong if the machine reboots before the
changes to system disk is flushed.

As a result, we want to revise the payload installation to its own
service, cros-payloads.conf, so it won't rely on factory stuff.
Also, change the "removal of install stubs" to be postponed to few
seconds after sync(1) is invoked. This should allow re-installation if
system is unstable in early stage.

BUG= chromium:731520 
TEST=Manually installed on system and verified cros_payload resources
     were installed.

Change-Id: Ia7b702997de7168fea56eed98525b74e81791bef
Reviewed-on: https://chromium-review.googlesource.com/527548
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/5b96a2545ad5821ca3d0f3d363e16470d7341653/init/upstart/test-init/factory_utils.sh
[modify] https://crrev.com/5b96a2545ad5821ca3d0f3d363e16470d7341653/init/upstart/test-init/factory-init.conf
[add] https://crrev.com/5b96a2545ad5821ca3d0f3d363e16470d7341653/init/upstart/test-init/cros-payloads.conf

Comment 7 by hungte@chromium.org, Jul 12 2017

Owner: hungte@chromium.org
Status: Fixed (was: Untriaged)
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 1 2017

Labels: merge-merged-factory-gru-9017.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory_installer/+/1d6c0c33a2ef55a0cf8d095de692a753c7f115f5

commit 1d6c0c33a2ef55a0cf8d095de692a753c7f115f5
Author: Hung-Te Lin <hungte@chromium.org>
Date: Tue Aug 01 00:51:19 2017

factory_install: Always try to get +1G free space for stateful partition.

To reduce installation issues, we want to always ensure the stateful
partition is large enough for installing toolkits.

The selected size is "+1G of current" because a stateful partition that
is too large may need very long time in finalization for wiping.
Chrome OS test images usually comes with 1.2G for stateful partition on
USB stick, and factory toolkit is usually 30~50M, so +1G should be large
enough for most cases.

BUG= chromium:731520 
TEST=build_image factory_install; tried RMA and success.

Change-Id: I373d910456f00a5415e0108d922edac680456f0c
Reviewed-on: https://chromium-review.googlesource.com/527865
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Mao Huang <littlecvr@chromium.org>
(cherry picked from commit 3285ffdd0924313a4f34f5f3ddce21b1ae53c452)
Reviewed-on: https://chromium-review.googlesource.com/594701
Reviewed-by: Philip Chen <philipchen@chromium.org>
Tested-by: Philip Chen <philipchen@chromium.org>

[modify] https://crrev.com/1d6c0c33a2ef55a0cf8d095de692a753c7f115f5/factory_install.sh

Sign in to add a comment