+apronin so he won't open a new one.

The Glimmer system should not have ARC++ on it, so I think this might just be normal growth?

It is kind of strange just Glimmer fails in this way, it should not be significantly different from the other BayTrail systems, it gets touchview mode which might account for some but I would expect Clapper to be failing in the same way if this is touchview space.

I guess the easy answer is to make the FS bigger, but it might be good to figure out why it is bigger. Nothing specific to glimmer in https://crosland.corp.google.com/log/8830.0.0..8831.0.0 my guess is that Chrome is now just slightly bigger and Glimmer happened to be closest to the edge?

+gwendal, +vapier

Did we ever implement any systematic approach to this? (I suspect not.)  That is, every time we exceed the limit, does anything happen other than doing verbal self-penance and genuflexions and then we bump the limit up again?

I was hoping that we could look up some UMA stat for this, but I cannot find one.  We have Platform.DiskUsageChronos and Platform.DiskUsageVar, and also a couple for "caches" (http?) but nothing for the rootfs.  Which kind of makes sense since its size is fixed---at least on each release.

Does anybody know if we have the rootfs size by board precomputed somewhere?

UMA is for live devices.  for buildbots, we have perfdashboard, and we have been keeping track there.

here's a generated report:
https://chromeperf.appspot.com/report?sid=4453b93330305a978bed5687aaedf51e3dbbc5b858ef1a29f14c7c07481ef17d
test=build_RootFilesystemSize
bot=cros-glimmer
subtest=bytes_rootfs_prod

Very nice, thank you!

So it seems like we gained 6MB in https://crosland.corp.google.com/log/8808.0.0..8809.0.0

Maybe some of the print stuff pushed us over the edge?

Not sure there is anything we can really do about this aside from starting a review of where we can cut space, or just increasing the space :(.

This test has now failed x86-zgb paladin build 7850.

The test name starts with "build_" so perhaps it is owned by the build team?  Some of the committers are Mike F., Don G., also Alex D., Gaurav, Ken M.

I am fine with just increasing the space.  Let's make sure we don't increase it too much, otherwise it will be too long before we reach this level of entertainment again.

no, that's not how "build_" works :).  the test keeps track of rootfs usage.  the people responsible is everyone on the team, especially with an eye towards commits that add a lot of content.

OK but what do we do then?  Send mail to chromeos-kernel?

"Team: bad bad bad!  Our clever tests are telling us our images are getting too big *again*!  Please reconsider your awfully space-wasting commits, be thrifty, and you will be rewarded, in a time frame and manner which depend on your religion."

Sorry, my meds are affecting my sarcasm center.

At VMware we had serious space restrictions for the virtual machine monitor on x86 (32 bit), and every commit message had to include the growth (or reduction) in data and text sizes, which were important factors in accepting or rejecting the change.  Maybe we should consider something similar?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/fc813d0121cf65693d5ad67d25d05ac58ec273fc

commit fc813d0121cf65693d5ad67d25d05ac58ec273fc
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Sep 29 20:50:02 2016

disk layout: add another 16MiB to the rootfs

We've bumped up against the limit again with more features we've
been adding (like cups).  Increase the rootfs since we want these
more than we don't want them.

BUG= chromium:650105 
BUG= chromium:651224 
TEST=nope!

Change-Id: I1821a911a36d0962ed012a0d7ef47fc8ffa7a1ac
Reviewed-on: https://chromium-review.googlesource.com/391191
Trybot-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Tested-by: Andrew de los Reyes <adlr@chromium.org>

[modify] https://crrev.com/fc813d0121cf65693d5ad67d25d05ac58ec273fc/build_library/legacy_disk_layout.json

yes, people should be aware of the overhead they're adding to the rootfs, and large additions should be required to be approved.  unfortunately, our tracking is much too late, and we end up just silently dying from the last thousand cuts instead of the big dumps from days/weeks earlier.

this test is basically a low watermark.  the next failure will be build_image reporting out of disk space in the rootfs.

if we ignore it and just keep increasing the rootfs, then we'll run out of space in the root partition, and then we'll def be screwed with nowhere to go.

i've filed  issue 651574  to track future work.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/cros-signing/+/b556b3384450249d370008979eca18a371a6bfa3

commit b556b3384450249d370008979eca18a371a6bfa3
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 30 02:59:49 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/cros-signing/+/497526f6957bbb9be17c6dfee8b33dc086a84a97

commit 497526f6957bbb9be17c6dfee8b33dc086a84a97
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 30 04:42:52 2016