x86-zgb paladin fails at build_RootFilesystemSize due to not enough free space on rootfs |
||||||||||||||||
Issue descriptionLink to the build: https://uberchromegw.corp.google.com/i/chromeos/builders/x86-zgb-paladin/builds/7848 /28 14:39:20.902 ERROR| base_utils:0280| [stderr] 2016-09-28 14:39:20 URL:http://100.115.219.131:8082/static/x86-zgb-paladin/R55-8845.0.0-rc2/autotest/packages/packages.checksum [33560/33560] -> "/usr/local/autotest/packages.checksum" [1] 09/28 14:39:21.822 ERROR| base_utils:0280| [stderr] cat: /usr/local/autotest/.checksum: No such file or directory 09/28 14:39:40.505 ERROR| logging_manager:0626| {'builds': "{'cros-version': 'x86-zgb-paladin/R55-8845.0.0-rc2'}", 'job_started': 1475098753, 'offload_failures_only': 'True', 'hostname': 'chromeos4-row4-rack1-host7', 'status_version': 1, 'label': 'x86-zgb-paladin/R55-8845.0.0-rc2/bvt-inline/build_RootFilesystemSize', 'parent_job_id': 78466630, 'drone': 'chromeos-server20.cbf.corp.google.com', 'user': 'chromeos-test', 'suite': 'bvt-inline', 'job_queued': 1475098671, 'retry_original_job_id': 78471580, 'experimental': 'False', 'build': 'x86-zgb-paladin/R55-8845.0.0-rc2'} 09/28 14:39:40.506 ERROR| logging_manager:0626| MACHINE NAME: chromeos4-row4-rack1-host7 09/28 14:39:40.506 ERROR| logging_manager:0626| MACHINE GROUP: x86-zgb 09/28 14:39:40.506 ERROR| logging_manager:0626| parsing partial test ---- SERVER_JOB 09/28 14:39:40.507 ERROR| logging_manager:0626| 09/28 14:39:40.507 ERROR| logging_manager:0626| STATUS: INFO ---- ---- kernel=3.8.11 localtime=Sep 28 14:39:18 timestamp=1475098758 09/28 14:39:40.509 ERROR| logging_manager:0626| 09/28 14:39:40.509 ERROR| logging_manager:0626| STATUS: START ---- ---- timestamp=1475098772 localtime=Sep 28 14:39:32 09/28 14:39:40.509 ERROR| logging_manager:0626| parsing partial test None CLIENT_JOB.0 09/28 14:39:40.509 ERROR| logging_manager:0626| RUNNING: RUNNING 09/28 14:39:40.510 ERROR| logging_manager:0626| CLIENT_JOB.0 09/28 14:39:40.510 ERROR| logging_manager:0626| 09/28 14:39:40.510 ERROR| logging_manager:0626| 09/28 14:39:40.510 ERROR| logging_manager:0626| STATUS: START build_RootFilesystemSize build_RootFilesystemSize timestamp=1475098772 localtime=Sep 28 14:39:32 09/28 14:39:40.510 ERROR| logging_manager:0626| parsing partial test build_RootFilesystemSize build_RootFilesystemSize 09/28 14:39:40.510 ERROR| logging_manager:0626| RUNNING: RUNNING 09/28 14:39:40.511 ERROR| logging_manager:0626| Subdir: build_RootFilesystemSize 09/28 14:39:40.511 ERROR| logging_manager:0626| Testname: build_RootFilesystemSize 09/28 14:39:40.511 ERROR| logging_manager:0626| 09/28 14:39:40.511 ERROR| logging_manager:0626| 09/28 14:39:40.511 ERROR| logging_manager:0626| STATUS: FAIL build_RootFilesystemSize build_RootFilesystemSize timestamp=1475098776 localtime=Sep 28 14:39:36 15081472 bytes free is less than the 15728640 required. 09/28 14:39:40.511 ERROR| logging_manager:0626| update RUNNING reason: 15081472 bytes free is less than the 15728640 required. 09/28 14:39:40.511 ERROR| logging_manager:0626| 09/28 14:39:40.512 ERROR| logging_manager:0626| STATUS: END FAIL build_RootFilesystemSize build_RootFilesystemSize timestamp=1475098776 localtime=Sep 28 14:39:36 09/28 14:39:40.512 ERROR| logging_manager:0626| parsing test build_RootFilesystemSize build_RootFilesystemSize 09/28 14:39:40.512 ERROR| logging_manager:0626| ADD: FAIL 09/28 14:39:40.512 ERROR| logging_manager:0626| Subdir: build_RootFilesystemSize 09/28 14:39:40.512 ERROR| logging_manager:0626| Testname: build_RootFilesystemSize 09/28 14:39:40.512 ERROR| logging_manager:0626| 15081472 bytes free is less than the 15728640 required. 09/28 14:39:40.512 ERROR| logging_manager:0626| 09/28 14:39:40.513 ERROR| logging_manager:0626| STATUS: END GOOD ---- ---- timestamp=1475098777 localtime=Sep 28 14:39:37 09/28 14:39:40.513 ERROR| logging_manager:0626| parsing test None CLIENT_JOB.0 09/28 14:39:40.513 ERROR| logging_manager:0626| ADD: GOOD 09/28 14:39:40.513 ERROR| logging_manager:0626| Subdir: None 09/28 14:39:40.513 ERROR| logging_manager:0626| Testname: CLIENT_JOB.0 09/28 14:39:40.513 ERROR| logging_manager:0626| 09/28 14:39:40.513 ERROR| logging_manager:0626| parsing test ---- SERVER_JOB I am not sure whether it is caused by the specific DUT or the test itself fails for some reason
,
Sep 29 2016
This bug is blocking CQ, I think it is test has some issue https://uberchromegw.corp.google.com/i/chromeos/builders/x86-zgb-paladin
,
Sep 29 2016
,
Sep 29 2016
No, the test is working as intended. The problem is that the root file system image size for ZGB is too big. That's the bug that this test is meant to find, so the test is WAI. IIUC, vapier@ has committed, or will soon commit, a fix.
,
Sep 29 2016
Before the fix is in, do we need to make this build as experimental?
,
Sep 29 2016
my CL is waiting for the CQ keep in mind that the way this test fails is for some CLs to be adding new stuff. so you should check out the CLs that were in that run because they're the ones causing the failure. i don't think that means the CQ is blocked.
,
Sep 29 2016
Suspicious CL: https://chromium-review.googlesource.com/#/c/390412/
,
Sep 29 2016
,
Sep 29 2016
It also failed on https://uberchromegw.corp.google.com/i/chromeos/builders/x86-zgb-paladin/builds/7852 , whereas that change was not part of that build.
,
Sep 29 2016
Wrong CL. This is the CL https://chromium-review.googlesource.com/#/c/388798/
,
Sep 29 2016
Neither of these CLs is contributing to the CQ failures:
https://chromium-review.googlesource.com/#/c/388798/
https://chromium-review.googlesource.com/#/c/390412/
As a practical matter, most changes that can tickle this problem
will involve changes to an overlay. And by and large, CLs
that tickle this problem probably aren't really at fault; the
true permanent fix is this CL:
https://chromium-review.googlesource.com/#/c/390851/
,
Sep 29 2016
I took a look at the most recent ToT builds. Both of them have roughly 30MB of leeway for the failure threshold, so the CQ failures aren't occurring because of a bug in ToT. So, although nothing jumped out at me, it's likely that there's one or more CLs in recent runs that are bumping the image size significantly. We might want to do a wider search for possible culprits.
,
Sep 29 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/crosutils/+/fc813d0121cf65693d5ad67d25d05ac58ec273fc commit fc813d0121cf65693d5ad67d25d05ac58ec273fc Author: Mike Frysinger <vapier@chromium.org> Date: Thu Sep 29 20:50:02 2016 disk layout: add another 16MiB to the rootfs We've bumped up against the limit again with more features we've been adding (like cups). Increase the rootfs since we want these more than we don't want them. BUG= chromium:650105 BUG= chromium:651224 TEST=nope! Change-Id: I1821a911a36d0962ed012a0d7ef47fc8ffa7a1ac Reviewed-on: https://chromium-review.googlesource.com/391191 Trybot-Ready: Mike Frysinger <vapier@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Andrew de los Reyes <adlr@chromium.org> Tested-by: Andrew de los Reyes <adlr@chromium.org> [modify] https://crrev.com/fc813d0121cf65693d5ad67d25d05ac58ec273fc/build_library/legacy_disk_layout.json
,
Sep 29 2016
,
Sep 30 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/cros-signing/+/b556b3384450249d370008979eca18a371a6bfa3 commit b556b3384450249d370008979eca18a371a6bfa3 Author: Mike Frysinger <vapier@chromium.org> Date: Fri Sep 30 02:59:49 2016
,
Sep 30 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/cros-signing/+/497526f6957bbb9be17c6dfee8b33dc086a84a97 commit 497526f6957bbb9be17c6dfee8b33dc086a84a97 Author: Mike Frysinger <vapier@chromium.org> Date: Fri Sep 30 04:42:52 2016
,
Oct 7 2016
,
Nov 19 2016
,
Jan 21 2017
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
Aug 1 2017
,
Oct 14 2017
,
Jun 21 2018
|
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by shuqianz@chromium.org
, Sep 28 2016