Issue metadata
Sign in to add a comment
|
various builds failing sporadically in BuildImage |
||||||||||||||||||||||||
Issue descriptionSee for instance https://luci-milo.appspot.com/buildbot/chromeos/lakitu-st-paladin/2706 https://luci-milo.appspot.com/buildbot/chromeos/lakitu-gpu-paladin/3177 both failed with IOError: [Errno 30] Read-only file system: '/mnt/host/source/src/build/images/lakitu-gpu/R67-10538.0.0-rc1/rootfs/var/cache/edb/vdb_metadata.pickle.2873' INFO : Unmounting image from /mnt/host/source/src/build/images/lakitu-gpu/R67-10538.0.0-rc1/stateful and /mnt/host/source/src/build/images/lakitu-gpu/R67-10538.0.0-rc1/rootfs INFO : Unmounting temporary rootfs /mnt/host/source/src/build/images/lakitu-gpu/R67-10538.0.0-rc1/rootfs//build/rootfs. rmdir: failed to remove '/mnt/host/source/src/build/images/lakitu-gpu/R67-10538.0.0-rc1/rootfs//build': Directory not empty An error occurred in your build so your latest output directory is invalid. Running in non-interactive mode so deleting output directory.
,
Apr 2 2018
lakitu-gpu-paladin failed once for the linked reason in the last 100 runs. Same was true for lakitu-st-paladin. They both failed four or five more times in the last 100 runs for different reasons (sometimes because of a bad CL in CQ), but considering the rate I'm going to mark this a P2.
,
Apr 3 2018
I suspect this has to do with the recent changes that stopped installing vmlinuz to rootfs but I can't be sure. FWIW, I see this error message in the BuildImage output since those changes: "/mnt/host/source/src/scripts/bin/cros_make_image_bootable: line 67: portageq-: command not found", which apparently was because the FLAGS_board was referenced before it was parsed, but the same error also appears in passing runs so I am sure it was the cause. Since the builder only failed once and is continuously passing, I am going to silence this until it fails again, in which case we'll hopefully have more logs to debug effectively.
,
Apr 3 2018
How about we leave the bug open for ease of discovery in case it recurs.
,
Apr 4 2018
Hey wonderfly@, I'm curious, why do yo think this is related to https://chromium-review.googlesource.com/c/chromiumos/platform/crosutils/+/969449 ? This should definitely be considered a risky change, but I'm not sure I see why the failure is related?
,
Apr 4 2018
Hi Craig, I think I tried to convey that idea that "I wasn't sure but that CL seemed suspicious". My main lead was the error message "cros_make_image_bootable: line 67: portageq-: command not found" that started showing in the BuildImage output at the same time (approximately) the lakitu-gpu build failed. But again, I didn't spend a ton of time on something that seemed intermittent so I could totally be wrong. Wanted to bring that up any way because the portageq- thing did seem a bug.
,
Apr 4 2018
The same problem just popped up on caroline-arcnext so it doesn't seem to be limited to lakitu: https://luci-milo.appspot.com/buildbot/chromeos/caroline-arcnext-paladin/119
,
Apr 4 2018
OK then. There is already a CL that's reverting the vmlinuz change because it has caused update engine failures in other cases, and I believe Craig is on top of them, so I'll just fold this into his pocket. :)
,
Apr 4 2018
,
Apr 5 2018
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by wonderfly@google.com
, Apr 2 2018Owner: wonderfly@google.com