PaygenBuildCanary failing for reef-release since Oct 4 |
|||||||
Issue descriptionhttps://uberchromegw.corp.google.com/i/chromeos/builders/reef-release PaygenBuildCanary failure is maybe builds #1545->head (#1561). There was a BuildPackages failure introduced at #1552 and fixed at #1560 which masked this. 22:26:38: ERROR: <class 'chromite.lib.paygen.paygen_payload_lib.PayloadVerificationError'>: Payload integrity check failed: install_operations[9](SOURCE_BSDIFF).src_extents[491]: extent (448032->448034 * 4096) exceeds usable partition size (1835008000). That's the only log that I can find that explains the error. adlr, I'm not sure why this is failing or where to route. Can you assist in routing?
,
Oct 10 2017
,
Oct 10 2017
I'm trying to reproduce is locally and see what is the problem.
,
Oct 11 2017
I can reproduce the bug but, I'm still investigating the exact cause of this bug. it seems to me that the build version 10002 (which was used as the source partition) had a significant increase in rootfs partition size from 1835008000 bytes to 1939865600 bytes and goes back to 1835008000 in subsequent builds. This probably caused an uncaught bug in update_payload scripts. It might take a while to fix the bug and merge it in. If necessary, the deputies can force the release builders to use a newer version of an image (something higher than 10002.0.0) temporarily to bypass this error until the fix gets in. There is actually a term for this that I forgot :)
,
Oct 11 2017
Oh..... There was a long standing payload generation bug with rootfs sizes that go down, though I thought deymo had fixed it. The easy fix is to increase the rootfs size again. This doesn't cause the payload to get much bigger because the blocks are zero'd out, though it does cause more client disk writes than needed. The right fix is to handle a shrinking rootfs size, of course. It shouldn't really be a problem.
,
Oct 11 2017
This is not a payload generation bug. The delta is generated correctly. It is the python scripts that check for the integrity of the payload. I uploaded a fix patch, but waiting for senj@ to look and see if it is the proper way of fixing the issue. The solution is not increasing the rootfs size at this point. https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/713463/1/scripts/update_payload/checker.py#1062
,
Oct 13 2017
Still waiting for the CL to get +2ed :/
,
Oct 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/ae85374770d4492d28e62d792824eea55349c981 commit ae85374770d4492d28e62d792824eea55349c981 Author: Amin Hassani <ahassani@google.com> Date: Mon Oct 16 21:14:00 2017 update_payload: Fix larger source issue the _CheckOperations() function passes the new usable partition size as the old usable partition size. If the source image is larger than the target image, this will be errornous as some checks fail. BUG= chromium:773336 TEST=scripts/paycheck.py --check delta.bin (10002.0.0 reef -> 10019.0.0 reef) Change-Id: I8c30129831daff8e70df3dcb9639ff240e4a37ba Reviewed-on: https://chromium-review.googlesource.com/713463 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> [modify] https://crrev.com/ae85374770d4492d28e62d792824eea55349c981/scripts/update_payload/checker_unittest.py [modify] https://crrev.com/ae85374770d4492d28e62d792824eea55349c981/scripts/update_payload/checker.py
,
Oct 20 2017
I'm marking this as fixed and verified.
,
Oct 20 2017
Sorry, just fixed, someone should verify it too :)
,
Jan 22 2018
,
Jan 23 2018
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by jclinton@chromium.org
, Oct 10 2017