New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 820243 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 0
Type: Bug



Sign in to add a comment

paygen is calculating the wrong metadata size

Project Member Reported by ahass...@chromium.org, Mar 8 2018

Issue description

Omaha is sending the wrong metadata size. In the recent change crrev.com/899845, the value of metadata size was sent instead of data_offset. They are a little bit different and confusing. This needs to be fixed.


0308/081738:INFO:omaha_request_action.cc(681)] Request: <?xml version="1.0" encoding="UTF-8"?>
<request protocol="3.0" version="ChromeOSUpdateEngine-0.1.0.0" updaterversion="ChromeOSUpdateEngine-0.1.0.0" installsource="ondemandupdate" ismachine="1">
    <os version="Indy" platform="Chrome OS" sp="10455.0.0_aarch64"></os>
    <app appid="{90F229CE-83E2-4FAF-8479-E368A34938B1}" cohort="1:3e:" cohortname="scarlet_canary" version="10455.0.0" track="canary-channel" lang="en-US" board="scarlet-signed-prempkeys" hardware_class="SCARLET B5B-A2D-D2K-L74" delta_okay="false" fw_version="" ec_version="" installdate="4060" >
        <event eventtype="13" eventresult="1"></event>
    </app>
</request>
...
[0308/081739:INFO:omaha_request_action.cc(940)] Omaha request response: <?xml version="1.0" encoding="UTF-8"?><response protocol="3.0" server="prod"><daystart elapsed_days="4084" elapsed_seconds="29859"/><app appid="{90F229CE-83E2-4FAF-8479-E368A34938B1}" status="ok"><event status="ok"/></app></response>
[0308/081739:INFO:action_processor.cc(116)] ActionProcessor: finished OmahaRequestAction with code ErrorCode::kSuccess
[0308/081739:INFO:action_processor.cc(143)] ActionProcessor: starting DownloadAction
[0308/081739:INFO:install_plan.cc(71)] InstallPlan: new_update, payload type: full, source_slot: B, target_slot: A, url: http://storage.googleapis.com/chromeos-releases-public/canary-channel/canary-channel/10468.0.0/chromeos_10468.0.0_scarlet_canary-channel_full_premp.bin-0a507c3111a4c8f691cd2dc438800644.signed, payload size: 998375458, payload hash: ib18e/aoMRon2NIumsZwSAaTvwCrYd7PRBeUY9URk98=, metadata size: 46888, metadata signature: qxYLK2rg+ALEVxE8eoCrTJdwb1xa02Otjfdy3VTxudLBmIobqOWm2EH1OOH7pONBjIVlm0J7OBUEkmJAFB6U1d+buM+6NFhSePXzyqTI7AISSb1hCUAIrY9YxcKZHbYfQEJ6E1vCOzJzQDVzACZs8WYSvewyAynbV/pbIwKRQ8oC32OJ7TB3P2YY3LsoFzb24pwsTQwGUTR1RXr1RCWKiRfGgqXcTURlRURrUm8HEtzHVqDv+k7ekXF7dbabWgfUH9ClYTLRhPFKi8a0bS/8VnM3W4Nrw6NSyTuffRtxI4iMKImqi3KeKCTzjwFZ5SMY/GL5gJJEOsPeRDwwWAsF9A==, hash_checks_mandatory: true, powerwash_required: false
...
[0308/081637:ERROR:delta_performer.cc(546)] Mandatory metadata size in Omaha response (46888) is missing/incorrect, actual = 46925
[0308/081637:ERROR:download_action.cc(280)] Error ErrorCode::kDownloadInvalidMetadataSize (32) in DeltaPerformer's Write method when processing the received payload -- Terminating processing
[0308/081637:INFO:delta_performer.cc(320)] Discarding 24 unused downloaded bytes
[0308/081637:INFO:multi_range_http_fetcher.cc(172)] Received transfer terminated.
[0308/081637:INFO:multi_range_http_fetcher.cc(124)] TransferEnded w/ code 200
[0308/081637:INFO:multi_range_http_fetcher.cc(126)] Terminating.
[0308/081637:INFO:action_processor.cc(116)] ActionProcessor: finished DownloadAction with code ErrorCode::kDownloadInvalidMetadataSize
[0308/081637:INFO:action_processor.cc(121)] ActionProcessor: Aborting processing due to failure.
...
 
Cc: briannorris@chromium.org bhthompson@chromium.org josa...@chromium.org

Comment 3 by leecy@chromium.org, Mar 8 2018

Cc: dgarr...@chromium.org
Summary: paygen is calculating the wrong metadata size (was: omaha is sending the wrong metadata size to clients.)
I think the title was not correct. Fixed it.
In order words, it is not GE or omaha problem, it is paygen's problem.
I uploaded crrev.com/c/956532 for review. I ran a tryjob on the same canary build and You can see the correct number for metadata size 46925 in the following json file:

https://storage.cloud.google.com/chromeos-releases/canary-channel/scarlet/10468.0.0/payloads/chromeos_10468.0.0_scarlet_canary-channel_full_premp.bin-0a507c3111a4c8f691cd2dc438800644.signed.json
Project Member

Comment 7 by bugdroid1@chromium.org, Mar 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/2b4b71805bd37a76776a7e52d9de60b7c1c37e2f

commit 2b4b71805bd37a76776a7e52d9de60b7c1c37e2f
Author: Amin Hassani <ahassani@google.com>
Date: Sat Mar 10 01:39:11 2018

paygen: calculate metadata size after signing

metadata size should be calculated after signing, not before. This patch fixes
it.

BUG= chromium:820243 
TEST=cros tryjob --version 10468.0.0 --channel canary -g "956532 930172" scarlet-payloads

Change-Id: I8edf963714bb82c1030296b7bde1328e6a1773b5
Reviewed-on: https://chromium-review.googlesource.com/956532
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/2b4b71805bd37a76776a7e52d9de60b7c1c37e2f/lib/paygen/paygen_payload_lib_unittest.py
[modify] https://crrev.com/2b4b71805bd37a76776a7e52d9de60b7c1c37e2f/lib/paygen/paygen_payload_lib.py

Status: Fixed (was: Started)
Project Member

Comment 9 by bugdroid1@chromium.org, Mar 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/update_engine/+/a86b108fc604d296edbd3f6c158a17175ef6f9fe

commit a86b108fc604d296edbd3f6c158a17175ef6f9fe
Author: Amin Hassani <ahassani@google.com>
Date: Fri Mar 23 21:51:11 2018

update_payload: Allow check for given metadata size

Allow passing metadata size to check_update_payload so we can verify the
metadata size in omaha equals to the one in the payload.

BUG= chromium:820243 
TEST=run paycheck.py with both valid and invalid metadata sizes reports as expected
TEST=unittests

Change-Id: Ib41ce77af77636fffec6752201c363e7fbbf868d
Reviewed-on: https://chromium-review.googlesource.com/955679
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>

[modify] https://crrev.com/a86b108fc604d296edbd3f6c158a17175ef6f9fe/scripts/update_payload/checker_unittest.py
[modify] https://crrev.com/a86b108fc604d296edbd3f6c158a17175ef6f9fe/scripts/update_payload/checker.py
[modify] https://crrev.com/a86b108fc604d296edbd3f6c158a17175ef6f9fe/scripts/update_payload/payload.py
[modify] https://crrev.com/a86b108fc604d296edbd3f6c158a17175ef6f9fe/scripts/paycheck.py

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 3 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/e84defad5612c36f22ac0e475c8140b4607f9308

commit e84defad5612c36f22ac0e475c8140b4607f9308
Author: Amin Hassani <ahassani@google.com>
Date: Tue Apr 03 19:50:43 2018

paygen: Check for the correct metadata size in the verify section

We need to verify the generated metadata size in the signing section equals to
the one in the payload. This patch passes the generated metadata to
check_update_payload for verification.

BUG= chromium:820243 
TEST=unittests
TEST=payload tryjob
CQ-DEPEND=CL:955679

Change-Id: I665d72bcc59f8d5180d48b0d4de1e3173b3176d1
Reviewed-on: https://chromium-review.googlesource.com/956670
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/e84defad5612c36f22ac0e475c8140b4607f9308/lib/paygen/paygen_payload_lib_unittest.py
[modify] https://crrev.com/e84defad5612c36f22ac0e475c8140b4607f9308/lib/paygen/paygen_payload_lib.py

Project Member

Comment 11 by bugdroid1@chromium.org, Apr 3 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/527d0604ab5ddb68549c24b0396cd5ba6c75f813

commit 527d0604ab5ddb68549c24b0396cd5ba6c75f813
Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Apr 03 23:39:32 2018

Roll src/third_party/chromite/ 87c1c3dbe..ec8d7a766 (3 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/87c1c3dbea9a..ec8d7a766a7e

$ git log 87c1c3dbe..ec8d7a766 --date=short --no-merges --format='%ad %ae %s'
2018-03-01 achuith [chrome_committer]: Split out chrome_committer.
2018-03-08 ahassani paygen: Log all chroot operations in the log file
2018-03-08 ahassani paygen: Check for the correct metadata size in the verify section

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:814910 , chromium:808495 , chromium:820243 


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: I2dc84a4feb54038e73df54f3f123e7cf8e6d3ae0
Reviewed-on: https://chromium-review.googlesource.com/993704
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#547869}
[modify] https://crrev.com/527d0604ab5ddb68549c24b0396cd5ba6c75f813/DEPS

Sign in to add a comment