New issue
Advanced search Search tips

Issue 852017 link

Starred by 4 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

xbuddy cannot flash signed images to DUTs directly

Project Member Reported by lhchavez@chromium.org, Jun 12 2018

Issue description

Since signed images are stored in gs://chromeos-releases/${CHANNEL}-channel/${BOARD}/${VERSION}/ instead of gs://chromeos-image-archive/chromeos-image-archive/${BOARD}-release/R${RELEASE}-${VERSION}/, and both buckets follow a _slightly_ different directory layout, it's not easy for xbuddy to find and flash signed images.

Right now we need to manually download the image from GoldenEye and flash using commandline, which is a bit cumbersome.
 
 Issue 852081  has been merged into this issue.
The other problem that was mentioned in  crbug.com/852081  is that cros flash cannot install a signed recovery image even if you pass it the local path to the recovery image. It has to do with the stateful partition as error below suggest. I'll inspect more.

[12/Jun/2018:13:14:46] UPDATE Running cros_generate_update_payload --image /mnt/host/source/devserver/static/local_imageGZJ_I7/link/chromiumos_test_image.bin --out_metadata_hash_file /mnt/host/source/devserver/static/cache/cda0c70d49ff7067226949173d03f0d3/metadata_hash --output /mnt/host/source/devserver/static/cache/cda0c70d49ff7067226949173d03f0d3/update.gz
[12/Jun/2018:13:15:32] UPDATE Running cros_generate_stateful_update_payload --image /mnt/host/source/devserver/static/local_imageGZJ_I7/link/chromiumos_test_image.bin --output_dir /mnt/host/source/devserver/static/cache/cda0c70d49ff7067226949173d03f0d3
[12/Jun/2018:13:15:34] HTTP Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib64/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/lib/devserver/devserver.py", line 1520, in xbuddy
    image_name=file_name)
  File "/usr/lib64/devserver/autoupdate.py", line 500, in GetUpdateForLabel
    rel_path = self.GenerateUpdateImageWithCache(static_image_path)
  File "/usr/lib64/devserver/autoupdate.py", line 433, in GenerateUpdateImageWithCache
    self.GenerateUpdateImage(image_path, cache_dir)
  File "/usr/lib64/devserver/autoupdate.py", line 399, in GenerateUpdateImage
    raise AutoupdateError('Failed to generate update in %s' % output_dir)
AutoupdateError: Failed to generate update in /mnt/host/source/devserver/static/cache/cda0c70d49ff7067226949173d03f0d3

[12/Jun/2018:13:15:34] HTTP 
Request Headers:
  HOST: 172.22.93.166:38643
  CONNECTION: close
  Remote-Addr: ::ffff:172.22.93.166
  USER-AGENT: Python-urllib/2.7
  ACCEPT-ENCODING: identity
::ffff:172.22.93.166 - - [12/Jun/2018:13:15:34] "GET /xbuddy/local_imageGZJ_I7/link/test?for_update=true&return_dir=true HTTP/1.1" 500 1890 "" "Python-urllib/2.7"
--- End output from /tmp/devserver_wrapper7Ms5mr/dev_server.log ---
13:15:35: ERROR: Unable to get payloads from local path: /tmp/cros-flashminUC5
13:15:35: ERROR: Device update failed.
13:15:35: ERROR: cros flash failed before completing.
13:15:35: ERROR: HTTP Error 500: Internal Server Error

Cc: ahass...@chromium.org

Comment 4 by vapier@chromium.org, Jun 12 2018

Components: Infra>Client>ChromeOS>Build
Labels: OS-Chrome
Summary: xbuddy cannot flash signed images to DUTs directly (was: xbuddy cannot flash signed images)
https://chromium-review.googlesource.com/#/c/chromiumos/platform/dev-util/+/1097609 should allow you to download the iamge from xbuddy into a USB stick.
It seems that signed recovery images do not have var_overlay and dev_image in the stateful partition. But mount_gpt_image.sh needs that. Is it ok to pass a flag to mount_gpt_image.sh to ignore mounting those directories when needed (in this case for example)? Well, not ignore it, but don't fail if the directories do not exits. In addition, we can ignore those when generating the stateful partition tar ball if they don't exist. Is that a valid solution? Any other suggestion?
Project Member

Comment 7 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/dev-util/+/dca9dd79af1d112940ca003c04d636cb2d9e626e

commit dca9dd79af1d112940ca003c04d636cb2d9e626e
Author: Luis Hector Chavez <lhchavez@google.com>
Date: Tue Jun 19 10:48:55 2018

xbuddy: Introduce the 'signed' image type

This change makes it possible to flash signed images from cros flash
using the `signed` alias.

BUG=chromium:852017
TEST=cros flash usb:// xbuddy://remote/eve-arcnext/latest-canary/signed
TEST=./xbuddy_unittest.py

Change-Id: I1c4c76ca0dd29845ef8639744745904e209a34eb
Reviewed-on: https://chromium-review.googlesource.com/1097609
Commit-Ready: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Luis Hector Chavez <lhchavez@chromium.org>
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/build_artifact.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/devserver.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/xbuddy_unittest.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/downloader.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/xbuddy.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/devserver_constants.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/downloader_unittest.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/artifact_info.py
[modify] https://crrev.com/dca9dd79af1d112940ca003c04d636cb2d9e626e/build_artifact_unittest.py

@vapier: Can I get your consent on #6?
Project Member

Comment 9 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/bdfbcec460207b7d66d2cb7dc6470355791a99d6

commit bdfbcec460207b7d66d2cb7dc6470355791a99d6
Author: Luis Hector Chavez <lhchavez@google.com>
Date: Tue Jun 19 21:38:42 2018

Add 'signed' image type examples / docs to cros flash

This change adds some examples and docs about the image types, including
the recently introduced 'signed' image type.

CQ-DEPEND=CL:1097609
BUG=chromium:852017
TEST=cros flash --help

Change-Id: I065a0bf3fd838d5d8e7695bc9ccff1f6cc56a75c
Reviewed-on: https://chromium-review.googlesource.com/1099262
Commit-Ready: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Luis Hector Chavez <lhchavez@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/bdfbcec460207b7d66d2cb7dc6470355791a99d6/cli/cros/cros_flash.py

Project Member

Comment 10 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/84e07da8461b4b76da32670f411417a117a8340c

commit 84e07da8461b4b76da32670f411417a117a8340c
Author: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Jun 19 23:46:40 2018

Roll src/third_party/chromite efd58885b78d..9cccc5ce4fb9 (7 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/efd58885b78d..9cccc5ce4fb9


git log efd58885b78d..9cccc5ce4fb9 --date=short --no-merges --format='%ad %ae %s'
2018-06-19 ayatane@chromium.org sysmon: Ignore chromeos3-* devservers
2018-06-19 dgarrett@google.com gclient: Stop using implicit git_cache_dir.
2018-06-19 lhchavez@google.com Add 'signed' image type examples / docs to cros flash
2018-06-19 dgarrett@google.com chromeos_config: Move toolchain builders to swarming.
2018-06-19 dgarrett@google.com build_status: Remove config_lib.UseBuildbucketScheduler.
2018-06-19 yunlian@google.com cros_gdb: set debug-file-directory for simple chrome.
2018-06-19 vapier@chromium.org pylintrc: convert warnings list into readable ids


Created with:
  gclient setdep -r src/third_party/chromite@9cccc5ce4fb9

The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.



BUG= chromium:853866 , chromium:853391 ,chromium:852017, chromium:853969 , chromium:853025 , chromium:843275 ,chromium:None
TBR=chrome-os-gardeners@chromium.org

Change-Id: I38b6b86c7271b9ca0ec2148faa144816e012ac81
Reviewed-on: https://chromium-review.googlesource.com/1107006
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#568648}
[modify] https://crrev.com/84e07da8461b4b76da32670f411417a117a8340c/DEPS

Comment 11 by jkop@chromium.org, Jun 21 2018

Owner: lhchavez@chromium.org
Status: Started (was: Untriaged)
Cc: lhchavez@chromium.org
Owner: ahass...@chromium.org
let's keep this in ahassani@'s plate. i only did part of the job, but there's still more to do.
Cc: norvez@chromium.org dgarr...@chromium.org
 Issue 804437  has been merged into this issue.
Cc: -lhchavez@chromium.org

Comment 15 by johndhong@chromium.org, Jan 18 (4 days ago)

Cc: englab-sys-cros@google.com

Sign in to add a comment