Clean up paygen/update_engine
Reported by
tbrindus@chromium.org,
Jul 11
|
|||
Issue descriptionThis is a catch-all issue for code cleanups performed as part of implementing crbug.com/794404 .
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/5ec5bd1fe2834ce417d335901d5eb7cc888bf1e3 commit 5ec5bd1fe2834ce417d335901d5eb7cc888bf1e3 Author: Tudor Brindus <tbrindus@chromium.org> Date: Wed Jul 18 04:52:51 2018 brillo_update_payload: Clean up arguments in calls to delta_generator This commit normalizes using -- instead of a mix of - and -- prefixes to delta_generator invocations. delta_generator accepts both. BUG=chromium:862679 TEST=ran commands Change-Id: I5688e8dc11b6d120ae785ebab76dac9f306b845c Reviewed-on: https://chromium-review.googlesource.com/1133706 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/5ec5bd1fe2834ce417d335901d5eb7cc888bf1e3/scripts/brillo_update_payload
,
Aug 3
This bug has an owner, thus, it's been triaged. Changing status to "assigned".
,
Aug 15
,
Nov 30
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/ae7e8f53c6ebd9a360627192142d68d71cfb2c84 commit ae7e8f53c6ebd9a360627192142d68d71cfb2c84 Author: Amin Hassani <ahassani@chromium.org> Date: Fri Nov 30 03:31:16 2018 chroot_util: bring TempDirInchroot from moblab_vm.py Creating a temporary directory in chroot is used in a few places. It's better to just have a utility function to do this. Presenting to you: chroot_util.TempDirInchroot(). Usage: with chroot_util.TempDirInChroot() as tempdir: # do stuff The temp directory will be deleted after the context is exited. BUG=chromium:862679 TEST=precq Change-Id: I18937f6e1fccf6009a56e54ee994f8d08940feeb Reviewed-on: https://chromium-review.googlesource.com/1278087 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/ae7e8f53c6ebd9a360627192142d68d71cfb2c84/lib/moblab_vm.py [modify] https://crrev.com/ae7e8f53c6ebd9a360627192142d68d71cfb2c84/lib/chroot_util.py [modify] https://crrev.com/ae7e8f53c6ebd9a360627192142d68d71cfb2c84/lib/chroot_util_unittest.py [modify] https://crrev.com/ae7e8f53c6ebd9a360627192142d68d71cfb2c84/lib/paygen/paygen_payload_lib.py
,
Dec 10
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/8ea1957b8bd6b58ffd7d04d2f0d2d6ff6823c368 commit 8ea1957b8bd6b58ffd7d04d2f0d2d6ff6823c368 Author: Amin Hassani <ahassani@chromium.org> Date: Mon Dec 10 08:08:51 2018 update_payload: Add the remaining major version 2 signature supports This patch fixes the issues with signatures (sizes) in major version 2 and a few minor issues with the payload and metadata sizes. BUG=chromium:862679 TEST=manually signing the payload and running update_payload_check TEST=unittests Change-Id: I9b431379b0574a150474a913f1ec4a11e86288ae Reviewed-on: https://chromium-review.googlesource.com/1363339 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org> [modify] https://crrev.com/8ea1957b8bd6b58ffd7d04d2f0d2d6ff6823c368/scripts/update_payload/checker_unittest.py [modify] https://crrev.com/8ea1957b8bd6b58ffd7d04d2f0d2d6ff6823c368/scripts/update_payload/checker.py
,
Jan 2
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/bdda5e4967f24c46ad4fbb3708ea80703abdec48 commit bdda5e4967f24c46ad4fbb3708ea80703abdec48 Author: Amin Hassani <ahassani@chromium.org> Date: Wed Jan 02 06:46:17 2019 cros_generate_update_payload: Move utility funtions to paygen There are a few utility functions in the cros_generate_update_payload that are more suited to be moved to chromite/lib/paygen which they belong. Eventually the cros_generate_update_payload will just call a function like GenerateUpdatePayload() from lib/paygen instead of directly forking the delta_generator. The utility functions are mostly related to extracting partitions out of an image. They are moved to a new file lib/paygen/partition_lib.py. Appropriate unittests are also moved along. In addition these functions have been improved and udpated. Specifically, they don't return the (potentially temporary file) path to extracted partition anymore. The path needs to be passed to them. BUG=chromium:862679 TEST=cros flash inside and outside chroot TEST=unittests Change-Id: Ic8cd2eb58f4e451b74250eea6789e5f2a415354a Reviewed-on: https://chromium-review.googlesource.com/1278089 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [add] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/lib/paygen/partition_lib_unittest.py [modify] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/lib/paygen/filelib_unittest.py [add] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/lib/paygen/partition_lib.py [modify] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/scripts/cros_generate_update_payload.py [modify] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/lib/paygen/filelib.py [modify] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/scripts/cros_generate_update_payload_unittest.py [add] https://crrev.com/bdda5e4967f24c46ad4fbb3708ea80703abdec48/lib/paygen/partition_lib_unittest
,
Jan 14
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/4eff412dade110bf4920065034bdc3333ae06e8e commit 4eff412dade110bf4920065034bdc3333ae06e8e Author: Amin Hassani <ahassani@chromium.org> Date: Mon Jan 14 00:30:31 2019 paygen: Move some functions to their appropriate file There are a few functions that are only used in paygen_build_lib, but they are defined in paygen_payload_lib and not all are necessarily paygen related (They are not even used by paygen_payload_lib). So just move them there. Remove _GenerateSinglepayload in favor of CreateAnduploadpayload Moving cache creation in _PaygenPayload. BUG=chromium:862679 TEST=tryjob Change-Id: I1d8cc8b69fd27460bff763d15d753fd40a5d02c6 Reviewed-on: https://chromium-review.googlesource.com/1355466 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Yaakov Shaul <yshaul@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/4eff412dade110bf4920065034bdc3333ae06e8e/lib/paygen/paygen_build_lib.py [modify] https://crrev.com/4eff412dade110bf4920065034bdc3333ae06e8e/lib/paygen/paygen_payload_lib_unittest.py [modify] https://crrev.com/4eff412dade110bf4920065034bdc3333ae06e8e/lib/paygen/paygen_build_lib_unittest.py [modify] https://crrev.com/4eff412dade110bf4920065034bdc3333ae06e8e/lib/paygen/paygen_payload_lib.py
,
Jan 14
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/0041ec14860c99ec294689e2a6d5580b14f12abf commit 0041ec14860c99ec294689e2a6d5580b14f12abf Author: Amin Hassani <ahassani@chromium.org> Date: Mon Jan 14 00:30:31 2019 paygen: replace cros_generate_update_payload with delta_generator In paygen, start using delta_generator directly instead of calling cros_generate_update_payload. This patch includes: - Adding a new function for extracting partitions. - Modifying the list of partitions so they are more flexible and predictable. - And a few minor nits BUG=chromium:862679 TEST=unittests TEST=payload-tryjob Change-Id: I327f37e63458e3b1ede385d6d4d46c7150c6a2c1 Reviewed-on: https://chromium-review.googlesource.com/1369105 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Yaakov Shaul <yshaul@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/0041ec14860c99ec294689e2a6d5580b14f12abf/lib/paygen/paygen_payload_lib_unittest.py [modify] https://crrev.com/0041ec14860c99ec294689e2a6d5580b14f12abf/lib/paygen/paygen_payload_lib.py
,
Jan 14
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/13e98ca519405dd8d6ceecc2b34e505bef84b3c6 commit 13e98ca519405dd8d6ceecc2b34e505bef84b3c6 Author: Amin Hassani <ahassani@chromium.org> Date: Mon Jan 14 21:15:55 2019 paygen: Make _PaygenPayload a public class basically replace _PaygenPayload with PaygenPayload. We need this class be public so we can call it from other places. BUG=chromium:862679 TEST=payload-tryjob TEST=unittests Change-Id: If5e48bff5527110513f1aa7c2746ae6d52d0e873 Reviewed-on: https://chromium-review.googlesource.com/1369106 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Yaakov Shaul <yshaul@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/13e98ca519405dd8d6ceecc2b34e505bef84b3c6/lib/paygen/paygen_payload_lib_unittest.py [modify] https://crrev.com/13e98ca519405dd8d6ceecc2b34e505bef84b3c6/lib/paygen/paygen_payload_lib.py
,
Jan 14
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/3b6b8092e94f9707bd390f232a45cd226cf5abad commit 3b6b8092e94f9707bd390f232a45cd226cf5abad Author: Amin Hassani <ahassani@chromium.org> Date: Mon Jan 14 21:15:56 2019 paygen: Assigning payload and metadata hash files in PaygenPayload Make the metadata and payload hash files visisble (and not function temporarily) in PaygenPayloge so they can be inspected and copied. Needed for --out_metadata_hash_file. BUG=chromium:862679 TEST=payload-tryjob TEST=unittests Change-Id: I12858963f927cad392d088b08a0e6a60aea6200a Reviewed-on: https://chromium-review.googlesource.com/1369107 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Yaakov Shaul <yshaul@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/3b6b8092e94f9707bd390f232a45cd226cf5abad/lib/paygen/paygen_payload_lib_unittest.py [modify] https://crrev.com/3b6b8092e94f9707bd390f232a45cd226cf5abad/lib/paygen/paygen_payload_lib.py
,
Jan 18
(5 days ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/b91590cabc3d770003d347bdb8b0f4112a81f232 commit b91590cabc3d770003d347bdb8b0f4112a81f232 Author: Amin Hassani <ahassani@chromium.org> Date: Fri Jan 18 04:20:30 2019 paygen: Use gspaths.Build as a variable for all Images Currently, the combination of bucket, version, channel, and board are used in URIs for Build, Image, Image archive and their accompanying factory functions (including Payload). However, one can assume that each Image or Payload is associated with a Build, right? So why not instead of using those four variables everywhere in the code, add a 'build' key for them and simplify a lot of things. This allows us to extend the use of Build later for DLCs too easily. This change shortens the code by more than 150 lines and makes it much easier to understand and work with. This also means passing the bucket is mandatory now so we pass the 'chromeos-releases' in the release stage code. BUG=chromium:862679 TEST=payload-tryjob TEST=unittests Change-Id: I88376d0feeec151d38d662ab64ce19b843652c1a Reviewed-on: https://chromium-review.googlesource.com/1292672 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/signer_payloads_client_unittest.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/cbuildbot/stages/release_stages.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/paygen_build_lib_unittest.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/gspaths_unittest.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/paygen_payload_lib_unittest.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/gspaths.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/cbuildbot/stages/release_stages_unittest.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/paygen_build_lib.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/signer_payloads_client.py [modify] https://crrev.com/b91590cabc3d770003d347bdb8b0f4112a81f232/lib/paygen/paygen_payload_lib.py
,
Today
(2 hours ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/a2a8338cefe3d95ad52c6a82985ae952c2db72b9 commit a2a8338cefe3d95ad52c6a82985ae952c2db72b9 Author: Amin Hassani <ahassani@chromium.org> Date: Wed Jan 23 08:20:15 2019 cros_generate_stateful_update_payload: Move to paygen There is only one function which needs to go to paygen instead to be used directly by other callers instead of calling the script itself. BUG=chromium:862679 TEST=cros_generate_stateful_update_payload TEST=unittests Change-Id: I8788f75b11523d0fff3ede525199de24e712f66e Reviewed-on: https://chromium-review.googlesource.com/1403875 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [add] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib_unittest [delete] https://crrev.com/6fd3e070ddcf85122eab4efe94fcf1136c590290/scripts/cros_generate_stateful_update_payload_unittest [add] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib.py [modify] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/scripts/cros_generate_stateful_update_payload.py [rename] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib_unittest.py
,
Today
(2 hours ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/74deb30ac766332c2f116fd9d954b09450bd4764 commit 74deb30ac766332c2f116fd9d954b09450bd4764 Author: Amin Hassani <ahassani@chromium.org> Date: Wed Jan 23 08:20:15 2019 paygen: improve GenrateStatefulPayload's implementation - Instead of directly using tar, use cros_build_lib.CreateTarball() - Use image_lib.LoopbackPartitions instead of MountImageContext. BUG=chromium:862679 TEST=unittests TEST=cros_generate_stateful_update_payload -i chromiumos_test_image.bin -o /tmp/dir Change-Id: I474d1205d95de9022d858b29edbb2eed39eebb2d Reviewed-on: https://chromium-review.googlesource.com/1419081 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/74deb30ac766332c2f116fd9d954b09450bd4764/lib/paygen/paygen_stateful_payload_lib.py [modify] https://crrev.com/74deb30ac766332c2f116fd9d954b09450bd4764/lib/paygen/paygen_stateful_payload_lib_unittest.py
,
Today
(2 hours ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/a2a8338cefe3d95ad52c6a82985ae952c2db72b9 commit a2a8338cefe3d95ad52c6a82985ae952c2db72b9 Author: Amin Hassani <ahassani@chromium.org> Date: Wed Jan 23 08:20:15 2019 cros_generate_stateful_update_payload: Move to paygen There is only one function which needs to go to paygen instead to be used directly by other callers instead of calling the script itself. BUG=chromium:862679 TEST=cros_generate_stateful_update_payload TEST=unittests Change-Id: I8788f75b11523d0fff3ede525199de24e712f66e Reviewed-on: https://chromium-review.googlesource.com/1403875 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [add] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib_unittest [delete] https://crrev.com/6fd3e070ddcf85122eab4efe94fcf1136c590290/scripts/cros_generate_stateful_update_payload_unittest [add] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib.py [modify] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/scripts/cros_generate_stateful_update_payload.py [rename] https://crrev.com/a2a8338cefe3d95ad52c6a82985ae952c2db72b9/lib/paygen/paygen_stateful_payload_lib_unittest.py
,
Today
(2 hours ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/74deb30ac766332c2f116fd9d954b09450bd4764 commit 74deb30ac766332c2f116fd9d954b09450bd4764 Author: Amin Hassani <ahassani@chromium.org> Date: Wed Jan 23 08:20:15 2019 paygen: improve GenrateStatefulPayload's implementation - Instead of directly using tar, use cros_build_lib.CreateTarball() - Use image_lib.LoopbackPartitions instead of MountImageContext. BUG=chromium:862679 TEST=unittests TEST=cros_generate_stateful_update_payload -i chromiumos_test_image.bin -o /tmp/dir Change-Id: I474d1205d95de9022d858b29edbb2eed39eebb2d Reviewed-on: https://chromium-review.googlesource.com/1419081 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/74deb30ac766332c2f116fd9d954b09450bd4764/lib/paygen/paygen_stateful_payload_lib.py [modify] https://crrev.com/74deb30ac766332c2f116fd9d954b09450bd4764/lib/paygen/paygen_stateful_payload_lib_unittest.py |
|||
►
Sign in to add a comment |
|||
Comment 1 by bugdroid1@chromium.org
, Jul 16