Upload sha1 of cros images alongside the actual images in google storage |
||
Issue descriptionSo that an image like: gs:///chromeos-image-archive/amd64-generic-full/R66-10401.0.0-b21238/chromiumos_qemu_image.tar.xz would also have a small file at: gs:///chromeos-image-archive/amd64-generic-full/R66-10401.0.0-b21238/chromiumos_qemu_image.tar.xz.sha1 (or something to the same effect). This would be useful for chromium's use case so we can skip the download if the file is already mirrored in our content-addressed object storage.
,
Mar 10 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/db8d32eef11fc3b475fe85b1fcdb530fa39069d6 commit db8d32eef11fc3b475fe85b1fcdb530fa39069d6 Author: Achuith Bhandarkar <achuith@chromium.org> Date: Sat Mar 10 04:05:25 2018 cbuildbot: Generate and upload uploaded.json. uploaded.json contains the filename, size, sha1, and sha256 of all uploaded files. This allows chromium infra to cache large cros binaries with the sha of the binary. BUG= chromium:811989 TEST=unittest Change-Id: I7b0c357a0423df1704ad960980c4b5fe97a278e8 Reviewed-on: https://chromium-review.googlesource.com/956883 Commit-Ready: Achuith Bhandarkar <achuith@chromium.org> Tested-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: Achuith Bhandarkar <achuith@chromium.org> [modify] https://crrev.com/db8d32eef11fc3b475fe85b1fcdb530fa39069d6/cbuildbot/stages/report_stages.py [modify] https://crrev.com/db8d32eef11fc3b475fe85b1fcdb530fa39069d6/cbuildbot/commands.py [modify] https://crrev.com/db8d32eef11fc3b475fe85b1fcdb530fa39069d6/cbuildbot/commands_unittest.py
,
Mar 12 2018
https://pantheon.corp.google.com/storage/browser/chromeos-image-archive/amd64-generic-chromium-pfq/R67-10481.0.0-rc2 There's now an uploaded.json file with filenames, size, sha1, and sha256 hash as expected.
,
Mar 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/6769a99f588dd78ec347e18d2bdbbe74c4a85d4e commit 6769a99f588dd78ec347e18d2bdbbe74c4a85d4e Author: Achuith Bhandarkar <achuith@chromium.org> Date: Tue Mar 13 11:50:13 2018 cbuildbot: Log timing of GenerateUploadJSON. BUG= chromium:811989 TEST=unittest Change-Id: I8a23f59976d9ccff4e9f87617c06a507620074c7 Reviewed-on: https://chromium-review.googlesource.com/959588 Commit-Ready: Achuith Bhandarkar <achuith@chromium.org> Tested-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/6769a99f588dd78ec347e18d2bdbbe74c4a85d4e/cbuildbot/commands.py
,
Mar 13 2018
Great, thanks Achuith! Any idea when it'll start showing up on the full builders on https://build.chromium.org/p/chromiumos? The latest builds don't seem to have it: https://build.chromium.org/p/chromiumos/builders/amd64-generic-full/builds/21476
,
Mar 13 2018
Whoops; nvm. I was looking at the "Artifacts" link, but it's in the bucket: https://pantheon.corp.google.com/storage/browser/chromeos-image-archive/amd64-generic-full/R67-10485.0.0-b2377374
,
Mar 13 2018
Yup, I haven't added it to the index.html file, but maybe I should.
,
Mar 13 2018
all files that get uploaded should be indexed. i thought it was already, but i must have missed something in the CL.
,
Mar 13 2018
Everything in UPLOADED gets indexed. I think we're also missing build-events.json and the UPLOADED file itself. I can add all 3.
,
Mar 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/a22f70d6bd7ca539ab84b36d27ac49af5fa251f4 commit a22f70d6bd7ca539ab84b36d27ac49af5fa251f4 Author: Achuith Bhandarkar <achuith@chromium.org> Date: Thu Mar 15 00:23:16 2018 cbuildbot: Add missing files to index. updated.json, timeline-stages.html, and timeline-slaves.html will also be added to the index. BUG= chromium:811989 TEST=bots Change-Id: Ie4ca847e5e6c2fd85aeb394c54e412590e47cfe4 Reviewed-on: https://chromium-review.googlesource.com/961444 Commit-Ready: Achuith Bhandarkar <achuith@chromium.org> Tested-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: Achuith Bhandarkar <achuith@chromium.org> [modify] https://crrev.com/a22f70d6bd7ca539ab84b36d27ac49af5fa251f4/cbuildbot/stages/report_stages_unittest.py [modify] https://crrev.com/a22f70d6bd7ca539ab84b36d27ac49af5fa251f4/cbuildbot/stages/report_stages.py
,
Mar 16 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b8e7c2d9a192a0e7c441c6f34284fea6ef68dcd5 commit b8e7c2d9a192a0e7c441c6f34284fea6ef68dcd5 Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Fri Mar 16 22:50:01 2018 Roll src/third_party/chromite/ 3b75c9d82..3ad8f333d (31 commits) https://chromium.googlesource.com/chromiumos/chromite.git/+log/3b75c9d82ebf..3ad8f333d567 $ git log 3b75c9d82..3ad8f333d --date=short --no-merges --format='%ad %ae %s' 2018-03-16 dgarrett Revert "Reland "pre_cq_launcher: Swarming for chromeos-infra-puppet-pre-cq."" 2018-03-16 dgarrett Reland "pre_cq_launcher: Swarming for chromeos-infra-puppet-pre-cq." 2018-03-14 ayatane autotest-pre-cq: Remove builder and stage [2/2] 2018-03-16 dgarrett Revert "pre_cq_launcher: Swarming for chromeos-infra-puppet-pre-cq." 2018-03-15 dgarrett chromeos_config: Move fuzzer builds into new bucket. 2018-03-16 dgarrett Revert "commands: RunBranchUtilTest -> RunLocalTryjob" 2018-03-13 dgarrett pre_cq_launcher: Swarming for chromeos-infra-puppet-pre-cq. 2018-02-07 dgarrett commands: RunBranchUtilTest -> RunLocalTryjob 2018-03-14 dgarrett cbuildbot_run: Switch more build links to Legoland. 2018-03-13 dgarrett swarming_lib: Remove SWARMING_TASK_ID from cmds. 2018-03-08 dgarrett moblab_vm_unitest: Fix lint issues. 2018-03-14 ihf chromeos_config: add more arcnext experimental coverage. 2018-03-14 ayatane autotest-pre-cq: Remove this [1/2] 2018-03-14 norvez chromeos_config: remove dead code 2018-03-09 dgarrett summarize_build_stats: Add blank line at beginning. 2018-01-09 dgarrett cros tryjob: Remove buildbot URL generation. 2017-09-14 craigb image_test: Remove check that kernel is not ELF. 2018-03-15 ihf Revert "chromeos_config: temporarily mark eve-arcnext-paladin experimental" 2018-03-15 ihf Revert "chromeos_config: temporarily experimental eve-arcnext-mst-android-pfq" 2018-03-13 lhchavez chromeos_config: Add betty-arcnext builder config 2018-03-13 achuith cbuildbot: Add missing files to index. 2018-03-13 akeshet completion_stages: add a has_important_slave metric to master completion 2018-03-13 dgarrett precq-launcher: Start using Legoland build details page. 2018-03-08 dgarrett chromite-pre-cq: Disable CidbIntegrationTest. 2018-03-14 akeshet chromeos_config: temporarily experimental eve-arcnext-mst-android-pfq 2018-03-13 akeshet chromeos_config: temporarily mark eve-arcnext-paladin experimental 2018-03-12 haddowk [chromite] Make guado_moblab important again 2018-03-13 chrome-bot Update config settings by config-updater. 2018-03-12 gmeinke chromium-config: replace cros_config_host_py in chromite 2018-03-12 yunlian Enable ThinLTO on all AMD64 boards. 2018-03-12 achuith cbuildbot: Log timing of GenerateUploadJSON. Created with: roll-dep src/third_party/chromite BUG=821930, 822517 , 821615 ,None,821618,821227,None,821664,821930,None,815377,747385,461595,821664,821664,811989,819419,821618,820305,821664,821664,819017,813442,707803,811989 The AutoRoll server is located here: https://chromite-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. TBR=chrome-os-gardeners@chromium.org Change-Id: Ib6aaddf338307e994865a092ecb322a432148692 Reviewed-on: https://chromium-review.googlesource.com/967273 Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#543855} [modify] https://crrev.com/b8e7c2d9a192a0e7c441c6f34284fea6ef68dcd5/DEPS
,
Sep 20
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/589b105f17aef2ef64a5812daf3e1fba9b4276a4 commit 589b105f17aef2ef64a5812daf3e1fba9b4276a4 Author: Achuith Bhandarkar <achuith@chromium.org> Date: Thu Sep 20 18:39:45 2018 metadata_lib: Pretty print metadata.json Make partial-metadata.json and metadata.json human readable. BUG= chromium:811989 TEST=None Change-Id: Ic5b35b5f7ba4eb88a747f9fcbcddca3d74499b37 Reviewed-on: https://chromium-review.googlesource.com/959607 Commit-Ready: Achuith Bhandarkar <achuith@chromium.org> Tested-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/589b105f17aef2ef64a5812daf3e1fba9b4276a4/lib/metadata_lib.py |
||
►
Sign in to add a comment |
||
Comment 1 by achuith@chromium.org
, Mar 9 2018Labels: OS-Chrome
Status: Started (was: Assigned)