New issue
Advanced search Search tips

Issue 838000 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: May 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
Build-Toolchain



Sign in to add a comment

Uploaded generated toolchains and SDK as artifacts in chromiumos-sdk builder

Project Member Reported by manojgupta@chromium.org, Apr 29 2018

Issue description

Issues related to chromiumos-sdk builder can be sometimes hard to test since
1. Tryjobs do not upload artifcats.
2. While Production builds uploads the artifacts as prebuilts, this step happens only when everything passes. If something fails, the artifacts are not available for testing.

Uploading the toolchains and SDK will allow to test the standaAlone toolchains and also help in debugging generated SDK issues.
 
Project Member

Comment 1 by bugdroid1@chromium.org, May 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/394dddbed8e1e8d79e9ec74048549b97311d11ea

commit 394dddbed8e1e8d79e9ec74048549b97311d11ea
Author: Manoj Gupta <manojgupta@google.com>
Date: Tue May 01 03:06:38 2018

cbuildbot: Upload built toolchains and SDK as artifacts.

It is often useful to have access to the toolchains and SDK
built in the SDK builder for local testing. So upload these
as part of build artifacts. This makes it easier
to verify/repro a toolchain issue locally.

BUG= chromium:838000 
TEST=Unit tests pass.
TEST=SDK/Toolchains are uploaded to gs://chromeos-image-archive/<name>/<build#>.

Change-Id: I5431a67b0e74a0798ae8aa77679c3e623ac5d209
Reviewed-on: https://chromium-review.googlesource.com/1034167
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>

[modify] https://crrev.com/394dddbed8e1e8d79e9ec74048549b97311d11ea/cbuildbot/stages/sdk_stages.py
[modify] https://crrev.com/394dddbed8e1e8d79e9ec74048549b97311d11ea/cbuildbot/stages/sdk_stages_unittest.py

Project Member

Comment 2 by bugdroid1@chromium.org, May 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4e6e75bb83a0fca04ef6799d5bba9535a0cc4cab

commit 4e6e75bb83a0fca04ef6799d5bba9535a0cc4cab
Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue May 01 05:56:14 2018

Roll src/third_party/chromite/ cbf8394d4..394dddbed (1 commit)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/cbf8394d4a7b..394dddbed8e1

$ git log cbf8394d4..394dddbed --date=short --no-merges --format='%ad %ae %s'
2018-04-28 manojgupta cbuildbot: Upload built toolchains and SDK as artifacts.

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:838000 


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: Ic7aef4a004e18e621f259568358886776aebd830
Reviewed-on: https://chromium-review.googlesource.com/1036848
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#555003}
[modify] https://crrev.com/4e6e75bb83a0fca04ef6799d5bba9535a0cc4cab/DEPS

the SDK is not a tiny file.  uploading these all the time will increase overhead not insignificantly.  can you restrict this to trybots ?
It was actually intentional that both tryjobs/production jobs upload the artifacts. 
Otherwise, reproing an issue will mean launch a tryjob and wait for another 7-10 hours before artifacts are available for testing.

I did check the increase in build times but apparently it is less than 2 minutes since only standalone toolchains and SDK tarballs are uploaded (not all host packages). So, not worrying about it for now.
production jobs already upload the artifact to the SDK bucket, so it's spending time & network bandwidth (shared among systems) & GS storage on useless files

if your concern is for the production job to upload it sooner, we already have a bug open about that
Ok, once the other bug about uploading prebuilts early is fixed, I will be happy to remove the artifact uploading from production jobs.
Status: Verified (was: Untriaged)

Sign in to add a comment