New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 709532 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner:
Closed: Oct 12
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome , Fuchsia
Pri: 3
Type: Feature



Sign in to add a comment

builders should tar up package build logs as an artifact

Project Member Reported by vapier@chromium.org, Apr 7 2017

Issue description

sometimes people want to look at the output of a package from previous/existing builds and compare it to current ones.  the builders don't show those though unless there's a build failure, so it's not helpful for looking back through history.

basically:
- during archive stage, for each board, create a tarball of /build/$BOARD/tmp/portage/logs/ (path can be obtained via `portageq-$BOARD envvar PORT_LOGDIR`)
- upload that tarball as an artifact
 
Labels: Hotlist-GoodFirstBug
Cc: akes...@chromium.org
Owner: pmalani@chromium.org
Thanks Mike!

Could you kindly point me to the location where the builder scripts reside?
Also some pointer to the borg jobs (if any) that run these scripts.

I can then poke around a bit.

Thanks!
CrOS doesn't use borg anywhere

all builder log is in cbuildbot which is in chromite which is in the root of a Chromium OS checkout.  you most likely want to look at cbuildbot/stages/artifact_stages.py under there.

Comment 5 by sjg@chromium.org, May 24 2017

Cc: sjg@chromium.org
This bug is marked available. Has any work been done?

Comment 6 by sjg@chromium.org, May 24 2017

Cc: bmgordon@chromium.org
Ben, here is one that you could do. It is marked available.

Comment 7 by pmalani@google.com, Jul 26 2017

Hey Mike, Been a while since I looked at this bug. Do I need to run a specific command to be able to mimic the cbuildbot environment and behaviour on my local chroot (to test this out)? Or will a ./build_packages + ./build_image invoke the artifact_stages.py script.

Thanks!

Comment 8 by vapier@chromium.org, Jul 27 2017

`./build_packages --nousepkg` should produce a similar setup to test against
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/8fbc96cc64744b2413b1a62e26b2ba4e49b14230

commit 8fbc96cc64744b2413b1a62e26b2ba4e49b14230
Author: Prashant Malani <pmalani@google.com>
Date: Wed Aug 23 02:05:13 2017

cbuildbot: Add step to archive ebuild logs

BUG= chromium:709532 
TEST=./commands_unittest passes

Change-Id: I340516b3291bd2ee7876770b574396cb6a23b6c9
Reviewed-on: https://chromium-review.googlesource.com/595124
Commit-Ready: Prashant Malani <pmalani@google.com>
Tested-by: Prashant Malani <pmalani@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/8fbc96cc64744b2413b1a62e26b2ba4e49b14230/cbuildbot/stages/artifact_stages.py
[modify] https://crrev.com/8fbc96cc64744b2413b1a62e26b2ba4e49b14230/cbuildbot/commands.py
[modify] https://crrev.com/8fbc96cc64744b2413b1a62e26b2ba4e49b14230/cbuildbot/commands_unittest.py

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/ace2aa28daf7c91179d3097ab91afc98287e5987

commit ace2aa28daf7c91179d3097ab91afc98287e5987
Author: Prashant Malani <pmalani@google.com>
Date: Thu Aug 24 00:01:23 2017

cbuildbot: Add EbuildLogs step before RunParallelSteps

BUG= chromium:709532 
TEST=./commands_unittest passes

Change-Id: I65a0e249e4fcc0ce16d58efd9649fbea4aed21a7
Reviewed-on: https://chromium-review.googlesource.com/628136
Commit-Ready: Prashant Malani <pmalani@google.com>
Tested-by: Prashant Malani <pmalani@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/ace2aa28daf7c91179d3097ab91afc98287e5987/cbuildbot/stages/artifact_stages.py

Project Member

Comment 11 by bugdroid1@chromium.org, Aug 30 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/788aa03d950f28fc39ba351bf869b9d29dff6610

commit 788aa03d950f28fc39ba351bf869b9d29dff6610
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Aug 30 20:04:22 2017

cbuildbot: ArchiveEbuildLogs: fix upload queue usage

This is not a release artifact, so don't use the release queue.

BUG= chromium:709532 
TEST=unittests pass

Change-Id: I896103e4dd3cc4870009be48f9e70f9d0525ed79
Reviewed-on: https://chromium-review.googlesource.com/636270
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Prashant Malani <pmalani@google.com>

[modify] https://crrev.com/788aa03d950f28fc39ba351bf869b9d29dff6610/cbuildbot/stages/artifact_stages.py

Project Member

Comment 12 by bugdroid1@chromium.org, Sep 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/8cd6ca3c386d12449e32fe55a3e7368961f492ce

commit 8cd6ca3c386d12449e32fe55a3e7368961f492ce
Author: Mike Frysinger <vapier@chromium.org>
Date: Tue Sep 12 11:19:30 2017

cbuildbot: ArchiveStage: sanity check release queue misuse

If code that runs in the non-release steps posts a file to the release
queue, it might be uploaded anyways, or silently ignored (depending on
how the race plays out).  Add a sanity check to the release queue so,
if the file would have been ignored, we at least abort.

BUG= chromium:709532 
TEST=unittests pass

Change-Id: I7c449ba772d0cd85ffc61f1696d170b164f4bf59
Reviewed-on: https://chromium-review.googlesource.com/636271
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Prashant Malani <pmalani@google.com>

[modify] https://crrev.com/8cd6ca3c386d12449e32fe55a3e7368961f492ce/cbuildbot/stages/artifact_stages.py

Components: Infra>Client>ChromeOS>CI
Components: -Infra>Client>ChromeOS
Status: Assigned (was: Available)
Labels: OS-Fuchsia
Status: WontFix (was: Assigned)
I'm closing this since there's no strong need for this ATM.

Sign in to add a comment