New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Last visit 18 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

some canaries are running out of time

Project Member Reported by semenzato@chromium.org, Mar 3 2017

Issue description


Oak-release build 923 timed out.  Earlier and later builds are also close to the 8-hour limit.

https://uberchromegw.corp.google.com/i/chromeos/builders/oak-release/builds/923

So I started playing around with viceroy:

https://viceroy.corp.google.com/chromeos/build_details?build_config=oak-release&build_number=922

More precisely, the master-release:

https://viceroy.corp.google.com/chromeos/build_details?waterfall=chromeos&build_config=master-release&build_number=&milestone_version=&experimental=False&duration=1d&refresh=-1

Failed builds that probably timed out at the most recent build cycles:

caroline-release/453
pyro-release/437
snappy-release/422
reef-release/891

caroline-release/452
pyro-release/436
snappy-release/421
reef-release/890

<none>

pyro-release/434
snappy-release/419
reef-release/888

Other builders seem to have a solid 1-hour of wiggle room.  I wonder if these platforms share the same builder, and if there is something wrong with it.






 
Can we ssh into these builders? It shouldn't be too hard to troubleshoot why they're slow and tune the machines to make them faster...
ok I did a few things to oak-release, we'll see if the builder is faster starting with 925...

Comment 3 by moch@chromium.org, Mar 3 2017

marcheu@ can you list the steps you did. Will help folks in the future.

Comment 4 by oka@chromium.org, Mar 3 2017

Cc: -oka@chromium.org shunhsingou@chromium.org
@3: yeah if/once I have something that works I'll make a list, then I'll try to get it deployed more broadly. For now my changes didn't take because they weren't persistent and builders reboot at each build, so we'll have to wait a little longer for results.
Labels: -current-issue

Comment 7 by moch@chromium.org, Mar 21 2017

Cc: -moch@chromium.org
Is this still happening now?

Comment 9 by xixuan@chromium.org, Jun 20 2017

Status: WontFix (was: Untriaged)
Cc: xixuan@chromium.org
Owner: marc...@chromium.org
Status: Assigned (was: WontFix)
Shall we give Stefan a chance to add any findings he may have collected based on comment #5?
Labels: akeshet-pending-downgrade
ChromeOS Infra P1 Bugscrub.

P1 Bugs in this component should be important enough to get weekly status updates.

Is this already fixed?  -> Fixed
Is this no longer relevant? -> Archived or WontFix
Is this not a P1, based on go/chromeos-infra-bug-slo rubric? -> lower priority.
Is this a Feature Request rather than a bug? Type -> Feature
Is this missing important information or scope needed to decide how to proceed? -> Ask question on bug, possibly reassign.
Does this bug have the wrong owner? -> reassign.

Bugs that remain in this state next week will be downgraded to P2.
Labels: -akeshet-pending-downgrade Pri-2
ChromeOS Infra P1 Bugscrub.

Issue untouched in a week after previous message. Downgrading to P2.
Hi, this bug has not been updated recently. Please acknowledge the bug and provide status within two weeks (6/22/2018), or the bug will be closed. Thank you.
Cc: -shunhsingou@chromium.org

Sign in to add a comment