New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 650838 link

Starred by 1 user
Status: Fixed
Owner:
jo...@chromium.org
Closed: Sep 2016
Cc:
█ pschmidt@chromium.org
vhang@chromium.org
█ aiolos@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug



Sign in to add a comment

Cloud storage consistently hangs on Catapult Windows CQ buildslave build79-b1

Project Member Reported by aiolos@chromium.org, Sep 27 2016 
Link:
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Windows%20Tryserver

Almost half of the last 25 builds have failed after starting fetching the chrome binary.

Possible causes:

An issue with the binary manager.
An issue with cloud storage.
An issue with starting xvbf when it shouldn't (should only start on Linux)
An infra issue?
Project Member

Comment 1 by bugdroid1@chromium.org, Sep 28 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c7dc335011627cb047b31cadead5ada35b4f773c

commit c7dc335011627cb047b31cadead5ada35b4f773c
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Wed Sep 28 00:45:58 2016

Roll src/third_party/catapult/ 169faf8f9..74b1b4470 (2 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/169faf8f9cbd..74b1b4470c7c

$ git log 169faf8f9..74b1b4470 --date=short --no-merges --format='%ad %ae %s'
2016-09-27 aiolos Add more logging to dev server tests.
2016-09-27 lpy Use histogram for multi event details view.

BUG= 650838 

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2372973004
Cr-Commit-Position: refs/heads/master@{#421401}

[modify] https://crrev.com/c7dc335011627cb047b31cadead5ada35b4f773c/DEPS

Comment 2 by aiolos@chromium.org, Sep 28 2016

Components: Infra>Labs
Summary: Cloud storage consistently hangs on Catapult Windows CQ buildslave build79-b1 (was: win canary dev server tests faililng flakily)
OK, it looks like buildslave build79-b1 consistently hangs while trying to fetch the build from cloud storage.

Failing builds:
https://build.chromium.org/p/tryserver.client.catapult/buildslaves/build79-b1

Log with debug statements where it hung while trying to fetch from cloud storage:
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Windows%20Tryserver/builds/4930/steps/Dashboard%20Dev%20Server%20Tests%20Canary/logs/stdio

Comment 3 by aiolos@chromium.org, Sep 28 2016

Owner: ----
Status: Available (was: Assigned)

Comment 4 by aiolos@chromium.org, Sep 28 2016

Cc: vhang@chromium.org pschmidt@chromium.org

Comment 5 by aiolos@chromium.org, Sep 28 2016

Cc: aiolos@chromium.org

Comment 6 by aiolos@chromium.org, Sep 28 2016 

Could someone check that the buildslave in question has cloud storage configured on it, and there isn't anything wrong with the network connection?

Comment 7 by jo...@google.com, Sep 28 2016 

Don't see any particular network issues currently. Looks like this started on Sep 23 11:00:55 2016 ?

Comment 9 by jo...@google.com, Sep 28 2016 

Going to take the slave offline and have a closer look.

Comment 10 by aiolos@chromium.org, Sep 28 2016 

Yep. Thanks!

Comment 11 by jo...@google.com, Sep 28 2016

Owner: jo...@chromium.org
Status: Assigned (was: Available)

Comment 12 by jo...@google.com, Sep 28 2016

Status: Fixed (was: Assigned)
A clobber of the build dir seems to have fixed this:

https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Windows%20Tryserver/builds/4933

Comment 13 by benhenry@google.com, Feb 2 2017

Components: Speed>Dashboard
Labels: -Performance-Dashboard
►

Sign in to add a comment