New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 841993 link

Starred by 2 users
Status: Fixed
Owner:
bpastene@chromium.org
Closed: May 2018
Cc:
ynovikov@chromium.org
█ fjhenigman@chromium.org
bpastene@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug-Regression



Sign in to add a comment

capacity problem for angle unit tests on gpu fyi

Project Member Reported by rjkroege@chromium.org, May 10 2018 
The angle tests on the GPU FYI bots are failing with a capacity problem. Snippet:

+------------------------------------------------------------------------+
| Shard 0  https://chromium-swarm.appspot.com/user/task/3d64b9b9b5fd8010 |
+------------------------------------------------------------------------+
+------------------------------------------------------------------------+
| End of shard 0                                                         |
|  Pending: 3609.2s  EXPIRED (lack of capacity)                          |
+------------------------------------------------------------------------+
WARNING:root:collect_cmd had non-zero return code: 1
WARNING:root:No shard json files found in task_output_dir: '/b/swarming/w/ir/tmp/tmpH4YdeG'
Found ['/b/swarming/w/ir/tmp/tmpH4YdeG/summary.json']
There wasn't enough capacity to run your test
some shards did not complete: 0

bots: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%2032%20Vk%20Release%20%28Nexus%205X%29/3075, https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%2064%20Vk%20Release%20%28Nexus%205X%29/2617

Comment 1 by rjkroege@chromium.org, May 10 2018

Cc: fjhenigman@chromium.org
Labels: -Pri-3 Pri-2
fjhenigman@: Can you take a look and confirm that this is actually some kind of infra issue?

Comment 2 by kbr@chromium.org, May 10 2018

Cc: ynovikov@chromium.org
+ynovikov who deployed these bots.

Comment 3 by kbr@chromium.org, May 10 2018

Components: -Infra Infra>Labs
Labels: -Type-Bug -Pri-2 Pri-1 Type-Bug-Regression
Per https://chromium-swarm.appspot.com/task?id=3d64badc6e926d10&refresh=10&show_raw=1 : 

"18 bots could possibly run this task (0 busy, 18 dead, 0 quarantined, 0 maintenance)".

Labs team: could you please check on these bots?
https://chromium-swarm.appspot.com/botlist?c=id&c=os&c=task&c=status&f=device_os%3AO&f=os%3AAndroid&f=pool%3AChrome-GPU&f=device_type%3Abullhead&l=100&s=id%3Aasc

They're:

build251-m4--device1
build251-m4--device3
build251-m4--device4
build251-m4--device5
build251-m4--device6
build251-m4--device7
build252-m4--device2
build252-m4--device4
build252-m4--device5
build252-m4--device6
build254-m4--device6
build254-m4--device7
build260-m4--device6
build262-m4--device3
build262-m4--device6
build263-m4--device2
build263-m4--device3
build263-m4--device7


Thanks.

Comment 4 by ynovikov@chromium.org, May 11 2018 

Note: these bullhead devices should have OPM1.171019.011 os.

Comment 5 by ynovikov@chromium.org, May 11 2018

Cc: bpastene@chromium.org
I think this has something to do with 1bead776 Bot Version upgrade.

Comment 6 by bpastene@chromium.org, May 11 2018

Owner: bpastene@chromium.org
Status: Started (was: Untriaged)
Ah whoops. I took these offline for maintenance in b/78302027 and forgot a few bots on there belong to GPU's pool. Apologies; I'll work on getting them back online.

Comment 8 by bpastene@chromium.org, May 11 2018

Status: Fixed (was: Started)
They're all back:
https://chromium-swarm.appspot.com/botlist?c=id&c=os&c=task&c=status&f=pool%3AChrome-GPU&f=device_os%3AOPM1.171019.011&l=100&q=device_os%3AOPM1.171019.011&s=id%3Aasc

Sorry again for the trouble. As a bonus, the pool got quite a few more devices during the hardware refresh. Hopefully that smooths it over a little ;)

Comment 9 by ynovikov@chromium.org, May 11 2018 

Thanks, Ben!
►

Sign in to add a comment