New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 665636 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

android fyi perf bot frequently fails to get build products

Project Member Reported by rnep...@chromium.org, Nov 15 2016

Issue description


https://build.chromium.org/p/chromium.perf.fyi/builders/Android%20Power%20Nexus%205X%20Perf%20(1)

Nov 15 15:01	1bc2f0135011d68b7876c3ff5bf645d14ead0853	exception	#3289
Exception steps exception gsutil download_build_product
Nov 15 14:49	f0948a8c21aa053e51c7a864d1230470e792c928	exception	#3288
Exception steps exception gsutil download_build_product
Nov 15 14:37	c2a979f3c638e45d6481c68d0cea907d63bb61cb	exception	#3287
Exception steps exception gsutil download_build_product
Nov 15 14:25	37f82a57591c0da2629a9df2b49baf9428980bca	exception	#3286
Exception steps exception gsutil download_build_product
Nov 15 14:14	039733bf89cd1df6636f973bbb1750df609132a4	exception	#3285
Exception steps exception gsutil download_build_product
Nov 15 13:29	8c16443d571c056044eece9afb1066308fb0c593	retry	#3284
Build successful exception slave lost exception sharded perf tests
Nov 15 13:17	8331f8ad9053ee1effd499ad29b5a806d9959ef4	exception	#3283
Exception steps exception gsutil download_build_product
Nov 15 13:05	1a4d73beb118fed32cc6c53c78ec236e51a0ff32	exception	#3282
Exception steps exception gsutil download_build_product
Nov 15 12:54	df30c1872d899957659f5422930a3ad4a5585d91	exception	#3281
Exception steps exception gsutil download_build_product
Nov 15 12:42	be82f91e1f87efcdb4328cb463db5359accb7e73	exception	#3280
Exception steps exception gsutil download_build_product
Nov 15 12:31	4a94f7a4a9055e7b919c4c3ef9604324c75a70f5	exception	#3279
Exception steps exception gsutil download_build_product
Nov 15 12:24	1cd87c1fd8b685edcd9b053def59a5d3f7b26056	exception	#3278
Exception steps exception gsutil download_build_product
Nov 15 12:15	??	retry	#3277
Build successful exception slave lost exception bot_update
Nov 15 12:03	98afc7db00a7bfe749914e4d618ccb9cc76cbb97	exception	#3276
Exception steps exception gsutil download_build_product
Nov 15 11:51	a816bfc268006458526fb07e84ac1e03cd1ac03f	exception	#3275
Exception steps exception gsutil download_build_product
Nov 15 11:39	77cea01dec899643363c19f65e7dfff0e1281434	exception	#3274
Exception steps exception gsutil download_build_product
Nov 15 11:28	657a3d6a6cfeaf22ce948ad22f98ab19f7f54a80	exception	#3273
Exception steps exception gsutil download_build_product
Nov 15 11:15	b86938891ce73c4d38afb9207fab2976d01cfcfa	exception	#3272
Exception steps exception gsutil download_build_product
Nov 15 10:53	3a2dc3f49c57af52b9d3509bd6bcf047d38326f4	exception	#3271
Exception steps exception gsutil download_build_product
Nov 15 10:42	9cbb0b1f01687665a716a8797243723fab7bf71d	exception	#3270
Exception steps exception gsutil download_build_product
Nov 15 10:30	c764501f3bc7b6a5494d3b2da3bf13d43ee36bea	exception	#3269
Exception steps exception gsutil download_build_product
Nov 15 10:19	28cb08e7ce61cfb7fceae6cf6490df8b94ae01a1	exception	#3268
Exception steps exception gsutil download_build_product
Nov 15 10:07	c69b1761b06733edcb450949ce528f8646b0bc75	exception	#3267
Exception steps exception gsutil download_build_product
Nov 15 09:55	d53b87f3c2b57138ed109357b23abaeb7c473cf8	exception	#3266
Exception steps exception gsutil download_build_product
Nov 15 09:43	8527f94948f900202c93e815ce739f47a25130ad	exception	#3265
Exception steps exception gsutil download_build_product
Nov 15 09:31	737a80cd1ba84e6c60347b0950f679bcb6eef57f	exception	#3264
Exception steps exception gsutil download_build_product
Nov 15 09:16	168ffcf4f0f154cc52d7428f18e90b7d2c5ffb73	exception	#3263
Exception steps exception gsutil download_build_product
Nov 15 09:04	3ed22b610c31300b8c2746e1beb214a571d9f61b	exception	#3262
Exception steps exception gsutil download_build_product
Nov 15 08:53	939eda5b40ee1be9ad0c696c28d518a9fa5dc3ea	exception	#3261
Exception steps exception gsutil download_build_product
Nov 15 08:41	ded8b5b45e962d07a377d40e35448f49ff83b718	exception	#3260
Exception steps exception gsutil download_build_product
Nov 15 08:30	2064c3247e03ad89688194f7bd4fa79479363644	exception	#3259
Exception steps exception gsutil download_build_product
....

Looks like that for the whole 200 with some making it through that step.

martiniss@ if you are not the correct person to own the problem with download_build_product could you route it to someone who is or let me know I should find someone else?
 
Cc: dtu@chromium.org eyaich@chromium.org
Status: Started (was: Untriaged)
I'm going to convert the builder to the chromium recipe, and then make it work again.
Labels: BattOr
Project Member

Comment 5 by bugdroid1@chromium.org, Nov 22 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager.git/+/1999df2d56842b85340432a1f90f4653ec281fd0

commit 1999df2d56842b85340432a1f90f4653ec281fd0
Author: Stephen Martinis <martiniss@google.com>
Date: Tue Nov 22 21:18:23 2016

Ok, bot is running chromium. Now I need to make it actually work....
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/b9fa421c6abca1f5f956e3ab763c8915a3b6e8bb

commit b9fa421c6abca1f5f956e3ab763c8915a3b6e8bb
Author: Stephen Martinis <martiniss@google.com>
Date: Tue Nov 22 23:12:56 2016

Remove triggering for android/builder on chromium.perf

The android bot on chromium.perf.fyi is now on the chromium recipe, so
it shouldn't be triggered by the android/builder recipe.

BUG= 665636 

Change-Id: I454e1805875c90aed48946ba52b9f7355cbf0960
Reviewed-on: https://chromium-review.googlesource.com/413631
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/b9fa421c6abca1f5f956e3ab763c8915a3b6e8bb/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_Builder.json
[modify] https://crrev.com/b9fa421c6abca1f5f956e3ab763c8915a3b6e8bb/scripts/slave/recipes/android/builder.py

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/7fe393e699d4dc823a37b8afa3163698e08202a8

commit 7fe393e699d4dc823a37b8afa3163698e08202a8
Author: Stephen Martinis <martiniss@google.com>
Date: Wed Nov 23 00:09:19 2016

Fix perf.fyi Android Power Nexus name

Remove the (1) at the end. That is there from the android/perf recipe,
which always adds a suffix. The chromium recipe doesn't need a suffix.

This should fix triggering on the builder.

BUG= 665636 

Change-Id: Ia5eab5be6a1f341cd34ef0aa3c9671214a97b6be
Reviewed-on: https://chromium-review.googlesource.com/413714
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/7fe393e699d4dc823a37b8afa3163698e08202a8/masters/master.chromium.perf.fyi/master.cfg
[modify] https://crrev.com/7fe393e699d4dc823a37b8afa3163698e08202a8/masters/master.chromium.perf.fyi/slaves.cfg

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 23 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager.git/+/af8be645c4c3c154fd624925dfb5df5ea849a351

commit af8be645c4c3c154fd624925dfb5df5ea849a351
Author: Stephen Martinis <martiniss@google.com>
Date: Wed Nov 23 01:33:19 2016

Project Member

Comment 10 by bugdroid1@chromium.org, Nov 23 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager.git/+/af8be645c4c3c154fd624925dfb5df5ea849a351

commit af8be645c4c3c154fd624925dfb5df5ea849a351
Author: Stephen Martinis <martiniss@google.com>
Date: Wed Nov 23 01:33:19 2016

OK, it's running the chromium recipe now. https://build.chromium.org/p/chromium.perf.fyi/builders/Android%20Power%20Nexus%205X%20Perf/builds/0 was the first build. 

It failed, and I'm not sure why. It looks like it didn't do the sharded perf tests step?
It thinks it's not an android test, because the chromium_tests config is only set for 1 device shard. CL out!
Cc: alexandermont@chromium.org
This is being seen on main perf waterfall android bots now.
I think it's a different problem. I'll take a look though.
The main perf waterfall android bots are failing because the build per revision bots are backed up. I'll fix two of the bots, which seemed to have gone offline. Hopefully that should help.

There is a CL out which should hopefully fix this bug (for perf.fyi): https://chromium-review.googlesource.com/c/414208/
The perf waterfall got restarted, so things should be in a better state. 
Project Member

Comment 17 by bugdroid1@chromium.org, Dec 1 2016

Status: Fixed (was: Started)
Labels: Performance-Power

Sign in to add a comment