New issue
Advanced search Search tips

Issue 661047 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

test suites w/ patch not displayed in testType selector for android try bots

Project Member Reported by jbudorick@chromium.org, Nov 1 2016

Issue description

^

e.g., there are options for "android_webview_test_apk" and "android_webview_test_apk:", but not for "android_webview_test_apk (with patch)" despite JSONs being uploaded for that: http://test-results.appspot.com/testfile?builder=linux_android_rel_ng&testtype=android_webview_test_apk%20(with%20patch)
 
Status: Available (was: Untriaged)
The odd colon test type is derived from this step name:
https://uberchromegw.corp.google.com/i/chromium.android/builders/KitKat%20Tablet%20Tester/builds/5907/steps/android_webview_test_apk%3A%20generate%20result%20details
That's a bug.

The list is derived from a single scraped build from buildbot, e.g. https://uberchromegw.corp.google.com/i/tryserver.chromium.android/builders/linux_android_rel_ng/builds/172102.

It seems that no tests ran because of analyze, which can happen sometimes. It's probably the case that this is why the test list doesn't include them. That's also a bug.

A good fix to the second one is to stop using buildbot scraping as the source of truth. The fix just requires querying datastore for the results we have and using that to generate the dropdown list. The query would be something like:
"select * from TestFile where name = 'results-small.json' and date > datetime('2016-10-24T00:00:00.000000Z')"

and would need to be served in a format like test-results.appspot.com/builders and that wouldn't require any client side changes. It would probably be a few hours of work.
re how that list is generated: 😱
Summary: test suites w/ patch not displayed in testType selector for android try bots (was: instrumentation test suites w/ patch not displayed in testType selector)
So what's the next step here? Who's expected to act on this...? After looking for other results this week, it seems that this makes the flakiness dashboard unusable for the android try bots (at least).
Also see issue 663713. If we allow searching by test name alone, this issue will become less urgent, but the fix suggested by Erik is probably much faster than fixing the issue 663713.
Project Member

Comment 5 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/ba0044d3eac27f216508c461ca615436f3892b2f

commit ba0044d3eac27f216508c461ca615436f3892b2f
Author: Erik Staab <estaab@google.com>
Date: Thu Dec 01 14:01:21 2016

test-results: Remove dependency on buildbot / milo.

This changes the way we generate the list of known (master, builder, test)
tuples to use what has recently been uploaded instead of scraping step names
from recent passing buildbot builds.

The benefits to doing this are:
1) No more outages from empty builder data caused by milo queries that
timeout from being too large.
2) The javascript client will now only fetch results for combinations that
exist instead of an approximate set.
3) New test combinations will be able to upload and serve without
needing to be whitelisted.

BUG= 666889 ,659952, 655967 , 666810 , 661047 , 643944 , 405799 

Change-Id: Icd0a8a88eecbd920677f704cf816b143e7437e50
Reviewed-on: https://chromium-review.googlesource.com/413142
Commit-Queue: Erik Staab <estaab@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>

[modify] https://crrev.com/ba0044d3eac27f216508c461ca615436f3892b2f/appengine/test_results/index.yaml
[delete] https://crrev.com/fb6cd3035cc307a93948a87be99320e24ece75c0/go/src/infra/appengine/test-results/buildextract/buildextract.go
[delete] https://crrev.com/fb6cd3035cc307a93948a87be99320e24ece75c0/go/src/infra/appengine/test-results/buildextract/buildextract.infra_testing
[delete] https://crrev.com/fb6cd3035cc307a93948a87be99320e24ece75c0/go/src/infra/appengine/test-results/buildextract/buildextract_test.go
[delete] https://crrev.com/fb6cd3035cc307a93948a87be99320e24ece75c0/go/src/infra/appengine/test-results/buildextract/testing.go
[delete] https://crrev.com/fb6cd3035cc307a93948a87be99320e24ece75c0/go/src/infra/appengine/test-results/buildextract/testing_test.go
[modify] https://crrev.com/ba0044d3eac27f216508c461ca615436f3892b2f/go/src/infra/appengine/test-results/cron.yaml
[modify] https://crrev.com/ba0044d3eac27f216508c461ca615436f3892b2f/go/src/infra/appengine/test-results/frontend/builders.go
[modify] https://crrev.com/ba0044d3eac27f216508c461ca615436f3892b2f/go/src/infra/appengine/test-results/frontend/builders_test.go
[modify] https://crrev.com/ba0044d3eac27f216508c461ca615436f3892b2f/go/src/infra/appengine/test-results/frontend/handlers.go

Owner: estaab@chromium.org
Status: Fixed (was: Available)
Looks correct now:
http://test-results.appspot.com/testfile?builder=linux_android_rel_ng&testtype=android_webview_test_apk%20(with%20patch)
:D

Sign in to add a comment