New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 809401 link

Starred by 1 user
Status: Assigned
Owner:
oprypin@chromium.org
Cc:
█ srte@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Make pass through client.webrtc.perf and make sure there's only ONE slave/phone per perf bot

Project Member Reported by srte@chromium.org, Feb 6 2018 
Might be due to change in the devices used.

Device status before alert seems to be constant as:
flounder LMY47M HT593JT00040
HT4ADJT03192: adb status offline
HT4AEJT01652: missing

At the alert it is:
flounder LMY47M HT593JT00040
HT4ADJT03192: blacklisted
HT4AEJT01652: blacklisted

And after the alert it is:
flounder LMY47M HT593JT00040
flounder LMY47M HT4ADJT03192
flounder LMY47M HT4AEJT01652
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Feb 6 2018 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=809401

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=665dea74a041dcdfc824872cb95a10cd6af7a4929c94bf95f943bdd36c15c955


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus9

Comment 2 by srte@chromium.org, Feb 7 2018

Summary: Regression in webrtc_perf_tests at 21861:21863, possibly due to bot status. (was: 5.5%-85.8% regression in webrtc_perf_tests at 21861:21863)

Comment 3 by phoglund@chromium.org, Feb 12 2018

Owner: ehmaldonado@chromium.org
You are right. I'm not entirely sure why that happened.

We can see the numbers go back again and now we have

HT593JT00040: missing
flounder LMY47M HT4ADJT03192
HT4AEJT01652: missing

After all this we ended up switching from HT593JT00040 to HT4ADJT03192.

Even having more than one device attached to a performance test bot sounds wrong to me. What's the idea behind the configuration here, Edward?

Comment 4 by ehmaldonado@chromium.org, Feb 12 2018 

Yeah, it sounds wrong to me too.
The reason is that it has always been like that, and I think last time we discussed it, we decided to leave it as it was to make tests run faster.

There is also a Builder that has two slaves connected to it.
Maybe we should retire all Android devices except one in each bot?

Comment 5 by phoglund@chromium.org, Feb 14 2018

Owner: oprypin@chromium.org
Summary: Make pass through client.webrtc.perf and make sure there's only ONE slave/phone per perf bot (was: Regression in webrtc_perf_tests at 21861:21863, possibly due to bot status.)
Ok, we should remove those then. It will make the bot more sensitive but it will at least produce correct results. I'll repurpose this bug for that task, then.

Edward says there might be more incorrectly configured perf bots, and we should fix those too (phones is one case, perf bots with multiple backing slaves is another).

Oleh, this is tangentially related to the waterfall restructuring so I'll give you this one.
►

Sign in to add a comment