New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 639308 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug



Sign in to add a comment

Enable BattOr agent for erikchen's power tests

Project Member Reported by charliea@chromium.org, Aug 19 2016

Issue description

His design doc discussing the tests: https://docs.google.com/document/d/1FVwxT21R4p2EffmSlKtnOqfO_UE0VSKF-fvYUgtp08o/edit#

Tracking bug for his tests: https://bugs.chromium.org/p/chromium/issues/detail?id=616811

In the doc, Erik mentioned that he's only seeing 1% noise. We'd really like to see what level of noise we get with BattOr sampling.
 
Specifically, I think we want these enabled for power.trivial_pages. Those are the user stories that he's seeing 1% noise on.
rnephew@, would you be willing to look into doing this next week?
Cc: charliea@google.com
I can look into it; just want to make sure I have this right. Do you want me to turn battor on on his benchmark, or make a seperate battor benchmark with his pageset?
Owner: rnep...@chromium.org
Status: Assigned (was: Untriaged)
Until we are sure the battor tests are stable, I vote to make separate battor. benchmark with his pageset. It also makes sense to separate the power.* benchmarks which people tend to think of as being MSR driven from the new battor ones. One change I would suggest would be to run their pagesets for a shorter period of time. 30s per page is likely overkill to get a stable energy measurement from BattOr.
Ah, that's a fair point: I was thinking of just enabling BattOr tracing for the existing tests, but if we run into serial problems in the middle of a test, then it'll prevent him from getting MSR results too. Given that, I'm with Aaron.
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 24 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6175392bf638b00303602cb7a8895fc59e23fd83

commit 6175392bf638b00303602cb7a8895fc59e23fd83
Author: rnephew <rnephew@chromium.org>
Date: Wed Aug 24 07:49:10 2016

[Telemetry] Add trivial pages page set to BattOr benchmarks.

BUG= 639308 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2261713003
Cr-Commit-Position: refs/heads/master@{#414009}

[modify] https://crrev.com/6175392bf638b00303602cb7a8895fc59e23fd83/tools/perf/benchmarks/battor.py
[modify] https://crrev.com/6175392bf638b00303602cb7a8895fc59e23fd83/tools/perf/page_sets/mac_gpu_sites.py

https://chromeperf.appspot.com/report?sid=9bfb359c83bdd3aa1c2de57afc39d1a21b7e59c6670e899534ac66670558825b&rev=414039

I'm going to keep my eye on this test for a day or 2 and make sure its working well as a battor test then close the bug out.
Eric: can you also keep an eye on the battor based benchmarks to see how well can they reflect regression compared with the msr ones?
For some reason the link you to the perf dashboard isn't working for me. Does this one work for you and Erik?

https://chromeperf.appspot.com/report?sid=13fbfb068540a3ff636ca158664b7901e0e00bfd15ad6c473910c33a6b0cb257
Yeah, I can look at these metrics occasionally and see how they compare against the MSRs

Status: Fixed (was: Assigned)
These tests look pretty stable, I'm going to go ahead and close this.
Project Member

Comment 14 by bugdroid1@chromium.org, Sep 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6380c710d7ee2d6739e563d2957fc7475d663f70

commit 6380c710d7ee2d6739e563d2957fc7475d663f70
Author: rnephew <rnephew@chromium.org>
Date: Thu Sep 01 01:17:59 2016

[BattOr] Add mac BattOr test for steady state cases.

BUG= 640398 ,  638365 ,  639308 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2301653002
Cr-Commit-Position: refs/heads/master@{#415832}

[modify] https://crrev.com/6380c710d7ee2d6739e563d2957fc7475d663f70/tools/perf/benchmarks/battor.py
[modify] https://crrev.com/6380c710d7ee2d6739e563d2957fc7475d663f70/tools/perf/page_sets/idle_after_loading_stories.py

Sign in to add a comment