This flag should only be applied for system_health & power harness, I think?

Probably we will want it for media.* benchmarks as well.

Yeah, pretty much anything that reports power, which I think will include webrtc as well, but i haven't kept up with their plans.

I'm concerned about our approach to this.

Right now, power benchmarks are our main proxy to ensuring that Chrome is using CPU efficiently. If we start running media, system_health, and all power benchmarks as if they're on battery power, we're essentially saying that we don't care about CPU use in all other cases. However, if Chrome uses up 100% of the CPU whenever it's not on battery power, this clearly creates a negative user experience. How do we make sure that we're not consuming too much CPU when the computer's _not_ running on battery? How do we do this in a way where we're not running every benchmark twice, once with --force-on-battery and once without?

What about using some sort of pairwise testing approach on the bots?

* For bots with BattOrs, run as if on Battery
* For bots without BattOrs, run as if plugged in

+benhenry who's been looking into what hardware we need on the perf waterfall, to see how many configs we'd want CPU usage tested on.

>  However, if Chrome uses up 100% of the CPU whenever it's not on battery power, this clearly creates a negative user experience. How do we make sure that we're not consuming too much CPU when the computer's _not_ running on battery?

This is a good concern. Now I am leaning toward only enabling "--force-on-battery" flag for power harness, which the whose purpose is for benchmarking Chrome power only. We should only switch all benchmarks to "--force-on-battery" if that increases realism (most users are using CHrome on battery) & reliablity ( I suspect the opposite is true).

I don't like the proposed approach in #6, because that creates inconsistency between bots based on the presence of Battor, which is surprising. It's very easy to forget about such config & equipping Battors on 90% of the bots, which makes the 90% of benchmarks in the labs having  --force-on-battery unintentionally.

I see a couple of potential problems with that approach:

1) It'd be really hard to answer questions like "do we use more CPU when we're not running on battery", because we wouldn't have the same metric collected on the same configuration for both battery and AC power
2) Over time, we should have fewer non-BattOr devices, which will decrease our AC power coverage even if we don't want that

There's also a huge and obvious upside to this approach, though, in that it's easy and obvious, which may outweigh either of those two drawbacks. I'm interested in what benhenry@'s opinion is here.

(just saw that nednguyen@ commented after sullivan@'s comment - my comment was in response to sullivan@'s)

*It's not just about CPU metrics, but also about other metrics like smoothness, time to first meaningful paint, etc.. 

For example, people may decide to throttle rendering when CHrome is on battery, so by switching system_health.common_* & video benchmarks to "--force-on-battery", we lose the coverage of those rendering/loading metrics non-throttled Chrome, which seems bad.

I could see us doing one of two things:
- run benchmarks on battery in parallel to on DC - despite Charlie's concerns, this might just mean we need more bot coverage to deal with affinity, etc.
- Decide that the telemetry lab is useful for catching regressions, find that on battery has an insignificant regression potential as compared to on DC, realize that we're using WPR and not testing live things, but we're testing very controlled scenarios for the purpose of finding CLs that cause regressions and state that as our official opinion for 2017.

Ben, your point #2 might be my preferred avenue, along with "be conscientious to limit the scenarios where on-battery differs from plugged-in". The only one I can think of right now is that we change the timer frequency.

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot