Yes, I agree. I think that all benchmarks not using the system health stories should move out of system_health.py and vice versa.

Another alternatives are:
i)  add the blank page as part of system health & have webview using it, 
ii) have webview benchmark use any of the more contentful system health page we have.

I think the core question here is whether a benchmark that capture "startup metrics of webview" belong to system-health team. I am leaning to say "yes" here.

I think that it makes sense to include the benchmark in System Health only if we do both (i) and (ii). In my opinion, adding about:blank to the SH story set and then using only that single story in the benchmark is a very hacky way to include it in SH. Having said that, I completely agree with having about:blank in the SH story set.

The reason why the benchmark has "system health" on its name is because the "startup" metric is important. And we intend to track it as part of (android) system health plan.

In my mind the solution should be:
(1) add about:blank as a SH story. (good to have, but actually ortogonal to the main issue)
(2) add "startup" as one of the metrics collected when we run system health benchmarks (in particular for webview); probably as part of the "non-memory" mobile system health benchmark.

perezju: I agree with everything you said.

Sorry, just want to make sure we close the loop on this: keeping this as-is for now is okay, and when we have an about:blank SH loading story and a startup metric, we'll get this for free and we can delete this story?

I think that just happened here: https://codereview.chromium.org/2307873003/

I'm not sure: it seems like that code review adds a second webivew startup metric that's pretty much the same as the first: just load about:blank. I thought that the plan was to do something like add about:blank to the list of loading stories, enable system_health.common_mobile on the Webview bots, and add the webview startup metric to the list of metrics that we compute?

In the current state, we're not collecting the webview startup metric on any "real" sites, whereas in the state we talked about before, we would be.

charliea: You're right, this should be merged into system_health.common_*. I thought that we were already running the benchmark on WebView, but I can't find any system_health.common_* data on the dashboard. Does anybody know why?

Yep, that was my fault. I forgot to add any metrics to the benchmark. I'll shoot you a CL momentarily that does so.

Re #8: Ah, my bad. Yes. For some reason I thought we were closer to the desired end state.

If I understand correctly the "final" plan is to have:

  system_health.memory_mobile
  system_health.mobile ("all non-memory metrics")

and similar for desktop.

Could the existing battor.system_health_loading_mobile be turned into that desired "system_health.mobile", and add the startup metric to be tracked there? How close would we be to make that happen?

Ah, I've just seen  issue 637158 , and the associated CLs, so looks we're nearly getting there!

Hey Alex,

Could you take a look at this when you have a chance?