browse:news:toi seems to fail very often at Android One for the last 200+ builds. Looping in benchmark owners to triage.

mythria@, can you please take a look?

It always fails with this error: "Error: Exceptions raised when trying to stop tracing:"

May be it is related to tracing? Cc'ing people from telemetry. Toi works on system_health.common_mobile though.

Stack Trace:
Error: Exceptions raised when trying to stop tracing:
Traceback (most recent call last):
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 128, in StopTracing
    agent.CollectAgentTraceData(builder)
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 248, in CollectAgentTraceData
    '\n'.join(raised_exception_messages))
ChromeTracingStoppedError: Exceptions raised when trying to collect Chrome devtool tracing:
Error when collecting Chrome tracing on devtools at port localabstract:chrome_devtools_remote:
Traceback (most recent call last):
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 237, in CollectAgentTraceData
    client.CollectChromeTracingData(trace_data_builder)
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 371, in CollectChromeTracingData
    self._tracing_backend.CollectTraceData(trace_data_builder, timeout)
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 230, in CollectTraceData
    self._CollectTracingData(trace_data_builder, timeout)
  File "/b/c/b/Android_One_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 258, in _CollectTracingData
    traceback.format_exc())
TracingUnrecoverableException: Exception raised while collecting tracing data:

It looks like the page is not loading properly:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_6-2017-05-16_01-10-0196298.png

Also that error means that for some reason the devtools connection got closed prematurely.

The error appears somewhat flaky. Do we know when it started happening?

+ssid

Yes, it is flaky, but it fails more often than it succeeds. I am not sure when it started, but it seems to be happening from a long time. For ex there are failures from builds 6292, 6293 
(https://build.chromium.org/p/chromium.perf/builders/Android%20One%20Perf%20%282%29/builds/6292)

ssid, what are your thoughts?

Maybe disable until we can reproduce and fix the issue?

I was trying to see if it is still flaky, but can't see the Android One Perf(2) bot. Did somethings changed? Though on Android One, toi seems to be quite flaky. (https://build.chromium.org/p/chromium.perf/builders/Android%20One%20Perf?numbuilds=200). Should we disable toi?

The bots are all now swarmed, so they lost the "(%d)" at the end. Now there is just one Android One bot (the one you found).

It's true I see v8.runtimestats.browsing_mobile failing a lot, but it's hard to keep track of which stories fail on which builds and why until we get better integration with the test_results/flakiness dashboard.

Ned, what do you suggest we do here?

I was just looking at the failures on Android One again, atleast on Android One only cnn and toi fail. All the others are working fine.

Another thing that surprised me is that v8.runtimestats.browsing_desktop is now failing on Android One. It fails on several pages. But shouldn't v8.runtimestats.browsing_desktop be disabled on Android?

> I was just looking at the failures on Android One again, at least on
> Android One only cnn and toi fail. All the others are working fine.

I think it's fine to disable those two, but ideally only for v8.runtimestats benchmarks (and not the whole system health). +rnephew would that be possible after your CL lands?

> Another thing that surprised me is that v8.runtimestats.browsing_desktop is 
> now failing on Android One. It fails on several pages. But shouldn't 
> v8.runtimestats.browsing_desktop be disabled on Android?

Yep, that's a bug. Mythri, could you write a CL adding  "@benchmark.Disabled('android')" on all desktop benchmarks here:
https://cs.chromium.org/chromium/src/tools/perf/benchmarks/v8_browsing.py

Thanks!

With StoryExpectations we can disable this on A1 only, yes.

Toi is now disabled on Android. Marking it as fixed.