The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/11022bc34830293848771ca6563378a4b938b5c4

commit 11022bc34830293848771ca6563378a4b938b5c4
Author: charliea <charliea@chromium.org>
Date: Sun Apr 10 01:33:31 2016

[telemetry] Disable system health key mobile sites benchmark on Android

This test is failing on many of the Android bots (both reference and
ToT).

TBR=nednguyen@google.com
BUG= 601953 
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:linux_perf_cq

Review URL: https://codereview.chromium.org/1869383005

Cr-Commit-Position: refs/heads/master@{#386314}

[modify] https://crrev.com/11022bc34830293848771ca6563378a4b938b5c4/tools/perf/benchmarks/system_health.py

Thanks, Charlie!

Ethan/Ned: here's one timeout on cuteoverload on Nexus6. Not sure how to proceed.

Traceback (most recent call last):
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 336, in RunStory
    self._current_page.Run(self)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 84, in Run
    shared_state.page_test.RunNavigateSteps(self, current_tab)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/page_test.py", line 182, in RunNavigateSteps
    page.RunNavigateSteps(action_runner)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 93, in RunNavigateSteps
    url, script_to_evaluate_on_commit=self.script_to_evaluate_on_commit)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/action_runner.py", line 121, in Navigate
    timeout_in_seconds=timeout_in_seconds))
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/action_runner.py", line 44, in _RunAction
    action.RunAction(self._tab)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/actions/navigate.py", line 29, in RunAction
    time_left_in_seconds)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 73, in WaitForDocumentReadyStateToBeInteractiveOrBetter
    'document.readyState == "complete"', timeout)
  File "/b/build/slave/Android_Nexus6_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 111, in WaitForJavaScriptExpression
    e.message + '\n' + debug_message)
TimeoutException: 
Exception thrown when trying to capture console output: TimeoutException('',)

(Assigning this to benjhayden@ now)

302ing to Ethan.

Okay, so, it turns out that the problem here is with the KeyMobileSites page set.

All the benchmarks that use it are either 1. disabled on Android or 2. currently failing (in particular, it looks like blink_style.key_mobile_sites is the only other partially-enabled KeyMobileSites benchmark, and it looks to be failing on every device on which it was enabled.

I think the key first step here is to rerecord and see what happens.

Also, reenable bugs should generally be P2s, not P3s.

CC current perf sheriff charliea@.

Charlie: Can you link the bug where the return code bisect for key_mobile_sites is happening?

It actually turns out that this page set is passing on reference, and I couldn't reproduce the failures locally with android-system-chrome. So my current hunch is that there's something broken in ToT, and hopefully Charlie's bisect will find it.

(Whoops, actually CC charliea@)

Charlie, see #7.

Ben: can you rerecord the page sets?

Also the key_mobile_sites doesn't do anything besides loading the pages. Use key_mobile_sites_smooth instead.

Although key_mobile_sites_smooth has a bunch of interaction record from TBM v1, I think it's fine to keep those in the traces for now so we can calibrate how the UE model builder create the animation compared with those manually annotated IR.

Sorry, somehow didn't see this until now. I assume my input's no longer needed?

@nednguyen, does SH still want this pageset?

If so, I'll run record_wpr key_mobile_sites? Or do you want to?
https://www.chromium.org/developers/telemetry/record_a_page_set

No, let's kill this page set.

Great! Anybody else want to handle it? I can if nobody else wants to.

Go for it.

It looks like blink_style and rasterize_and_record_micro still use this pageset.
The r&r benchmark is suspiciously named rasterize_and_record_micro.key_mobile_sites_smooth.
I'll see if I can switch those two benchmarks over to the key_mobile_sites_smooth pageset.

It looks like rasterize_and_record_micro still want this pageset, so not deleting it.

https://codereview.chromium.org/2388423002/