All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=641986

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_r35tgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvvbr4wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_pymuwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_pmUpAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_o_Y5AkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggdKHoQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvrzB6QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_pfp9wgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggbLPvAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_p-G4AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggbLP_AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggZjbpgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggYCMsgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_qfXqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggaH7rgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggZb1sAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_uPzvgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggYPjqQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggaSBsAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggdW-4goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg_vGp3AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgwbKssgoM


Bot(s) for this bug's original alert(s):

chromium-rel-win10
chromium-rel-win7-dual
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-intel
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
chromium-rel-win8-dual

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9002979783463417136

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64intel_perf_bisect/builds/1151
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000181532423845536

=== Auto-CCing suspected CL author pfeldman@chromium.org ===

Hi pfeldman@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : DevTools: merge devtools target with devtools host, part 1
Author  : pfeldman
Commit description:
  
BUG= 627517 
TBR=

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2263843002
Cr-Commit-Position: refs/heads/master@{#414707}
Commit  : a9e7ddae8f1174e71c2e999eec4813ee24c1f41a
Date    : Fri Aug 26 14:37:20 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@414703  211.572  66.6385  5  good
chromium@414706  218.241  30.3338  5  good
chromium@414707  773.519  115.077  5  bad    <--
chromium@414708  645.008  132.781  5  bad
chromium@414713  616.133  132.036  5  bad
chromium@414723  740.467  127.394  5  bad

Bisect job ran on: win_x64_perf_bisect
Bug ID: 641986

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler_v2_site_isolation.basic_oopif
Test Metric: timeToFirstContentfulPaint_avg/pcv1-cold/http___booking.com
Relative Change: 249.98%
Score: 99.5

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1488
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000181532423845536


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5108187219361792

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Ping.

pfeldman, could you please look into this?

Hi Ned, how do I repro this? We need to have a better info somewhere on how to approach telemetry tests - every time something breaks I have to bug you on how to run those locally...

Pavel, the test command is:
src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=html2 --also-run-disabled-tests page_cycler_v2_site_isolation.basic_oopif

Metric to look at:
timeToFirstContentfulPaint (the average), of page http://booking.com, group "pcv1-cold"

The information about using telemetry are in: https://catapult.gsrc.io/telemetry

I obviously tried that, but I can't parse the output: 

INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb kill-server
WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.
INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /usr/local/google/code/chromium/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb kill-server
WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.
INFO:root:Try printing formatted exception: <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'platform' <traceback object at 0x7f17936beb00>

Traceback (most recent call last):
  <module> at /usr/local/google/code/chromium/src/tools/perf/run_benchmark:22
    sys.exit(main())
  main at /usr/local/google/code/chromium/src/tools/perf/run_benchmark:19
    return benchmark_runner.main(config, [trybot_command.Trybot])
  main at /usr/local/google/code/chromium/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:468
    return command_instance.Run(options)
  Run at /usr/local/google/code/chromium/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:245
    return min(255, self._benchmark().Run(args))
  Run at /usr/local/google/code/chromium/src/third_party/catapult/telemetry/telemetry/benchmark.py:91
    return story_runner.RunBenchmark(self, finder_options)
  RunBenchmark at /usr/local/google/code/chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:325
    if (possible_browser.platform.GetOSName() == 'chromeos' and
AttributeError: 'NoneType' object has no attribute 'platform'

Locals:
  benchmark                                   : <benchmarks.page_cycler_v2.PageCyclerV2BasicOopifIsolated object at 0x7f17936ae050>
  benchmark_metadata                          : <telemetry.benchmark.BenchmarkMetadata object at 0x7f17a0960bd0>
  disabled_attr_name                          : '_benchmarks.page_cycler_v2_PageCyclerV2BasicOopifIsolated_disabled_strings'
  finder_options                              : [('android_blacklist_file', None), ('browser_executable', None), ('browser_options', [('_browser_startup_timeout', 60), ('_extra_browser_args', set(['--site-per-process'])), ('browser_type', 'release_x64'), ('browser_user_agent_type', None), ('clear_sytem_cache_for_browser_and_profile_on_start', False), ('disable_background_networking', True), ('disable_component_extensions_with_background_pages', True), ('disable_default_apps', True), ('dont_override_profile', False), ('extensions_to_load', []), ('extra ... 'profiler', None), ('remote_platform_options', <telemetry.internal.platform.remote_platform_options.AndroidPlatformOptions object at 0x7f17a0a18790>), ('reset_results', None), ('results_label', None), ('run_disabled_tests', True), ('show_stdout', None), ('story_filter', None), ('story_filter_exclude', None), ('story_label_filter', None), ('story_label_filter_exclude', None), ('suppress_gtest_report', False), ('upload_bucket', 'output'), ('upload_results', None), ('use_live_sites', None), ('verbosity', 1)] (truncated)
  possible_browser                            : None
  pt                                          : <telemetry.web_perf.timeline_based_measurement.TimelineBasedMeasurement object at 0x7f1793661450>
  should_tear_down_state_after_each_story_run : True
  stories                                     : <page_sets.oopif_basic_page_set.OopifBasicPageSet object at 0x7f17936614d0>

INFO:root:Try running <function _ListAllSubprocesses at 0x7f17a0a1b7d0>
INFO:root:Did run <function _ListAllSubprocesses at 0x7f17a0a1b7d0>
pfeldman@pfeldman-z620:/usr/local/google/code/chromium$ 

Ooops, sorry for the cryptic stack. What it means is such browser does not exist on your platform.

You can run src/tools/perf/run_benchmark -v --browser=list to find out which one are available.

I make a bug about Telemetry's error reporting in https://github.com/catapult-project/catapult/issues/2917

For this case, seems like you need to build Chrome 64bit on Windows.

@nednguyen: is this a Windows-specific regression of out-of-process iframes-related functionality? How do I locate the owner of the test? It is unlikely I get this resolved on my own without understanding what the test is about.

Nasko, Kouhei: can you help Pavel to investigate the page_cycler_v2_site_isolation.basic_oopif  regression?

I assume this has been resolved.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982584626714869696