Adding nasko@, since he was the owner of the old test, before migration to v2 (I think...)

Friendly sheriff ping

Looks like a crash of some sort.  Started happening consistently somewhere within 414704 and 414718 (https://chromium.googlesource.com/chromium/src/+log/61e97d3dd..08cd9d655c?pretty=full).  Best guesses from the changelog:

DevTools: merge devtools target with devtools host, part 1
https://codereview.chromium.org/2263843002

Add PlatformScreenDelegate and start implementation.
https://codereview.chromium.org/2274353003

If there's no obvious culprit, I'll kick off a bisect and revert the offending CL.

Actually the intersection of blame-range from all broken bots is 414705..414708 (https://chromium.googlesource.com/chromium/src/+log/16c88dfc4060..166a7cf0a05?pretty=full), so it's almost certainly one of the following 3 CLs:

DevTools: merge devtools target with devtools host, part 1
https://codereview.chromium.org/2263843002

Bundle fake Widevine component manifest for stub CDM
https://codereview.chromium.org/2136983002


Pavel, can you please take a look to see if this is indeed your CL causing the failures ASAP?

(Sorry - one of these 2, the 3rd is iOS-only).

Pavel says it was indeed probably his CL and he'll take a look later today or tomorrow.  Given that it's only the site_isolation tests I think it's fine to wait another day or so.  But there's a LOT of redness on the tree that's hard to wade through, so if it's not fixed in a day or two we should either revert the CL or disable the test.

@Ned: Annie pointed me your way. Could you help me reproducing this? I'll fix it, but I need the repro.

To reproduce locally, you can run:
./tools/perf/run_benchmark page_cycler_v2_site_isolation.basic_oopif --browser=release

I can't get to the bottom of it. Could you tell me where telemetry fails and what expectations are not met?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7e4d11e55c607caad099671f2d46aa7726b65e99

commit 7e4d11e55c607caad099671f2d46aa7726b65e99
Author: pfeldman <pfeldman@chromium.org>
Date: Thu Sep 01 02:53:26 2016

DevTools: make telemetry happy via calling oopifs 'iframes' in the discovery.

BUG= 641528 
TBR=

Review-Url: https://codereview.chromium.org/2301633003
Cr-Commit-Position: refs/heads/master@{#415864}

[modify] https://crrev.com/7e4d11e55c607caad099671f2d46aa7726b65e99/content/browser/devtools/devtools_agent_host_impl.cc

Based on the log, telemetry fails at the step of closing the tab:
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 228, in WillRunStory
    self.browser.tabs[-1].Close()
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/browser/tab.py", line 100, in Close
    self._tab_list_backend.CloseTab(self.id)
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/tab_list_backend.py", line 66, in CloseTab
    util.WaitFor(lambda: tab_id not in self.IterContextIds(), timeout=5)
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/core/util.py", line 86, in WaitFor
    res = condition()
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/tab_list_backend.py", line 66, in <lambda>
    util.WaitFor(lambda: tab_id not in self.IterContextIds(), timeout=5)
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend_list.py", line 86, in IterContextIds
    self._Update()
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend_list.py", line 99, in _Update
    backends_map = self._devtools_client.GetUpdatedInspectableContexts()
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 298, in GetUpdatedInspectableContexts
    contexts = self._ListInspectableContexts()
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 223, in _ListInspectableContexts
    return self._devtools_http.RequestJson('')
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_http.py", line 107, in RequestJson
    return json.loads(self.Request(path, timeout))
  File "/b/c/b/Linux_Perf__2_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_http.py", line 88, in Request
    response = self._conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1051, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 415, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 379, in _read_status
    raise BadStatusLine(line)


To close the tab, telemetry sends a GET request like 127.0.0.1:<devtool_port>/json/close/<tab_id>. (<tab_id> is the id of context from  devtool's response to 127.0.0.1:<devtool_port>/json/, this is devtools_http.RequestJson('') with  |path| param equals to '', code is in https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_http.py?rcl=0&l=96)

After sending that close tab request, telemetry does a polling loop that keep calling devtools_http.RequestJson('') & check whether the return contexts dict from devtool no longer have the context id that it supposed to close. However, this fail as some point due to a bad response message from devtool:

File "/usr/lib/python2.7/httplib.py", line 1051, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 415, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 379, in _read_status
    raise BadStatusLine(line)

Note that you can add more logging to browser to figure out what's going on when devtools response with that bad message. The way to do that is to rerun the benchmark command in #8 with "--browser-logging-verbosity=verbose".

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/71ff18e2385d68b077b081f2c98d8ee9c9b42108

commit 71ff18e2385d68b077b081f2c98d8ee9c9b42108
Author: pfeldman <pfeldman@chromium.org>
Date: Thu Sep 01 18:43:21 2016

DevTools: check for target being oopif prior to matching its web contents.

BUG= 641528 
TBR=dgozman

Review-Url: https://codereview.chromium.org/2303823002
Cr-Commit-Position: refs/heads/master@{#416006}

[modify] https://crrev.com/71ff18e2385d68b077b081f2c98d8ee9c9b42108/chrome/browser/devtools/chrome_devtools_manager_delegate.cc

The run is green again: https://build.chromium.org/p/chromium.perf/builders/Linux%20Perf%20%282%29

Issue 643457 has been merged into this issue.