I noticed a couple of tests fail in the chromedriver_py_tests suite in https://chromium-review.googlesource.com/1119622 and did a spot check on the bot.

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng?limit=200 has failed in 50 of the last 200 CLs on the bot.

Just a few examples:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27609
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27607
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27779
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27778
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27602

I'm pretty sure all of these CLs are unrelated to the failing tests.

Looking at a recent failure:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/27781

these two tests failed:

__main__.ChromeDriverTest.testSlowIFrame
__main__.ChromeDriverTest.testUnexpectedAlertOpenExceptionMessage

here's the failure log from one of them:

ERROR: testSlowIFrame (__main__.ChromeDriverTest)
Verify ChromeDriver waits for slow frames to load.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../../chrome/test/chromedriver/test/run_py_tests.py", line 370, in setUp
    self._driver = self.CreateDriver()
  File "../../chrome/test/chromedriver/test/run_py_tests.py", line 292, in CreateDriver
    **kwargs)
  File "e:\b\s\w\ir\chrome\test\chromedriver\test\..\client\chromedriver.py", line 237, in __init__
    response = self._ExecuteCommand(Command.NEW_SESSION, params)
  File "e:\b\s\w\ir\chrome\test\chromedriver\test\..\client\chromedriver.py", line 291, in _ExecuteCommand
    raise _ExceptionForLegacyResponse(response)
SessionNotCreatedException: session not created exception
from tab crashed
  (Session info: chrome=69.0.3477.0)
  (Driver info: chromedriver=2.40.571409 (9b2c83b74be21fec5fe66818e142de14d81ba02f),platform=Windows NT 6.1.7601 SP1 x86_64)


Basically – it looks like tab crashes are severely affecting this test suite. Unfortunately it doesn't look like minidumps are being generated and symbolized by this test harness.

This would be an ideal job for FindIt if it were handling this test suite.

Unfortunately the flakiness goes back to the beginning of the 200 builds on this bot so it looks like it's going to be difficult to track down whatever introduced this flakiness.

(Could this be related to the recent changes to these tests to handle retries?)