New issue
Advanced search Search tips

Issue 628765 link

Starred by 3 users

Issue metadata

Status: Duplicate
Merged: issue 628022
Owner: ----
Closed: Aug 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: Bug



Sign in to add a comment

GPU tests flaky because of problems launching the browser

Project Member Reported by kbr@chromium.org, Jul 15 2016

Issue description

The GPU tests seem to have become flaky recently because of problems starting the browser reliably. Here are a couple of builds on Mac and Linux which demonstrate the problem:

https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Retina%20Release/builds/5617
Failed:
GpuRasterization.BlueBox
GpuRasterization.ConcavePaths

https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Retina%20Release/builds/5615
Failed shard #9:
https://chromium-swarm.appspot.com/user/task/300805d55514a210

https://build.chromium.org/p/chromium.gpu.fyi/builders/Linux%20Debug%20%28New%20Intel%29/builds/2842
Failed:
ContextLost.WebGLContextLostFromLoseContextExtension

Failure mode:
[ RUN      ] ContextLost.WebGLContextLostFromLoseContextExtension
...
(WARNING) 2016-07-15 07:48:32,391 desktop_browser_backend._GetAllCrashpadMinidumps:346  No path to crashpad_database_util found
(INFO) 2016-07-15 07:48:32,391 desktop_browser_backend._GetMostRecentMinidump:414  No minidump found via crashpad_database_util
Can't get standard output with --show-stdout
(WARNING) 2016-07-15 07:48:37,474 desktop_browser_backend.Close:586  Failed to gracefully shutdown.
(WARNING) 2016-07-15 07:48:37,474 desktop_browser_backend.Close:590  Proceed to kill the browser.
(ERROR) 2016-07-15 07:48:37,477 browser.__init__:62  Failure while starting browser backend.
Traceback (most recent call last):
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 55, in __init__
    self._browser_backend.Start()
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/backends/chrome/desktop_browser_backend.py", line 291, in Start
    self._WaitForBrowserToComeUp()
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 174, in _WaitForBrowserToComeUp
    raise exceptions.BrowserConnectionGoneException(self.browser, e)
BrowserConnectionGoneException: Timed out while waiting 60s for HasBrowserFinishedLaunching.
Found Minidump: False
Stack Trace:
********************************************************************************
	No crash dump found.
********************************************************************************
Standard output:
********************************************************************************
********************************************************************************
(WARNING) 2016-07-15 07:48:37,478 shared_page_state.DumpStateUponFailure:138  Cannot dump browser state: No browser.
(WARNING) 2016-07-15 07:48:37,478 shared_page_state.DumpStateUponFailure:144  Taking screenshots upon failures disabled.
(INFO) 2016-07-15 07:48:37,482 cmd_helper._ValidateAndLogCommand:153  [host]> /tmp/isolated_runYKtdNK/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
Traceback (most recent call last):
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 78, in _RunStoryAndProcessErrorIfNeeded
    state.WillRunStory(story)
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 242, in WillRunStory
    self._StartBrowser(page)
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 202, in _StartBrowser
    self._browser = self._possible_browser.Create(self._finder_options)
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/backends/chrome/desktop_browser_finder.py", line 68, in Create
    browser_backend, self._platform_backend, self._credentials_path)
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 55, in __init__
    self._browser_backend.Start()
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/backends/chrome/desktop_browser_backend.py", line 291, in Start
    self._WaitForBrowserToComeUp()
  File "/tmp/isolated_runYKtdNK/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 174, in _WaitForBrowserToComeUp
    raise exceptions.BrowserConnectionGoneException(self.browser, e)
BrowserConnectionGoneException: Timed out while waiting 60s for HasBrowserFinishedLaunching.
Found Minidump: False
Stack Trace:
********************************************************************************
	No crash dump found.
********************************************************************************
Standard output:
********************************************************************************
********************************************************************************

[  FAILED  ] ContextLost.WebGLContextLostFromLoseContextExtension (69295 ms)

I'm not sure whether this is happening on the waterfalls -- haven't checked, but haven't received widespread notices from chromium-try-flakes -- but it is new behavior, and serious, that the browser doesn't launch reliably.

 
stdout-gpu-rasterization-mac.txt
33.5 KB View Download
stdout-webgl2-conformance-failed-shard.txt
7.6 KB View Download
stdout-context-lost-tests-linux.txt
109 KB View Download
Labels: -Pri-2 Pri-1
Browser not launching seems like a very serious problem, hence I pump up the priority.

Comment 2 by kbr@chromium.org, Jul 15 2016

Sounds good.

Filed https://github.com/catapult-project/catapult/issues/2486 about figuring out how to get stack traces when this happens.

Comment 3 by kbr@chromium.org, Jul 15 2016

Also, emailed chromium-dev asking for help.

I'm no longer seeing this failure on this builder, though I do see regular conformance test failures.

kbr@, am I reading this correctly? If so, can we close?
https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Retina%20Release

Comment 5 by kbr@chromium.org, Aug 16 2016

Mergedinto: 628022
Status: Duplicate (was: Untriaged)
Yes, eyaich@'s work in  Issue 628022  seems to have addressed the problem -- or at least highlighted that they seemed to be bot configuration problems. Duplicating this into the other.

Sign in to add a comment