New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 627277 link

Starred by 3 users

Issue metadata

Status: Verified
Owner:
please use my google.com address
Closed: Jul 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 0
Type: Bug-Regression

Blocked on:
issue 627177

Blocking:
issue 612500
issue 623398



Sign in to add a comment

Browser hangs on Windows during startup, on multiple bots

Project Member Reported by kbr@chromium.org, Jul 11 2016

Issue description

Hangs in many full-browser tests (generally, those launched via Telemetry) are being observed on multiple bots. The main bots affected are the GPU and Perf bots, though random hangs are also seen in browser_tests on the main Win bots.

The hangs are extremely problematic because they're highly intermittent on most of the machines.

Fortunately, this machine has been failing reliably since the hangs started:
https://build.chromium.org/p/chromium.perf/builders/Win%207%20Perf%20%284%29?numbuilds=200

and the first failing builds from these two machines:

https://build.chromium.org/p/chromium.perf/builders/Win%207%20Perf%20%284%29/builds/6723
https://build.chromium.org/p/chromium.gpu/builders/Win7%20Release%20%28NVIDIA%29/builds/52049

point to this CL as the very likely root cause:
https://chromium.googlesource.com/chromium/src/+/0771b1d71a3295691f9ffaefdf50652bd58e01c4

I'm going to attempt a revert after contacting rockot@ because this CL touched a lot of files and conflicts seem likely after 2 days.

 

Comment 1 by kbr@chromium.org, Jul 11 2016

Talked with Ken and he will handle the revert.

Comment 2 by kbr@chromium.org, Jul 11 2016

Blockedon: 627177

Comment 3 by kbr@chromium.org, Jul 12 2016

Revert occurring in https://codereview.chromium.org/2138263002/ .

Comment 4 by roc...@chromium.org, Jul 14 2016

Status: Fixed (was: Assigned)

Comment 5 by kbr@chromium.org, Jul 15 2016

Status: Verified (was: Fixed)
Verified that https://build.chromium.org/p/chromium.perf/builders/Win%207%20Perf%20%284%29/builds/6755 , which contains the revert above, and subsequent builds on that machine are green.

We should figure out how to write a more targeted test when attempting to re-land this change.

Comment 6 by roc...@chromium.org, Jul 15 2016

I'm trying to figure out how to repro now. Seems like this will be quite
difficult.

Comment 7 by roc...@chromium.org, Jul 15 2016

I've managed to repro hangs locally under heavy CPU load. They aren't
actually caused by the CL, but are significantly exacerbated by it due to
changes in timing. I'll have a fix for the real bug out soon, and then I'd
like to try relanding in off-peak hours assuming the local repro disappears.

Comment 8 by kbr@chromium.org, Jul 15 2016

That's great. Thanks Ken for persevering with this. See also  Issue 628765 , where random browser hangs upon startup are seen on multiple platforms. I wonder whether these issues might have the same root cause. Please go ahead with your re-land when you're ready, and please watch the Perf bots afterward, since they were the ones most reliably affected. Thanks.

Comment 9 by roc...@chromium.org, Jul 15 2016

Yeah, I checked 628765 earlier hoping it might have been my fault too. Unfortunately it looks like the those hangs are still occurring after my fix landed.

Fix is in r405776.

This failure https://build.chromium.org/p/chromium.gpu.fyi/builders/Linux%20Debug%20%28New%20Intel%29/builds/2847 was built from r405822, and this one https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Retina%20Release/builds/5618 was built from r405805.

I do however have a Windows machine crunching non-stop on starting new browser processes under very heavy CPU load, in the hopes that I might repro the hangs anyway. Maybe I should spin up a Mac too.
Labels: -Performance-Sheriff-Regressions Performance-Sheriff

Sign in to add a comment