New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 615180 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
OOO until 2019-01-24
Closed: May 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocked on:
issue 530197

Blocking:
issue 615178



Sign in to add a comment

Apparent race condition in Telemetry's minidump processing on Perf bots

Project Member Reported by kbr@chromium.org, May 26 2016

Issue description

In https://bugs.chromium.org/p/chromium/issues/detail?id=530197#c22 it was pointed out that there may be a race condition in Telemetry's minidump processing where if the minidump hasn't been fully generated yet, it might be skipped. This assertion needs to be verified and any race condition fixed.

 
fwiw, I guessed at that in #c21 there, but wasn't able to cause a problem by making the crash dump take very long. It seemed to wait as expected.

The only time I reproduced telemetry mistakenly thinking the tab was gone was by shutting down the browser mid-test.

Comment 2 by kbr@chromium.org, May 27 2016

Cc: nedngu...@google.com aiolos@chromium.org eakuefner@chromium.org
Summary: Apparent race condition in Telemetry's minidump processing on Perf bots (was: Apparent race condition in Telemetry's minidump processing)
As far as I can tell, at least one bug in the Perf bots' setup is that they build the chromium_builder_perf target (see any recent build on https://build.chromium.org/p/chromium.perf/builders/Win%20x64%20Builder ) , and this doesn't include a dependency on crashpad_database_util.exe, or any of the other important dependencies like the copy_cdb_to_output step in src/tools/perf/chrome_telemetry_build/BUILD.gn. I'm going to add logging to Telemetry's desktop_browser_backend to narrow down why it didn't find the minidump. On my machine, adding a DCHECK early on in renderer startup, if crashpad_database_util was missing then the symptom on the Perf bots happened. Once that target was built, the minidumps started getting found and symbolized.

Comment 3 by kbr@chromium.org, May 27 2016

Status: Started (was: Assigned)
Project Member

Comment 4 by bugdroid1@chromium.org, May 27 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d078c69020beaab39b72bba3d6a1a45a2386cd12

commit d078c69020beaab39b72bba3d6a1a45a2386cd12
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri May 27 06:37:56 2016

Roll src/third_party/catapult/ baa2999e8..aa8636016 (1 commit).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/baa2999e8462..aa8636016c8a

$ git log baa2999e8..aa8636016 --date=short --no-merges --format='%ad %ae %s'

BUG= 615180 

TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2015143003
Cr-Commit-Position: refs/heads/master@{#396408}

[modify] https://crrev.com/d078c69020beaab39b72bba3d6a1a45a2386cd12/DEPS

Project Member

Comment 5 by bugdroid1@chromium.org, May 27 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e58e78ecce28f5891912c000c5d2520e4f5b09d

commit 1e58e78ecce28f5891912c000c5d2520e4f5b09d
Author: kbr <kbr@chromium.org>
Date: Fri May 27 17:21:19 2016

Refactor dependencies for chromium_builder_perf target.

The perf bots build this target, but don't use isolates and therefore
aren't on the same overall path as the bots which run
telemetry_perf_unittests, where minidump processing is well tested.

Make this target depend on telemetry_chrome_test, and remove redundant
dependencies on chrome and bitmaptools. This should cause the tools
which generate and process minidumps to be properly built.

BUG= 615180 

Review-Url: https://codereview.chromium.org/2019553002
Cr-Commit-Position: refs/heads/master@{#396497}

[modify] https://crrev.com/1e58e78ecce28f5891912c000c5d2520e4f5b09d/BUILD.gn

Comment 6 by kbr@chromium.org, May 27 2016

Blocking: 615178
Status: Fixed (was: Started)
I think this is fixed. Unfortunately the only way to really test it is to induce a crash on the Perf bots, which obviously isn't desirable. Maybe we could try re-enabling the test from  Issue 615178  briefly and see if a reasonable crash stack is printed.

Sign in to add a comment