New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 739783 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug
Hotlist-MemoryInfra

Blocked on:
issue 740045



Sign in to add a comment

Move symbolisation script to use breakpad format

Project Member Reported by etienneb@chromium.org, Jul 6 2017

Issue description

The catapult script used for symbolizing chrome traces is using addr2line (elf/pdb debug information) to translate PC to symbols.

Chrome official build provide the light-weight breakpad format.
see: https://chromium.googlesource.com/breakpad/breakpad/

They are stored on the server under the name breakpad-info or breakpad-info.zip.
It's an archive of textual files containing symbols information.

This format is probably faster to download and process than the PDB/ELF.
 
The llvm-symbolizer is not using this format but asan_symbolizer is:
see https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/asan/scripts/asan_symbolize.py

search for class BreakpadSymbolizer(Symbolizer):
I feel this should speed symbolization considerably!

However, old path should still be there for local builds. Or maybe we can always generate that breakpad info during profiling builds? 
Agree, old symbolisation should works as-is.
Also, these files are not public available, so it will not work for a local build off-corp.
Blockedon: 740045
I super-approve, especially if you say that is faster (not surprised, the dwarf data addr2line has to traverse is insanely complex)

About symbol fetching I left some notes in Issue 740045. Not sure whether that is useful, up to you to figure out the best strategy to fetch symbols.

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c2e37410c66a143b428e1fe6ef3f19fc03c5509e

commit c2e37410c66a143b428e1fe6ef3f19fc03c5509e
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Wed Jul 12 01:37:42 2017

Roll src/third_party/catapult/ fc9479ad5..08d8c9f08 (10 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/fc9479ad5e3b..08d8c9f08602

$ git log fc9479ad5..08d8c9f08 --date=short --no-merges --format='%ad %ae %s'
2017-07-11 eakuefner [Dashboard] Include common/py_utils in dashboard deployment
2017-07-11 loloangela Removed abstract-class-little-used and abstract-class-not-used; Fixed errors related to bad-continuation
2017-07-11 etienneb Support for breakpad symbolisation from a symbols folder
2017-07-11 eakuefner [Tracing] use discover instead of REGISTRY to deserialize Python diagnostics
2017-07-11 benjhayden Split HistogramGrouping to its own file.
2017-07-11 etienneb Fix incorrect path with symbolisation script
2017-07-11 ashleymarie Add json3 output formatter
2017-07-11 simonhatch Dashboard - Rename OauthTest -> ApiAuthTest
2017-07-11 sullivan Cache whether an identity is a group member.
2017-07-11 etienneb Implements breakpad symbolisation

Created with:
  roll-dep src/third_party/catapult
BUG= 739783 , 717394 , 739783 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: Ia997fc6f7d00dc9c939c0eee4b0f9fde9d0710f4
Reviewed-on: https://chromium-review.googlesource.com/567640
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#485776}
[modify] https://crrev.com/c2e37410c66a143b428e1fe6ef3f19fc03c5509e/DEPS

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/49743d172b736ffd619215c32b6eb3e2582b3b80

commit 49743d172b736ffd619215c32b6eb3e2582b3b80
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Thu Jul 13 01:41:01 2017

Roll src/third_party/catapult/ 6c40c273a..1286055c7 (8 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/6c40c273a7fe..1286055c7baf

$ git log 6c40c273a..1286055c7 --date=short --no-merges --format='%ad %ae %s'
2017-07-12 adithyas Add table for Blink RuntimeCallStats
2017-07-12 bpastene devil: Add a device_utils method to mv-then-rm a device file.
2017-07-12 nednguyen Rename webpagereplay_go.py to webpagereplay_go_server.py
2017-07-12 nednguyen Add WPR go binary for Mac platform
2017-07-12 bpastene devil: When pushing data deps to device, don't mkdir redundantly.
2017-07-12 simonhatch Dashboard - Fix test-picker add button never un-greying
2017-07-12 simonhatch Dashboard - Add api_request_handler
2017-07-12 etienneb Fix symbolisation for address out-of-range

Created with:
  roll-dep src/third_party/catapult
BUG= 724543 ,739899,739899, 739783 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I114d56bf15f445060eb48951a120fd99819e0024
Reviewed-on: https://chromium-review.googlesource.com/569138
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486207}
[modify] https://crrev.com/49743d172b736ffd619215c32b6eb3e2582b3b80/DEPS

Project Member

Comment 8 by bugdroid1@chromium.org, Jul 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/88ab20412467348e8b4387cca3ee48c5ea302379

commit 88ab20412467348e8b4387cca3ee48c5ea302379
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Fri Jul 14 01:23:09 2017

Roll src/third_party/catapult/ 6daeed6d7..2569648cd (14 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/6daeed6d7311..2569648cdd4f

$ git log 6daeed6d7..2569648cd --date=short --no-merges --format='%ad %ae %s'
2017-07-13 simonhatch Pinpoint - First pass new job dialog
2017-07-13 benjhayden Fix CSVBuilder.
2017-07-13 benjhayden Reserve names for split diagnostics.
2017-07-13 dtu [pinpoint] Correct device affinity when multiple executions are started together.
2017-07-13 eakuefner [Catapult] Fix bin_checks PRESUBMIT not allowing deletions in bin/
2017-07-13 nednguyen Revert of Add logging to debug battor not running (patchset #1 id:1 of https://codereview.chromium.org/2869913002/ )
2017-07-13 sullivan Add support for additional flag-specified tracing categories.
2017-07-13 eakuefner [Dashboard] Make add_histograms an ApiRequestHandler
2017-07-13 etienneb Validate code-identifiers from traces/modules when available
2017-07-13 eakuefner [Tracing] Fix bin/run_tests
2017-07-13 kbr Fix run_browser_tests.py on Android.
2017-07-13 sullivan Add APIs to list the timeseries for a benchmark, and download them.
2017-07-13 sullivan Add alert timestamps, bisect start, and convert bug timestamps to ISO format.
2017-07-13 nednguyen Add smoke test for webpagereplay_go_server

Created with:
  roll-dep src/third_party/catapult
BUG= 719447 , 739783 , 740769 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I44b1b5541e2c258f061e3138318a8c1af9cdcbc1
Reviewed-on: https://chromium-review.googlesource.com/570836
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486599}
[modify] https://crrev.com/88ab20412467348e8b4387cca3ee48c5ea302379/DEPS

Project Member

Comment 9 by bugdroid1@chromium.org, Jul 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/56b10602bbb71d324a292d8a045e4cb590d2de27

commit 56b10602bbb71d324a292d8a045e4cb590d2de27
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Fri Jul 14 17:50:45 2017

Roll src/third_party/catapult/ b63abe05c..ebf00ecf2 (1 commit)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/b63abe05c654..ebf00ecf2e83

$ git log b63abe05c..ebf00ecf2 --date=short --no-merges --format='%ad %ae %s'
2017-07-14 etienneb Fix incorrect computation of sections when symbolizing traces

Created with:
  roll-dep src/third_party/catapult
BUG= 739783 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: Icd11d46ea973a4069910c3aacd742ed4e228124d
Reviewed-on: https://chromium-review.googlesource.com/571173
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486799}
[modify] https://crrev.com/56b10602bbb71d324a292d8a045e4cb590d2de27/DEPS

Project Member

Comment 10 by bugdroid1@chromium.org, Jul 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fcef19bb3c9810d6a515bb5e4a084c576caeea8b

commit fcef19bb3c9810d6a515bb5e4a084c576caeea8b
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Fri Jul 14 20:15:39 2017

Roll src/third_party/catapult/ ebf00ecf2..1b26db0a1 (1 commit)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/ebf00ecf2e83..1b26db0a1723

$ git log ebf00ecf2..1b26db0a1 --date=short --no-merges --format='%ad %ae %s'
2017-07-14 etienneb Remove useless debug variable

Created with:
  roll-dep src/third_party/catapult
BUG= 739783 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I032b7a73221c674df6d5a93bed297a672d6048c8
Reviewed-on: https://chromium-review.googlesource.com/572120
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486863}
[modify] https://crrev.com/fcef19bb3c9810d6a515bb5e4a084c576caeea8b/DEPS

Project Member

Comment 11 by bugdroid1@chromium.org, Jul 26 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c98e5405cccddf557a9c99069d7c0e859b7a310c

commit c98e5405cccddf557a9c99069d7c0e859b7a310c
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Wed Jul 26 15:08:24 2017

Roll src/third_party/catapult/ b5d2ffa3c..624deb924 (34 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/b5d2ffa3c55a..624deb924d12

$ git log b5d2ffa3c..624deb924 --date=short --no-merges --format='%ad %ae %s'
2017-07-26 kraynov Android systrace: Show memory maps in TraceViewer.
2017-07-25 benjhayden Fix LineChart's dots.
2017-07-25 benjhayden Unambiguously indicate demo dashboard builds.
2017-07-25 crouleau Add an option to enable Chrome's --v=2
2017-07-25 pcc run_dev_server_tests: Print stderr after stdout.
2017-07-25 phsilva Ownership into GenericSets
2017-07-25 benjhayden Allow users to dismiss import warnings.
2017-07-25 loloangela Fixed errors related to bad-continuation pt.24
2017-07-25 loloangela Fixed errors related to bad-continuation pt.25
2017-07-25 xunjieli [wpr-go] Minor fix to legacyformatconvertor.go
2017-07-25 eakuefner [eslint] Delete comment from .eslintrc
2017-07-25 loloangela Fixed errors related to bad-continuation pt.23
2017-07-25 benjhayden Refactor tests to use new reserved GenericSets instead of TelemetryInfo.
2017-07-25 simonhatch Tracing - Move diagnostics to use a registry instead of discover module.
2017-07-25 etienneb Add support for breakpad symbolisation of linux trace.
2017-07-25 loloangela Fixed errors related to bad-continuation pt.20
2017-07-25 loloangela Fixed errors related to bad-continuation pt.19
2017-07-25 loloangela Fixed errors related to bad-continuation pt.18
2017-07-25 kraynov Android systrace: Atrace process dump importer V1.
2017-07-25 perezju [Telemetry] Restore telemetry.util.bot_utils
2017-07-25 perezju [Telemetry] Remove CheckSharedStates in StorySetUnitTest.testSmoke
2017-07-24 loloangela Fixed errors related to bad-continuation pt.16
2017-07-24 loloangela Fixed errors related to bad-continuation part 15
2017-07-24 loloangela Fixed errors related to bad-continuation pt.17
2017-07-24 loloangela Fixed errors related to bad-continuation
2017-07-24 loloangela Fixed errors related to bad-continuation Part 14
2017-07-24 xunjieli [wpr-go] Make ephemeral port support less flaky
2017-07-24 sullivan Remove benchmark health report link from dashboard menu
2017-07-24 loloangela Fixed errors related to bad-continuation
2017-07-24 xunjieli [wpr-go] Add a temporary script to convert legacy format
2017-07-24 loloangela Fixed errors related to bad-continuation
2017-07-24 perezju [Telemetry] Remove unused story_runner.StoryGroup
2017-07-24 nednguyen Remove support for json output format for benchmark list command
2017-07-24 perezju [Telemetry] Remove support for mixed shared states

Created with:
  roll-dep src/third_party/catapult
BUG= 739783 , 740090 ,746251,746251, 740090 ,746251


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: Iabd4aad0c4fefd98ec4d19cc25dd942ffac9eda9
Reviewed-on: https://chromium-review.googlesource.com/586767
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489644}
[modify] https://crrev.com/c98e5405cccddf557a9c99069d7c0e859b7a310c/DEPS

Status: Fixed (was: Untriaged)

Sign in to add a comment