📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/139381cd640000

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/139381cd640000

Add container name for src= playbacks to UKM. by dalecurtis@chromium.org
https://chromium.googlesource.com/chromium/src/+/5bba03236922b2ed9a48264062d99793ddde0d9e
0 → 1 (+1)

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

These logs are really indecipherable. The microdump has even been stripped from the logs so I can't symbolize any crash dump. Is there a better way to figure out what exactly failed?

Sadly symbolize Telemetry test properly is a known bug that we don't have staffing for yet :-(

Thats issue 703692

Dale: can you revert your CL for now & debug later. These tests are real world tests, so whatever problem the CL introduce would also affect actual users.

Screenshot which confirms this is a Chrome crash: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/edfe97c2-ace0-11e8-b25f-0242ac110008

Also this is a Telemetry log: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/ed1edf4c-ace0-11e8-b25f-0242ac110008

Not sure if that helps (breakpad still unsymbolized)

Let me take a look before reverting since it's in branch and needs to stay in branch. That log has the full microdump so let me symobolize it.

Thanks!

	Stack from Logcat
	Reading native crash info from stdin
	Reading Android symbols from: /b/swarming/w/ir
	Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release
	signal 11 (SIGSEGV), code 1, fault addr 0xc in tid 17065 (CrRendererMain)
	pid: 17051, tid: 17065, name: CrRendererMain  >>> org.chromium.chrome:sandboxed_process0 <<<
	signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
	     r0 dc115538  r1 00000000  r2 00000008  r3 00000042
	     r4 ef497e00  r5 f4012c20  r6 f4012c60  r7 f4012d60
	     r8 dc4a7320  r9 00000003  sl f4012f88  fp d4bec4cc
	     ip eaa04048  sp f4012bb8  lr d6b6df33  pc d6b6b7fc
	
	Stack Trace:
	  RELADDR   FUNCTION                                            FILE:LINE
	  v------>  media::FFmpegGlue::container() const                ../../media/filters/ffmpeg_glue.h:75:12
	  v------>  media::FFmpegDemuxer::container() const             ../../media/filters/ffmpeg_demuxer.h:265:0
	  021307fc  media::WebMediaPlayerImpl::MaybeSetContainerName()  ../../media/blink/webmediaplayer_impl.cc:3286:0

Fix here: https://chromium-review.googlesource.com/c/chromium/src/+/1199772

This will cause a fair number of crashes once M70 hits beta, so marking as RBB to ensure it's included in that eventual beta.

Dale: what command did you run to symbolize the stack? 

Symbolize didn't work I got lucky that it was already in the log. What normally works is the following:

- Extract -----BEGIN BREAKPAD MICRODUMP----- to -----END BREAKPAD MICRODUMP----- section.
- Run that through microdump_stackwalk -m locally.
- Go to go/crsym and select minidump_stackwalk, paste the output from microdump_stackwalk and click symbolize.

There's a prodacess requiring crash script you can use too that +boliu@ suggested. I foolishly thought I bookmarked the doc, but apparently didn't.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/53150fcd2b65561797f5becfb628c37952a4eb04

commit 53150fcd2b65561797f5becfb628c37952a4eb04
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Sat Sep 01 00:32:51 2018

Avoid crash getting container when no FFmpegGlue has been created.

This can occur if an error occurs before FFmpegGlue has been
created by the demuxer.

BUG= 879400 
TEST=none

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I35349e94ab7c8d164210f91073e05b50d3c58482
Reviewed-on: https://chromium-review.googlesource.com/1199772
Reviewed-by: Dan Sanders <sandersd@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588230}
[modify] https://crrev.com/53150fcd2b65561797f5becfb628c37952a4eb04/media/filters/ffmpeg_demuxer.h

This issue is marked as a release blocker with no OS labels associated. Please add an appropriate OS label.

All release blocking issues should have OS labels associated to it, so that the issue can tracked and promptly verified, once it gets fixed.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Issue 879948 has been merged into this issue.

Pulling in RB-D label from the other issue.

Is this impacting all platforms? Is this really a blocker for desktop?

It should be, but does seem to only be crashing Android, so up to you.

Actually, I think the issue is something else. So lets hold off on the merge for a moment.

Previous fix is 1/2 issues, 2/2 is here: https://chromium-review.googlesource.com/c/chromium/src/+/1205451

Okay, both fixes have landed so MR-70 again.

+benmason@ for merge approval to M-70 once this is verified on canary. Issue 879948 is dev blocker on Android.


Note: There is no M-71 Android canary available for verification due to Issue 880736.

Steps to repro:
1.Launch Chrome
2.Open a New tab -> Search or enter URL for e.g:(https://www.cnbc.com/amp/2018/08/31/oil-prices-could-surge-35percent-into-mid-90s-within-months-john-kilduff.html)

Observed behavior:
Chrome crashed 

Expected behavior:
Chrome should not crash 

Crash id:662336b36159efc8

Verified on Android Chrome:71.0.3543.2 Device:Pixel XL/PPR1.181005.003 with the above mentioned steps.

> There's a prodacess requiring crash script you can use too that +boliu@ suggested. I foolishly thought I bookmarked the doc, but apparently didn't.

In answer to #14, it's internal in src/clank/bin/symbolize_microdump.py. Originally for symbolizing microdump of official builds, but has since gained more features, some of which do make sense upstream.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f01c826e3ec282701b7afd526eadd4cc55d413cd

commit f01c826e3ec282701b7afd526eadd4cc55d413cd
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Tue Sep 04 23:50:43 2018

Don't attempt to read container with MediaUrlDemuxer.

There is no FFmpegDemuxer in this case...

BUG= 879400 
TEST=none

Change-Id: I9b32c2923084813c93a19e84b0cdb606a56959bc
Reviewed-on: https://chromium-review.googlesource.com/1205451
Reviewed-by: Thomas Guilbert <tguilbert@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588698}
[modify] https://crrev.com/f01c826e3ec282701b7afd526eadd4cc55d413cd/media/blink/webmediaplayer_impl.cc

@prashanthpola: That crash is from 70.0.3538.7, can you actually try 71.0.3543.0 or higher? Or link a crash report from that version?

Your change meets the bar and is auto-approved for M70. Please go ahead and merge the CL to branch 3538 manually. Please contact milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), geohsu@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Done.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/692855293612e25d196e7d577c114bae53bd3984

commit 692855293612e25d196e7d577c114bae53bd3984
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Thu Sep 06 00:42:39 2018

Avoid crash getting container when no FFmpegGlue has been created.

This can occur if an error occurs before FFmpegGlue has been
created by the demuxer.

BUG= 879400 
TEST=none

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I35349e94ab7c8d164210f91073e05b50d3c58482
Reviewed-on: https://chromium-review.googlesource.com/1199772
Reviewed-by: Dan Sanders <sandersd@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#588230}(cherry picked from commit 53150fcd2b65561797f5becfb628c37952a4eb04)
Reviewed-on: https://chromium-review.googlesource.com/1208958
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#79}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/692855293612e25d196e7d577c114bae53bd3984/media/filters/ffmpeg_demuxer.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f94a0aeb2d2861b40c78247f9aa9ffab0c6efa64

commit f94a0aeb2d2861b40c78247f9aa9ffab0c6efa64
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Thu Sep 06 00:42:48 2018

Don't attempt to read container with MediaUrlDemuxer.

There is no FFmpegDemuxer in this case...

BUG= 879400 
TEST=none

Change-Id: I9b32c2923084813c93a19e84b0cdb606a56959bc
Reviewed-on: https://chromium-review.googlesource.com/1205451
Reviewed-by: Thomas Guilbert <tguilbert@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#588698}(cherry picked from commit f01c826e3ec282701b7afd526eadd4cc55d413cd)
Reviewed-on: https://chromium-review.googlesource.com/1208197
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#80}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/f94a0aeb2d2861b40c78247f9aa9ffab0c6efa64/media/blink/webmediaplayer_impl.cc

-> That crash is from 70.0.3538.7, can you actually try 71.0.3543.0 or higher? Or link a crash report from that version?

For C#16, I was able to repro the crash on Chrome:70.0.3538.7 and Issue doesn't repro on Android Chrome:71.0.3543.2 Device:Pixel XL/PPR1.181005.003 with the steps mentioned in C#28.

Crash link:https://crash.corp.google.com/browse?stbtiq=662336b36159efc8

Sorry its for C#31

-> That crash is from 70.0.3538.7, can you actually try 71.0.3543.0 or higher? Or link a crash report from that version?

For C#31, I was able to repro the crash on Chrome:70.0.3538.7 and Issue doesn't repro on Android Chrome:71.0.3543.2 Device:Pixel XL/PPR1.181005.003 with the steps mentioned in C#28.

Crash link:https://crash.corp.google.com/browse?stbtiq=662336b36159efc8

Verified on webview shell browser
webview: 70.0.3538.23
device: Nexus 5X/OPR1.170623.041