New issue
Advanced search Search tips

Issue 898474 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Nov 5
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: iOS
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Chrome beta fails to generate crash ID on inducing browser crash (chrome://inducebrowsercrashforrealz)

Project Member Reported by ghuphran@chromium.org, Oct 24

Issue description

App Version: 71.0.3578.21 Beta
iOS Version:  11.4.1, 12.0.1, 10.3.3, 
Device: iPhone X, iPhone 8, iPhone7+, iPad, iPad Pro
URL: N/A

Steps to reproduce:
  1. Launch Chrome.
  2. Navigate to chrome://inducebrowsercrashforrealz
  3. Relaunch the app after the app has crashed
  4. Navigate to chrome://crashes 
  
Observed results:
Chrome beta fails to generate crash ID on inducing browser crash (chrome://inducebrowsercrashforrealz)

Expected results:
Crash ID should be generated on inducing browser crash.

Number of times you were able to reproduce: 5/5
Bug reproducible after clean install: Yes
Bug reproducible after clearing cache and cookies: Yes
Bug reproducible on Chrome Mobile on Android:NA
Bug reproducible on Safari/Firefox: Firefox: NA, Safari: NA
Bug reproducible on current stable build (App Version, iOS Version): No on 70
Bug reproducible on the current beta channel build (App Version, iOS Version): Yes on M71

Link to video/image: 
https://drive.google.com/file/d/1WvKSwS1aSZZdSkKXFCQUr3t1nSkO6NHT/view?usp=sharing
 
Description: Show this description
Components: Internals>CrashReporting
Owner: pkl@chromium.org
Status: Assigned (was: Untriaged)
ghuphran@ you mention its a Regression, is this working on previous M71 betas or on M70?

pkl@ we're not really sure who to assign this to, could you PTAL?
Unable to find the issue on m70 looks like a regression in m71. Please find the revision numbers below:

Revision Number for 71.0.3559.0 (Good Version) - 156c3073b15f
Revision Number for 71.0.3560.0 (Bad Version) - d43b618acf65
Cc: olivierrobin@chromium.org
Labels: M-71
This seems serious if this is a M71 regression.
+olivierrobin as FYI
Labels: ReleaseBlock-Stable
I could generate one which was correctly uploaded (71.0.3578.21)
https://crash.corp.google.com/browse?q=reportid=%270bb617279fc59325%27
Actually no, it does not work anymore.
I will investigate tomorrow.
Cc: -olivierrobin@chromium.org pkl@chromium.org
Owner: olivierrobin@chromium.org
I spent the day debugging breakpad and I have an hypothesis.

It seems that NSURLSession that are sent too early in the life of the application life it systematically gets an NSURLDomain error.

I would suggest to add a 5 second delay (or so) before uploading the first crash report.
I tested in chromium and it could upload a report 
https://crash-staging.corp.google.com/browse?stbtiq=7186829e09ca9d39

This would also explain why it is flaky (if a two reports are sent in the same session, the second will pass).

I have no idea if this is an iOS change or a Chromium/breakpad change.
I don't think I can do the change today but we could do an experiment on canary to see.

WDYT?

SGTM
I did more tests today and it seems that the issue is triggered by the interaction between the networkstack and the breakpad upload.

It seems that if a network request is sent too early after 
  [NetworkStackSetup setUpChromeNetworkStack:&_requestTrackerFactory
                 httpProtocolHandlerDelegate:&_httpProtocolHandlerDelegate];
https://cs.chromium.org/chromium/src/ios/chrome/app/main_controller.mm?type=cs&q=main_contro&sq=package:chromium&g=0&l=664

it will fail.
As both the network stack and Breakpad run on their own thread, it is hard to know the sync issue.

We still could add the delay (and that will fix the issue) or we could investigate more to see the problem.
Independantly of this fix, it could be interesting to investigate as other feature may fail on the first upload (metrics?) so it is not obvious if we should add the delay.

WDYT?
Project Member

Comment 12 by bugdroid1@chromium.org, Oct 31

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2a898dc96a962abe250409cc54e60a07b9d46543

commit 2a898dc96a962abe250409cc54e60a07b9d46543
Author: Olivier Robin <olivierrobin@chromium.org>
Date: Wed Oct 31 14:37:04 2018

Do not upload crash reports when application is inactive.

Registering NSURLProtocol will cancel the NSURLSession in
progress. Documentation state that NSURLProtocol must be
registered before sending NSURLSession, but sending
NSURLSession just after setting a protocol will also fail
(at least synchronously in the
application didFinishLaunching runloop).

This CL adds an experiment to disable the breakpad upload
enabling in this runloop.
Upload will be triggered on the initialization of the
settings observer during the deferred initialization (about
4 seconds later).

Random notes:
- recovery mode does not initialize ChromeNetworkStack,
  so is not affected by the issue.
- There may be other NSURLsession triggered in this runloop
  that are affected.

Bug:  898474 
Change-Id: I4c9fddd4fef70d770b200d7264bf729b2f025d28
Reviewed-on: https://chromium-review.googlesource.com/c/1307501
Reviewed-by: Peter Lee <pkl@chromium.org>
Reviewed-by: Justin Cohen <justincohen@chromium.org>
Commit-Queue: Olivier Robin <olivierrobin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#604247}
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/BUILD.gn
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/about_flags.mm
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/crash_report/BUILD.gn
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/crash_report/breakpad_helper.mm
[add] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/crash_report/crash_report_flags.cc
[add] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/crash_report/crash_report_flags.h
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/ios_chrome_flag_descriptions.cc
[modify] https://crrev.com/2a898dc96a962abe250409cc54e60a07b9d46543/ios/chrome/browser/ios_chrome_flag_descriptions.h

Labels: Merge-Request-71
Status: Fixed (was: Assigned)
Project Member

Comment 14 by sheriffbot@chromium.org, Nov 5

Labels: -Merge-Request-71 Hotlist-Merge-Review Merge-Review-71
This bug requires manual review: Less than 25 days to go before AppStore submit on M71
Please contact the milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Is this for all possible crashes or just a certain signature?

Also, Olivier is it possible to get verification here before merge?
it is flaky, and I don't really know what triggers it or not.
I could not trigger it on other signatures, but I could not see any reason why it would not affect other crashes.

I verified it on Chromium top of tree and beta (manual cherry-pick) and it fixed the bug.
On canary, I could not reproduce the issue with flag disabled, but crash reports are uploaded correctly.
Labels: -Merge-Review-71 Merge-Approved-71
Thanks, approved.
Project Member

Comment 18 by bugdroid1@chromium.org, Nov 8

Labels: -merge-approved-71 merge-merged-3578
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eb36276ff220fd834b96b5de4332e2965420f221

commit eb36276ff220fd834b96b5de4332e2965420f221
Author: Olivier Robin <olivierrobin@chromium.org>
Date: Thu Nov 08 10:10:53 2018

Do not upload crash reports when application is inactive.

Registering NSURLProtocol will cancel the NSURLSession in
progress. Documentation state that NSURLProtocol must be
registered before sending NSURLSession, but sending
NSURLSession just after setting a protocol will also fail
(at least synchronously in the
application didFinishLaunching runloop).

This CL adds an experiment to disable the breakpad upload
enabling in this runloop.
Upload will be triggered on the initialization of the
settings observer during the deferred initialization (about
4 seconds later).

Random notes:
- recovery mode does not initialize ChromeNetworkStack,
  so is not affected by the issue.
- There may be other NSURLsession triggered in this runloop
  that are affected.

Bug:  898474 
Change-Id: I4c9fddd4fef70d770b200d7264bf729b2f025d28
Reviewed-on: https://chromium-review.googlesource.com/c/1307501
Reviewed-by: Peter Lee <pkl@chromium.org>
Reviewed-by: Justin Cohen <justincohen@chromium.org>
Commit-Queue: Olivier Robin <olivierrobin@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#604247}(cherry picked from commit 2a898dc96a962abe250409cc54e60a07b9d46543)
Reviewed-on: https://chromium-review.googlesource.com/c/1325983
Reviewed-by: Olivier Robin <olivierrobin@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#578}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/BUILD.gn
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/about_flags.mm
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/crash_report/BUILD.gn
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/crash_report/breakpad_helper.mm
[add] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/crash_report/crash_report_flags.cc
[add] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/crash_report/crash_report_flags.h
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/ios_chrome_flag_descriptions.cc
[modify] https://crrev.com/eb36276ff220fd834b96b5de4332e2965420f221/ios/chrome/browser/ios_chrome_flag_descriptions.h

Labels: Merge-Merged-71-3578
The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/eb36276ff220fd834b96b5de4332e2965420f221

Commit: eb36276ff220fd834b96b5de4332e2965420f221
Author: olivierrobin@chromium.org
Commiter: olivierrobin@chromium.org
Date: 2018-11-08 10:10:53 +0000 UTC

Do not upload crash reports when application is inactive.

Registering NSURLProtocol will cancel the NSURLSession in
progress. Documentation state that NSURLProtocol must be
registered before sending NSURLSession, but sending
NSURLSession just after setting a protocol will also fail
(at least synchronously in the
application didFinishLaunching runloop).

This CL adds an experiment to disable the breakpad upload
enabling in this runloop.
Upload will be triggered on the initialization of the
settings observer during the deferred initialization (about
4 seconds later).

Random notes:
- recovery mode does not initialize ChromeNetworkStack,
  so is not affected by the issue.
- There may be other NSURLsession triggered in this runloop
  that are affected.

Bug:  898474 
Change-Id: I4c9fddd4fef70d770b200d7264bf729b2f025d28
Reviewed-on: https://chromium-review.googlesource.com/c/1307501
Reviewed-by: Peter Lee <pkl@chromium.org>
Reviewed-by: Justin Cohen <justincohen@chromium.org>
Commit-Queue: Olivier Robin <olivierrobin@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#604247}(cherry picked from commit 2a898dc96a962abe250409cc54e60a07b9d46543)
Reviewed-on: https://chromium-review.googlesource.com/c/1325983
Reviewed-by: Olivier Robin <olivierrobin@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#578}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}
I tested 71.0.3578.49 and it seems to work fine.
Status: Verified (was: Fixed)
This issue has been verified and found fixed. Details are given below:
App Version: 71.0.3578.54 Beta
iOS Version:  11.4.1, 12.0.1
Device: iPhone 8 plus, iPad Air, iPad Pro.

Sign in to add a comment