New issue
Advanced search Search tips

Issue 778302 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 20
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Feature

Blocking:
issue 760498



Sign in to add a comment

Improve Android startup speed UMA metrics

Project Member Reported by alexilin@chromium.org, Oct 25 2017

Issue description

Some of the Android startup histograms are empty, absent or broken.

For instance, the Startup.FirstCommitNavigationTime* is the most important end-to-end startup speed histogram on Android and it is currently broken.
@pasko recreated a history of this metric: https://goo.gl/NyvX39

Following steps could make the Starup.FirstCommitNavigationTime* histogram better:
* Record the histogram only after the first commit. Currently it's recorded for all navigations what can be proved by the abnormally high histogram count and by too large values.
* Measure the time starting from the process creation instead of ForegroundStartTime. Currently we don't cover the important part of the startup latency because the ForegroundStartTime is measured after the native library is loaded. That's a lot of important data missed.
* Discard cases when the application was run in background (offliner, AMP background tab, etc.) See   http://crbug.com/711708   as an example of unexpected order of events causing a crash.
* Discard cases when a user action interrupts the startup sequence. It may shift the histogram to smaller values because users will likely interrupt the startup if it takes too long. It may be useful to create a separate histogram with "Uninterrupted" suffix instead of the discarding.

Other paths of Android startup speed metrics development:
* Make sure that we cover all user stories on Android such as:
  * Start with an intent
  * Start from the app launcher
  * Open Chrome from PWA shortcut
  * CustomTabs with warmup()
  * WebView startup
* Add startup histograms for First{,Contentful,Meaningful}PaintTime.
* Make Android startup metrics consistent with desktop. Startup metrics used on desktop platforms are tracked by chrome-speed-releasing@ and Chirp heartbeat metrics (so there's visibility and process if it regresses). In particular, we could implement some subset of Startup.FirstWebContents.* on Android.
 

Comment 1 by pasko@chromium.org, Oct 26 2017

Labels: -Pri-3 Pri-2
> * Measure the time starting from the process creation instead of ForegroundStartTime. Currently we are losing all the datapoints for starting the activity without the native library loaded. That's a lot of important data missed.

nit: I would say we have the datapoints, but they do not _cover_ the important part of the startup latency
Description: Show this description

Comment 3 by pasko@chromium.org, Oct 26 2017

Description: Show this description

Comment 4 by pasko@chromium.org, Oct 26 2017

I made the document public, and updated the link

Comment 5 by pasko@chromium.org, Oct 26 2017

Description: Show this description
Project Member

Comment 6 by bugdroid1@chromium.org, Oct 26 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ec2faca5c123958f778edb083072d43372855ca6

commit ec2faca5c123958f778edb083072d43372855ca6
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Thu Oct 26 15:52:52 2017

Fix MobileStartup.TimeSinceLastUse metric

The time delta between the start time and the last used time is compared
with Integer.MAX_VALUE before casting it to int but it's compared using
a wrong operator.

Bug:  778302 
Change-Id: I8cce18dd327766d54310a71fa897b65108a6cab9
Reviewed-on: https://chromium-review.googlesource.com/738244
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Reviewed-by: Ted Choc <tedchoc@chromium.org>
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Cr-Commit-Position: refs/heads/master@{#511830}
[modify] https://crrev.com/ec2faca5c123958f778edb083072d43372855ca6/chrome/android/java/src/org/chromium/chrome/browser/metrics/StartupMetrics.java
[modify] https://crrev.com/ec2faca5c123958f778edb083072d43372855ca6/tools/metrics/histograms/histograms.xml

Comment 7 by pasko@chromium.org, Oct 30 2017

Blocking: 760498
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/df1ab7e5662d4cf383e66bf3100ee69270b916f9

commit df1ab7e5662d4cf383e66bf3100ee69270b916f9
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Mon Dec 04 14:26:24 2017

Pass navigation id to Java page load metrics observers

Page load metrics observers in Java are not tied to a single
navigation as in C++. Instead of that they receive all metrics events
as long as they are registered in a global observer list. Even though a
pointer to web contents is passed in each function, it's impossible to
attribute a given event to particular navigation.

This CL fixes this by passing a navigation id along with web contents
pointer to all page load metrics observers in Java.

Bug:  778302 
Change-Id: I753a44abde849ce34b9472a57f14b23465e17982
Reviewed-on: https://chromium-review.googlesource.com/801015
Reviewed-by: Benoit L <lizeb@chromium.org>
Reviewed-by: Bryan McQuade <bmcquade@chromium.org>
Reviewed-by: David Trainor <dtrainor@chromium.org>
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521333}
[modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/java/src/org/chromium/chrome/browser/customtabs/CustomTabActivity.java
[modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/java/src/org/chromium/chrome/browser/metrics/PageLoadMetrics.java
[modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java
[modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/browser/page_load_metrics/observers/android_page_load_metrics_observer.cc
[modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/browser/page_load_metrics/observers/android_page_load_metrics_observer.h

Project Member

Comment 9 by bugdroid1@chromium.org, Dec 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8be495bb2adb20e7b74309ae213796c848d4526d

commit 8be495bb2adb20e7b74309ae213796c848d4526d
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Tue Dec 05 19:06:27 2017

android: Fix Startup.FirstCommitNavigationTime histogram

This CL makes following changes to the
Startup.FirstCommitNavigationTime3 histogram:
* Do not record the histogram multiple times per session
* Start recording from the application start time instead of the
  foreground time
* Record the histogram only for ChromeTabbedActivity
* Ignore foreground state, there will be separate histogram for this
  purpose
* Discard error, same document and non http(s) pages

The CL also renames the histogram to
Startup.Android.Experimental.Cold.TimeToFirstNavigationCommit

The rationale behind the recording the histogram from the application
start time instead of the foreground time:

The current metric doesn't cover the important part of the startup
latency because the ForegroundStartTime is measured after the native
library is loaded. That's a lot of important data missed. We're
exploring various ways to speed up the startup including the time
of native library initialization. We want to have a metric that will
show us these improvements.
Initially, this metric was measuring cold startups from the application
start but then it was changed. We would like to return to the initial
meaning. pasko@ described an evolution of the metric in
https://goo.gl/NyvX39

Bug:  778302 
Change-Id: I01d7a6a73fbc504e31c1e877b1253fcf69a54ad8
Reviewed-on: https://chromium-review.googlesource.com/788875
Reviewed-by: Yaron Friedman <yfriedman@chromium.org>
Reviewed-by: David Trainor <dtrainor@chromium.org>
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Reviewed-by: Egor Pasko <pasko@chromium.org>
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521771}
[modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java
[modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java/src/org/chromium/chrome/browser/tab/TabWebContentsObserver.java
[modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java_sources.gni
[add] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java
[modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/tools/metrics/histograms/histograms.xml

Project Member

Comment 10 by bugdroid1@chromium.org, Dec 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29

commit 7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Wed Dec 20 19:52:52 2017

android: Record FirstContentfulPaint startup histogram

This CL adds a new startup histogram
Startup.Android.Experimental.Cold.TimeToFirstContentfulPaint.

The histogram is recorded only for the first navigation in the application
lifetime under the same conditions as
Startup.Android.Experimental.Cold.TimeToFirstNavigationCommit.

Also this CL adds a C++ UMA_HISTOGRAM_COUNTS_100 equivalent to the Java side,
because now it's used by multiple clients.

Bug:  778302 
Change-Id: I8f81fe70c90e8c9ce9be597f22052786c8d28f63
Reviewed-on: https://chromium-review.googlesource.com/823847
Reviewed-by: Egor Pasko <pasko@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Reviewed-by: Bernhard Bauer <bauerb@chromium.org>
Reviewed-by: Mark Pearson <mpearson@chromium.org>
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#525417}
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/base/android/java/src/org/chromium/base/metrics/RecordHistogram.java
[add] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/metrics/StartupPageLoadMetricsObserver.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/Tab.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/TabUma.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/TabWebContentsObserver.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java_sources.gni
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java
[modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/tools/metrics/histograms/histograms.xml

Project Member

Comment 11 by bugdroid1@chromium.org, Jan 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec

commit b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Thu Jan 18 13:09:04 2018

Record startup loading histograms only in foreground

This CL discards recording startup histograms
Startup.Android.Experimental.Cold.TimeToFirst{ContentfulPaint,NavigationCommit}
if the application wasn't in the foreground state since start till the
end of event.

The reason of this change is that currently these histograms have long
tail with unrealistically high timings to load the first page.

It's okay to modify these histograms without renaming because they are
marked as experimental.

Bug:  778302 
Change-Id: I33341f34506b75e8671c21c0e19aeb4e9172d75d
Reviewed-on: https://chromium-review.googlesource.com/840003
Reviewed-by: Mark Pearson <mpearson@chromium.org>
Reviewed-by: Bernhard Bauer <bauerb@chromium.org>
Reviewed-by: Egor Pasko <pasko@chromium.org>
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#530129}
[modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/java/src/org/chromium/chrome/browser/firstrun/FirstRunActivityBase.java
[modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java
[modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java
[modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/test/android/javatests/src/org/chromium/chrome/test/util/ApplicationTestUtils.java
[modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/tools/metrics/histograms/histograms.xml

Project Member

Comment 12 by bugdroid1@chromium.org, Feb 7 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9bfff2d16aec309d883ea7a066ffe1b42de465ce

commit 9bfff2d16aec309d883ea7a066ffe1b42de465ce
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Wed Feb 07 14:48:09 2018

Fix flaky testBackgroundedPageNotRecorded test

This test doesn't use ActivityTestRule.startMainActivityFromIntent
becuase it needs to put an activity in background before the navigation
is committed. Then the test doesn't check that the tab was created and
starts the second load that fails.

Apparentely, the issue is reproduced only on slow devices, because
before the second load the main activity is put in background and then
back in foreground.

Bug:  778302 
Change-Id: I720689733b12907a6da1cc986dea1635df8bf2d9
Reviewed-on: https://chromium-review.googlesource.com/906427
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Reviewed-by: Bernhard Bauer <bauerb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#535003}
[modify] https://crrev.com/9bfff2d16aec309d883ea7a066ffe1b42de465ce/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java
[modify] https://crrev.com/9bfff2d16aec309d883ea7a066ffe1b42de465ce/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java

Comment 13 by pasko@chromium.org, Mar 22 2018

Cc: tdres...@chromium.org

Comment 14 by pasko@chromium.org, Mar 22 2018

Summary: Improve Android startup speed UMA metrics (was: Improve Android startup speed metrics)
Project Member

Comment 15 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/47b68d843471c211ef22d916752eeea1a4b9f702

commit 47b68d843471c211ef22d916752eeea1a4b9f702
Author: Alexandr Ilin <alexilin@chromium.org>
Date: Tue Mar 27 09:07:53 2018

android: Measure startup histograms starting from activity creation

This CL starts using activity creation time instead of applicaton creation
time for the Startup.Android.Experimental.Cold.TimeToFirst{ContentfulPaint,NavigationCommit}
histograms.

It's done in order to reduce histograms noise. These histograms are still
experimental, so renaming isn't required.

Bug:  778302 
Change-Id: Ic615c14d62566bb31fec651e706ded158abf9d99
Reviewed-on: https://chromium-review.googlesource.com/977968
Reviewed-by: Maria Khomenko <mariakhomenko@chromium.org>
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Reviewed-by: Egor Pasko <pasko@chromium.org>
Commit-Queue: Alexandr Ilin <alexilin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546068}
[modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/chrome/android/java/src/org/chromium/chrome/browser/init/AsyncInitializationActivity.java
[modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java
[modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/tools/metrics/histograms/histograms.xml

Status: Fixed (was: Started)

Sign in to add a comment