Improve Android startup speed UMA metrics |
||||||||
Issue descriptionSome of the Android startup histograms are empty, absent or broken. For instance, the Startup.FirstCommitNavigationTime* is the most important end-to-end startup speed histogram on Android and it is currently broken. @pasko recreated a history of this metric: https://goo.gl/NyvX39 Following steps could make the Starup.FirstCommitNavigationTime* histogram better: * Record the histogram only after the first commit. Currently it's recorded for all navigations what can be proved by the abnormally high histogram count and by too large values. * Measure the time starting from the process creation instead of ForegroundStartTime. Currently we don't cover the important part of the startup latency because the ForegroundStartTime is measured after the native library is loaded. That's a lot of important data missed. * Discard cases when the application was run in background (offliner, AMP background tab, etc.) See http://crbug.com/711708 as an example of unexpected order of events causing a crash. * Discard cases when a user action interrupts the startup sequence. It may shift the histogram to smaller values because users will likely interrupt the startup if it takes too long. It may be useful to create a separate histogram with "Uninterrupted" suffix instead of the discarding. Other paths of Android startup speed metrics development: * Make sure that we cover all user stories on Android such as: * Start with an intent * Start from the app launcher * Open Chrome from PWA shortcut * CustomTabs with warmup() * WebView startup * Add startup histograms for First{,Contentful,Meaningful}PaintTime. * Make Android startup metrics consistent with desktop. Startup metrics used on desktop platforms are tracked by chrome-speed-releasing@ and Chirp heartbeat metrics (so there's visibility and process if it regresses). In particular, we could implement some subset of Startup.FirstWebContents.* on Android.
,
Oct 26 2017
,
Oct 26 2017
,
Oct 26 2017
I made the document public, and updated the link
,
Oct 26 2017
,
Oct 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ec2faca5c123958f778edb083072d43372855ca6 commit ec2faca5c123958f778edb083072d43372855ca6 Author: Alexandr Ilin <alexilin@chromium.org> Date: Thu Oct 26 15:52:52 2017 Fix MobileStartup.TimeSinceLastUse metric The time delta between the start time and the last used time is compared with Integer.MAX_VALUE before casting it to int but it's compared using a wrong operator. Bug: 778302 Change-Id: I8cce18dd327766d54310a71fa897b65108a6cab9 Reviewed-on: https://chromium-review.googlesource.com/738244 Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Reviewed-by: Ted Choc <tedchoc@chromium.org> Reviewed-by: Alexei Svitkine <asvitkine@chromium.org> Cr-Commit-Position: refs/heads/master@{#511830} [modify] https://crrev.com/ec2faca5c123958f778edb083072d43372855ca6/chrome/android/java/src/org/chromium/chrome/browser/metrics/StartupMetrics.java [modify] https://crrev.com/ec2faca5c123958f778edb083072d43372855ca6/tools/metrics/histograms/histograms.xml
,
Oct 30 2017
,
Dec 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/df1ab7e5662d4cf383e66bf3100ee69270b916f9 commit df1ab7e5662d4cf383e66bf3100ee69270b916f9 Author: Alexandr Ilin <alexilin@chromium.org> Date: Mon Dec 04 14:26:24 2017 Pass navigation id to Java page load metrics observers Page load metrics observers in Java are not tied to a single navigation as in C++. Instead of that they receive all metrics events as long as they are registered in a global observer list. Even though a pointer to web contents is passed in each function, it's impossible to attribute a given event to particular navigation. This CL fixes this by passing a navigation id along with web contents pointer to all page load metrics observers in Java. Bug: 778302 Change-Id: I753a44abde849ce34b9472a57f14b23465e17982 Reviewed-on: https://chromium-review.googlesource.com/801015 Reviewed-by: Benoit L <lizeb@chromium.org> Reviewed-by: Bryan McQuade <bmcquade@chromium.org> Reviewed-by: David Trainor <dtrainor@chromium.org> Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Cr-Commit-Position: refs/heads/master@{#521333} [modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/java/src/org/chromium/chrome/browser/customtabs/CustomTabActivity.java [modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/java/src/org/chromium/chrome/browser/metrics/PageLoadMetrics.java [modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java [modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/browser/page_load_metrics/observers/android_page_load_metrics_observer.cc [modify] https://crrev.com/df1ab7e5662d4cf383e66bf3100ee69270b916f9/chrome/browser/page_load_metrics/observers/android_page_load_metrics_observer.h
,
Dec 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8be495bb2adb20e7b74309ae213796c848d4526d commit 8be495bb2adb20e7b74309ae213796c848d4526d Author: Alexandr Ilin <alexilin@chromium.org> Date: Tue Dec 05 19:06:27 2017 android: Fix Startup.FirstCommitNavigationTime histogram This CL makes following changes to the Startup.FirstCommitNavigationTime3 histogram: * Do not record the histogram multiple times per session * Start recording from the application start time instead of the foreground time * Record the histogram only for ChromeTabbedActivity * Ignore foreground state, there will be separate histogram for this purpose * Discard error, same document and non http(s) pages The CL also renames the histogram to Startup.Android.Experimental.Cold.TimeToFirstNavigationCommit The rationale behind the recording the histogram from the application start time instead of the foreground time: The current metric doesn't cover the important part of the startup latency because the ForegroundStartTime is measured after the native library is loaded. That's a lot of important data missed. We're exploring various ways to speed up the startup including the time of native library initialization. We want to have a metric that will show us these improvements. Initially, this metric was measuring cold startups from the application start but then it was changed. We would like to return to the initial meaning. pasko@ described an evolution of the metric in https://goo.gl/NyvX39 Bug: 778302 Change-Id: I01d7a6a73fbc504e31c1e877b1253fcf69a54ad8 Reviewed-on: https://chromium-review.googlesource.com/788875 Reviewed-by: Yaron Friedman <yfriedman@chromium.org> Reviewed-by: David Trainor <dtrainor@chromium.org> Reviewed-by: Alexei Svitkine <asvitkine@chromium.org> Reviewed-by: Egor Pasko <pasko@chromium.org> Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Cr-Commit-Position: refs/heads/master@{#521771} [modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java [modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java/src/org/chromium/chrome/browser/tab/TabWebContentsObserver.java [modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/java_sources.gni [add] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java [modify] https://crrev.com/8be495bb2adb20e7b74309ae213796c848d4526d/tools/metrics/histograms/histograms.xml
,
Dec 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29 commit 7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29 Author: Alexandr Ilin <alexilin@chromium.org> Date: Wed Dec 20 19:52:52 2017 android: Record FirstContentfulPaint startup histogram This CL adds a new startup histogram Startup.Android.Experimental.Cold.TimeToFirstContentfulPaint. The histogram is recorded only for the first navigation in the application lifetime under the same conditions as Startup.Android.Experimental.Cold.TimeToFirstNavigationCommit. Also this CL adds a C++ UMA_HISTOGRAM_COUNTS_100 equivalent to the Java side, because now it's used by multiple clients. Bug: 778302 Change-Id: I8f81fe70c90e8c9ce9be597f22052786c8d28f63 Reviewed-on: https://chromium-review.googlesource.com/823847 Reviewed-by: Egor Pasko <pasko@chromium.org> Reviewed-by: agrieve <agrieve@chromium.org> Reviewed-by: Bernhard Bauer <bauerb@chromium.org> Reviewed-by: Mark Pearson <mpearson@chromium.org> Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Cr-Commit-Position: refs/heads/master@{#525417} [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/base/android/java/src/org/chromium/base/metrics/RecordHistogram.java [add] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/metrics/StartupPageLoadMetricsObserver.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/Tab.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/TabUma.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java/src/org/chromium/chrome/browser/tab/TabWebContentsObserver.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/java_sources.gni [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java [modify] https://crrev.com/7035f55ca3f8ca4f7be82e8c132ed2d8443b9e29/tools/metrics/histograms/histograms.xml
,
Jan 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec commit b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec Author: Alexandr Ilin <alexilin@chromium.org> Date: Thu Jan 18 13:09:04 2018 Record startup loading histograms only in foreground This CL discards recording startup histograms Startup.Android.Experimental.Cold.TimeToFirst{ContentfulPaint,NavigationCommit} if the application wasn't in the foreground state since start till the end of event. The reason of this change is that currently these histograms have long tail with unrealistically high timings to load the first page. It's okay to modify these histograms without renaming because they are marked as experimental. Bug: 778302 Change-Id: I33341f34506b75e8671c21c0e19aeb4e9172d75d Reviewed-on: https://chromium-review.googlesource.com/840003 Reviewed-by: Mark Pearson <mpearson@chromium.org> Reviewed-by: Bernhard Bauer <bauerb@chromium.org> Reviewed-by: Egor Pasko <pasko@chromium.org> Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Cr-Commit-Position: refs/heads/master@{#530129} [modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/java/src/org/chromium/chrome/browser/firstrun/FirstRunActivityBase.java [modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java [modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java [modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/chrome/test/android/javatests/src/org/chromium/chrome/test/util/ApplicationTestUtils.java [modify] https://crrev.com/b28bdd7d0fc2d0afa71a387717ccc0d7264bcbec/tools/metrics/histograms/histograms.xml
,
Feb 7 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9bfff2d16aec309d883ea7a066ffe1b42de465ce commit 9bfff2d16aec309d883ea7a066ffe1b42de465ce Author: Alexandr Ilin <alexilin@chromium.org> Date: Wed Feb 07 14:48:09 2018 Fix flaky testBackgroundedPageNotRecorded test This test doesn't use ActivityTestRule.startMainActivityFromIntent becuase it needs to put an activity in background before the navigation is committed. Then the test doesn't check that the tab was created and starts the second load that fails. Apparentely, the issue is reproduced only on slow devices, because before the second load the main activity is put in background and then back in foreground. Bug: 778302 Change-Id: I720689733b12907a6da1cc986dea1635df8bf2d9 Reviewed-on: https://chromium-review.googlesource.com/906427 Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Reviewed-by: Bernhard Bauer <bauerb@chromium.org> Cr-Commit-Position: refs/heads/master@{#535003} [modify] https://crrev.com/9bfff2d16aec309d883ea7a066ffe1b42de465ce/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/PageLoadMetricsTest.java [modify] https://crrev.com/9bfff2d16aec309d883ea7a066ffe1b42de465ce/chrome/android/javatests/src/org/chromium/chrome/browser/metrics/StartupLoadingMetricsTest.java
,
Mar 22 2018
,
Mar 22 2018
,
Mar 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/47b68d843471c211ef22d916752eeea1a4b9f702 commit 47b68d843471c211ef22d916752eeea1a4b9f702 Author: Alexandr Ilin <alexilin@chromium.org> Date: Tue Mar 27 09:07:53 2018 android: Measure startup histograms starting from activity creation This CL starts using activity creation time instead of applicaton creation time for the Startup.Android.Experimental.Cold.TimeToFirst{ContentfulPaint,NavigationCommit} histograms. It's done in order to reduce histograms noise. These histograms are still experimental, so renaming isn't required. Bug: 778302 Change-Id: Ic615c14d62566bb31fec651e706ded158abf9d99 Reviewed-on: https://chromium-review.googlesource.com/977968 Reviewed-by: Maria Khomenko <mariakhomenko@chromium.org> Reviewed-by: Jesse Doherty <jwd@chromium.org> Reviewed-by: Egor Pasko <pasko@chromium.org> Commit-Queue: Alexandr Ilin <alexilin@chromium.org> Cr-Commit-Position: refs/heads/master@{#546068} [modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/chrome/android/java/src/org/chromium/chrome/browser/init/AsyncInitializationActivity.java [modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaUtils.java [modify] https://crrev.com/47b68d843471c211ef22d916752eeea1a4b9f702/tools/metrics/histograms/histograms.xml
,
Nov 20
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by pasko@chromium.org
, Oct 26 2017