benchmarking: JobHandler starts the process a lot before the startup benchmark sends an intent |
||
Issue descriptionThere are a few traces like the one attached. They record several browser starts and extract the length of event Startup.BrowserMessageLoopStartTimeFromMainEntry3 as a metric. The event starts in ChromeApplication.attachBaseContext(). On this trace the second attachBaseContext (at 20364ms) was triggered not because of the intent being sent, but because it was needed for android.app.job.JobService$JobHandler at 20413ms. Then the process was cached and the intent-based benchmark computed ~8 seconds of messageloop_start_time. This is an unrealistically large number, causing artificial noise in our measurements.
,
Sep 28
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9afce6048cbb4d737604ae1356a98876d5fbeaed commit 9afce6048cbb4d737604ae1356a98876d5fbeaed Author: Egor Pasko <pasko@chromium.org> Date: Fri Sep 28 19:55:19 2018 BackgroundTaskScheduler: allow ignoring jobs for benchmarking Introduce a commandline flag --ignore-background-tasks that would be set by Telemetry when running startup benchmarks to avoid random tasks from caching the process and skewing the metrics that are based on process creation. The bug has an example trace and more details. Bug: 890424 Change-Id: I456e0f15406ebba7c930f8ec445da0759e9a087d Reviewed-on: https://chromium-review.googlesource.com/1252382 Reviewed-by: Filip Gorski <fgorski@chromium.org> Commit-Queue: Egor Pasko <pasko@chromium.org> Cr-Commit-Position: refs/heads/master@{#595188} [modify] https://crrev.com/9afce6048cbb4d737604ae1356a98876d5fbeaed/components/background_task_scheduler/android/java/src/org/chromium/components/background_task_scheduler/BackgroundTaskSchedulerImpl.java
,
Oct 1
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/ad0631532d0cb16d2a72465147a5a8142ad0919a commit ad0631532d0cb16d2a72465147a5a8142ad0919a Author: Egor Pasko <pasko@chromium.org> Date: Mon Oct 01 15:17:05 2018 Avoid running Android background jobs **PERF SHERIFFS**: this should move metrics on Android. Likely startup, but there could be something else affected, like loading. Use the newly introduced flag in http://crrev.com/595188 to prevent spurious jobs from being created when benchmarking. Those jobs do little, but lead to Chrome process being cached, which artificially inflates the metrics that are based on process start. This should cut down those occasional stdev of 2 seconds. The bugs provide more context. Bug: chromium:890424 Bug: chromium:881384 Change-Id: Ifb599bff0e55041495aa87983262dfd0940df226 Reviewed-on: https://chromium-review.googlesource.com/1254101 Commit-Queue: Egor Pasko <pasko@chromium.org> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> [modify] https://crrev.com/ad0631532d0cb16d2a72465147a5a8142ad0919a/telemetry/telemetry/internal/backends/chrome/chrome_startup_args.py
,
Oct 1
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fc8fe0364d1fa8c20e308567a0d1a4340d27e046 commit fc8fe0364d1fa8c20e308567a0d1a4340d27e046 Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Mon Oct 01 22:31:14 2018 Roll src/third_party/catapult ac93684421fa..69f64b270397 (4 commits) https://chromium.googlesource.com/catapult.git/+log/ac93684421fa..69f64b270397 git log ac93684421fa..69f64b270397 --date=short --no-merges --format='%ad %ae %s' 2018-10-01 oysteine@google.com Allow bindId for separate begin/end slices as well 2018-10-01 simonhatch@chromium.org Dashboard - Create histograms per bot for alert statistics 2018-10-01 cbruni@chromium.org [pinpoint] Redirect to raw results.html on job completion 2018-10-01 pasko@chromium.org Avoid running Android background jobs Created with: gclient setdep -r src/third_party/catapult@69f64b270397 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:890025 ,chromium:876233, chromium:890424 , chromium:881384 TBR=sullivan@chromium.org Change-Id: I3fa5e360f1c7c98152f5078c58d9d41e34b66ebc Reviewed-on: https://chromium-review.googlesource.com/1254685 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#595601} [modify] https://crrev.com/fc8fe0364d1fa8c20e308567a0d1a4340d27e046/DEPS
,
Oct 2
|
||
►
Sign in to add a comment |
||
Comment 1 by pasko@chromium.org
, Sep 28