New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 880091 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Sep 10
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug

Blocking:
issue 760498



Sign in to add a comment

Start running "startup.mobile" on Android performance lab

Project Member Reported by pasko@chromium.org, Sep 3

Issue description

The benchmark "startup.mobile" is graduating from experimental after this change lands:

https://chromium-review.googlesource.com/c/chromium/src/+/1202665

We want to make it run on all Android bots from the performance lab.
 
Blocking: 760498
Project Member

Comment 2 by bugdroid1@chromium.org, Sep 8

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2ec67f49e61fcb6d83b7744f2f957cc88181250e

commit 2ec67f49e61fcb6d83b7744f2f957cc88181250e
Author: Ned Nguyen <nednguyen@google.com>
Date: Sat Sep 08 09:02:44 2018

Reshard perf benchmarks to include startup.mobile benchmark

** PERF SHERIFF: this CL may cause perf graphs to regress but it's expected **


Note that this CL also fixes a bug in sharding algorithm which makes the sharding map generation not very deterministic
(see changes to tools/perf/core/sharding_map_generator.py).

Step to generate the json files:
remove 'startup.mobile' from UNSCHEDULED_TELEMETRY_BENCHMARKS, then run:
$ ./tools/perf/generate_perf_sharding -r
$ ./tools/perf/generate_perf_data

TBR=jbudorick@chromium.org, perezju@chromium.org

Bug:  880091 ,863768
Cq-Include-Trybots: master.tryserver.chromium.perf:obbs_fyi
Change-Id: I71a2622f979d451eba53ed89f567b82d6ce549e7
Reviewed-on: https://chromium-review.googlesource.com/1214758
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#589776}
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/benchmark.csv
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/bot_platforms.py
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/android-go-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/android-nexus5x-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/android_nexus5_perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/android_nexus5x_webview_perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/android_nexus6_webview_perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/linux-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/mac-10_12_laptop_low_end-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/mac-10_13_laptop_high_end-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/android-go-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/android-nexus5x-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/android_nexus5_perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/android_nexus5x_webview_perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/android_nexus6_webview_perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/linux-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/mac-10_12_laptop_low_end-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/mac-10_13_laptop_high_end-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/win-10-perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/win_7_nvidia_gpu_perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/timing_data/win_7_perf_timing.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/win-10-perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/win_7_nvidia_gpu_perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/shard_maps/win_7_perf_map.json
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/sharding_map_generator.py
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/core/undocumented_benchmarks.py
[modify] https://crrev.com/2ec67f49e61fcb6d83b7744f2f957cc88181250e/tools/perf/generate_perf_sharding

Status: Fixed (was: Assigned)
We have data arriving: https://chromeperf.appspot.com/report?sid=62b401381da7a399cd6ce4b342b11a2194833036acb7e64253a1719afc4351da

nednguyen: Awesome! thanks a lot! Special thanks for careful resharding to avoid false blames for concurrently landed changes!!
@pasko: you might want to file a bug against perf dashboard team (Component: "Speed>Dashboard" to monitor this benchmark
thanks! For now I'm 'monitoring' it, there are a few discrepancies to be cleared before I want to start confusing others (for example: library prefetch is too fast for coldish benchmark - we did not see it on FYI bot because that one did not run official builds .. etc)
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/81178301cc244738fec377090dd6dec112ff3b90

commit 81178301cc244738fec377090dd6dec112ff3b90
Author: Egor Pasko <pasko@chromium.org>
Date: Tue Sep 11 09:42:39 2018

Do not run startup.mobile on webview bots

The benchmark cannot report metrics correctly for WebView. Also WebView
cares about different startup scenarios and has its own stories for
that.

Bug:  880091 
Change-Id: I76bec3c24843a5429bfeec45a53439b19b3d7ef6
Reviewed-on: https://chromium-review.googlesource.com/1217103
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#590252}
[modify] https://crrev.com/81178301cc244738fec377090dd6dec112ff3b90/tools/perf/benchmarks/startup_mobile.py

Sign in to add a comment