New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 883380 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner:
Closed: Sep 12
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

13% regression in loading.desktop at 589801:589801

Project Member Reported by ushesh@chromium.org, Sep 12

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=883380

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=d8df3d6e37019a9d1099c59caf9be6049ff6c3f015e3ff4996805afb226d1e7b


Bot(s) for this bug's original alert(s):

mac-10_12_laptop_low_end-perf

loading.desktop - Benchmark documentation link:
  https://bit.ly/loading-benchmarks
Owner: nedngu...@google.com
Status: Assigned (was: Untriaged)
Assigning to nednguyen@google.com because this is the only CL in range:
Fix a bug about ordering of stories in generate_perf_sharding

** PERF SHERIFF: this CL may cause perf graphs to regress but it's expected **


Previously, there is a big discrepancy between the predicted timing of
generate_perf_sharding's output & the actual timing. For details, see:
* Predicted timing: https://logs.chromium.org/v/?s=chrome%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8935880309586521856%2F%2B%2FBenchmarks_Shard_Map_ca71405d-0fb3-4722-a815-af11b3f08318
* Actual timing: https://ci.chromium.org/p/chrome/builders/luci.chrome.ci/android-go-perf/182

After debugging, I found the root cause of this discrepancy is due to the order
of stories input in generate_perf_sharding isn't the same of the order of stories
which are meant to be run. This is because we mistakenly sort the list of
story names in benchmark_utils.GetBenchmarkStoryNames(..) method.

This CL fixes it by removing the sorting logic. It also add unittest to ensure
that the order of story names return by the method always matches with the
order of stories to be run.

Bug:863768
Cq-Include-Trybots: master.tryserver.chromium.perf:obbs_fyi
Change-Id: I769ff867b0ee10a9afafafebbd4cb2621e0ab93f

TBR=jbudorick@chromium.org, perezju@chromium.org
NOTRY=true # content_browsertests flake

Change-Id: I769ff867b0ee10a9afafafebbd4cb2621e0ab93f
Reviewed-on: https://chromium-review.googlesource.com/1214806
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#589801}
Mergedinto: 883393
Status: Duplicate (was: Assigned)

Sign in to add a comment