New issue
Advanced search Search tips

Issue 885059 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 886958
Owner:
Closed: Sep 19
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

17.8%-36.5% regression in v8.browsing_desktop at 589801:589801

Project Member Reported by ulan@google.com, Sep 18

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=885059

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=a6bd0bd092982c6b6feaee910fd583529f9399267fdfa0e7feadc84a90a8880f


Bot(s) for this bug's original alert(s):

mac-10_13_laptop_high_end-perf

v8.browsing_desktop - Benchmark documentation link:
  None
Owner: nedngu...@google.com
Status: Assigned (was: Untriaged)
Assigning to nednguyen@google.com because this is the only CL in range:
Fix a bug about ordering of stories in generate_perf_sharding

** PERF SHERIFF: this CL may cause perf graphs to regress but it's expected **


Previously, there is a big discrepancy between the predicted timing of
generate_perf_sharding's output & the actual timing. For details, see:
* Predicted timing: https://logs.chromium.org/v/?s=chrome%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8935880309586521856%2F%2B%2FBenchmarks_Shard_Map_ca71405d-0fb3-4722-a815-af11b3f08318
* Actual timing: https://ci.chromium.org/p/chrome/builders/luci.chrome.ci/android-go-perf/182

After debugging, I found the root cause of this discrepancy is due to the order
of stories input in generate_perf_sharding isn't the same of the order of stories
which are meant to be run. This is because we mistakenly sort the list of
story names in benchmark_utils.GetBenchmarkStoryNames(..) method.

This CL fixes it by removing the sorting logic. It also add unittest to ensure
that the order of story names return by the method always matches with the
order of stories to be run.

Bug:863768
Cq-Include-Trybots: master.tryserver.chromium.perf:obbs_fyi
Change-Id: I769ff867b0ee10a9afafafebbd4cb2621e0ab93f

TBR=jbudorick@chromium.org, perezju@chromium.org
NOTRY=true # content_browsertests flake

Change-Id: I769ff867b0ee10a9afafafebbd4cb2621e0ab93f
Reviewed-on: https://chromium-review.googlesource.com/1214806
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#589801}
Mergedinto: 886958
Status: Duplicate (was: Assigned)

Sign in to add a comment