New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 783904 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

10.6%-90.7% regression in system_health.common_mobile at 515086:515178

Project Member Reported by alexclarke@chromium.org, Nov 10 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Nov 10 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=783904

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=27941cd9d9c0f0639ecbc92d8dbec44f1fa42dbc5fe5132e2a6f9e1b93554728


Bot(s) for this bug's original alert(s):

android-nexus5X
android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Nov 11 2017

Cc: benjhayden@chromium.org
Owner: benjhayden@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author benjhayden@chromium.org ===

Hi benjhayden@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Ben Hayden
  Commit : c169bba255815467c5a503d505f4e5cb088fff42
  Date   : Tue Nov 07 17:58:25 2017
  Subject: Make ChartJsonConverter add shared diagnostics.

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : total:500ms_window:renderer_eqt_max/background_news/background_news_nytimes
  Change       : 82.92% | 783.325333333 -> 1432.85616667

Revision                                 Result                  N
chromium@515139                          783.325 +- 245.026      6      good
chromium@515144                          767.898 +- 152.831      6      good
chromium@515147                          743.306 +- 255.476      6      good
chromium@515148                          758.809 +- 227.296      6      good
chromium@515148,catapult@a7fb6add75      772.715 +- 673.476      9      good
chromium@515148,catapult@564e33d345      1263.03 +- 645.368      6      good
chromium@515148,catapult@bbb1522e7f      1302.88 +- 192.573      6      good
chromium@515148,catapult@c169bba255      1472.71 +- 64.1961      6      bad       <--
chromium@515149                          1392.96 +- 211.194      6      bad
chromium@515159                          1392.19 +- 248.725      6      bad
chromium@515178                          1432.86 +- 88.8557      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8963282413721848208


For feedback, file a bug with component Speed>Bisection
Owner: dtu@chromium.org
Dave, any idea why bisect descended into catapult when http://crrev.com/515148 is an ICU roll?
Not sure what's happpening in #3 but trying a few more bisects.
Cc: nedngu...@google.com
Ah, it looks like it's r515149 and the output is printed incorrectly, but also it picked the wrong CL, the regression is between 564e33d345 and a7fb6add75, so probably one of:

https://chromium.googlesource.com/catapult.git/+/564e33d345ee080be36ad0e6b6107afbcd09df4d - "Update wprgo binaries to latest version"

https://chromium.googlesource.com/catapult.git/+/b33d83ad5d4f4f59a87d7a68efa50455fc10a76d - "[wpr-go] Ensure 'Date' is updated as current date in response header"

Ned, is this worth investigating, or should we just conclude this is a bug fix?
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jan 12 2018


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : total:500ms_window:renderer_eqt_max/background_news/background_news_nytimes


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8957569740992450864


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jan 13 2018

Owner: nedngu...@google.com

=== Auto-CCing suspected CL author nednguyen@google.com ===

Hi nednguyen@google.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Nghia Nguyen
  Commit : 564e33d345ee080be36ad0e6b6107afbcd09df4d
  Date   : Tue Nov 07 14:06:23 2017
  Subject: Update wprgo binaries to latest version

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : total:500ms_window:renderer_eqt_max/background_news/background_news_nytimes

Revision                                 Result                  N
chromium@514869                          885.401 +- 245.543      6      good
chromium@515033                          936.829 +- 205.955      6      good
chromium@515115                          1034.38 +- 326.385      6      good
chromium@515136                          717.214 +- 68.0969      6      good
chromium@515146                          741.704 +- 163.11       6      good
chromium@515148                          759.628 +- 173.938      6      good
chromium@515148,catapult@a7fb6add75      780.751 +- 595.799      9      good
chromium@515148,catapult@b33d83ad5d      747.024 +- 127.348      6      good
chromium@515148,catapult@564e33d345      1355.62 +- 349.404      9      bad       <--
chromium@515148,catapult@c169bba255      1407.35 +- 280.023      9      bad
chromium@515149                          1428.74 +- 289.14       6      bad
chromium@515151                          1400.11 +- 180.514      6      bad
chromium@515156                          1406.65 +- 214.437      6      bad
chromium@515197                          1399.14 +- 327.955      6      bad
chromium@515524                          1396.07 +- 230.437      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8957569706669332496


For feedback, file a bug with component Speed>Bisection
Status: WontFix (was: Assigned)
This is a test change
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jan 25 2018

Issue 776782 has been merged into this issue.

Sign in to add a comment