New issue
Advanced search Search tips

Issue 758582 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

51.3% regression in system_health.common_desktop at 495263:495450

Project Member Reported by nzolghadr@chromium.org, Aug 24 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Aug 24 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=758582

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=2d9a2df5d2bf69cb8495081109424f76cfda30dfeb9757dee13a45d317e5d2a0


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 24 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_retina_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstPaint_avg/load_social/load_social_pinterest

Revision             Result                  N
chromium@495262      543.589 +- 271.692      21      good
chromium@495450      532.512 +- 291.782      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.social.pinterest system_health.common_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970369719790197136


For feedback, file a bug with component Speed>Bisection
Trying a different page and a wider revision range.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Aug 25 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: mac_retina_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstPaint_avg/browse_news/browse_news_reddit


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.reddit system_health.common_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970364051126240784


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Aug 26 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : mousewheel_response:energy_sum/browse_media/browse_media_pinterest


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.media.pinterest system_health.common_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970258049537756752


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Sep 21 2017

๐Ÿ“ Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/148cba10780000
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Sep 21 2017

๐Ÿ“ Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/16c22180780000
Infra failures and tests not producing values :(

Trying again with pinpoint.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Sep 21 2017

๐Ÿ“ Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/148cba10780000

Disabled failed WPT test
By fserb@google.com ยท Thu Aug 10 18:25:22 2017
chromium@844d849ed3ce28e2b7c54f2b12662d9306906866
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Sep 21 2017

๐Ÿ“ Pinpoint job completed.
https://pinpoint-dot-chromeperf.appspot.com/job/16c22180780000

Comment 14 by dtu@chromium.org, Sep 22 2017

#12 is https://github.com/catapult-project/catapult/issues/3878
Please ignore that result, it is not correct. I'm going to re-run it.

#13 is difficult, because the metric is so flaky that only about half the runs return results, so we don't get enough data. I've filed https://github.com/catapult-project/catapult/issues/3902. The Pinpoint version deployed this evening also has an increased default repeat count of 15, so I'm going to re-run this job too, but I think we need closer to 18 for it.
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Sep 22 2017

๐Ÿ“ Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/11b6783fb80000
Project Member

Comment 16 by 42576172...@developer.gserviceaccount.com, Sep 22 2017

๐Ÿ“ Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14b462a0780000

Comment 17 by dtu@chromium.org, Sep 22 2017

I also noticed while re-kicking them that the bisect ranges here are 7000 and 3500 commits, which are very big. They'll be interesting stress tests for the stability of Pinpoint's statistical methods.
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Sep 22 2017

๐Ÿ“ Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14b462a0780000

Found significant differences after 1 commit:

Avoid calling EmitLoginPromptVisible in voice interaction OOBE flow
By updowndota@chromium.org ยท Fri Aug 18 21:11:16 2017
chromium@b89f8fb6be9087064b00bfd1ba74e853a0fde122
Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Sep 22 2017

๐Ÿ“ Pinpoint job completed.
https://pinpoint-dot-chromeperf.appspot.com/job/11b6783fb80000

Found significant differences after 1 commit:

Use swap time for all first paint times in TBM, UMA, UKM & web perf API
By shaseley@google.com ยท Thu Aug 10 00:40:36 2017
chromium@c3f876b18c07c50b360b93fb91ec67549b58deb8
Owner: panicker@chromium.org
Status: WontFix (was: Untriaged)
WontFix-ing as metrics change; assigning to panicker@ in case she thinks this should be reopened.

Sign in to add a comment