New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 741730 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Sep 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

66.3% regression in loading.desktop at 484779:484843

Project Member Reported by pmeenan@chromium.org, Jul 12 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Jul 12 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=741730

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=c11f834aff12964bc91dac8e9b0211b7cfd0e44f34a79aade32b11ba11497375


Bot(s) for this bug's original alert(s):

chromium-rel-win7-gpu-intel
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jul 12 2017

Cc: rch@chromium.org
Owner: rch@chromium.org

=== Auto-CCing suspected CL author rch@chromium.org ===

Hi rch@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : rch
  Commit : d6e1cab1edf5f432d2ce09ff8ee34fe14abaff5a
  Date   : Fri Jul 07 03:59:30 2017
  Subject: Enable QUIC v39 by default.

Bisect Details
  Configuration: winx64intel_perf_bisect
  Benchmark    : loading.desktop
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-warm/Pantip
  Change       : 63.57% | 490.478833333 -> 802.2575

Revision             Result                  N
chromium@484778      490.479 +- 11.7297      6      good
chromium@484811      486.416 +- 9.34123      6      good
chromium@484819      487.524 +- 10.3523      6      good
chromium@484823      484.647 +- 6.56154      6      good
chromium@484825      486.677 +- 6.57639      6      good
chromium@484826      801.602 +- 13.7283      6      bad       <--
chromium@484827      800.057 +- 5.4807       6      bad
chromium@484843      802.257 +- 7.90196      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Pantip loading.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8974256353391049824


For feedback, file a bug with component Speed>Bisection

Comment 4 by rch@chromium.org, Jul 12 2017

I don't think there's any chance that this CL caused the regression, but I guess it's possible? I'd be quite surprised if these tests actually used QUIC. Where could I get more information on the details of the test?
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jul 12 2017

 Issue 741680  has been merged into this issue.
Owner: ----
Pretty sure it's a bad bisect from a noisy test.  Most of the tests use WebPageReplay locally with no QUIC (or even HTTP/2) support.
The graphs are noisy but the regression is significantly above the level of noise.  Lets try again and see what they find.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jul 17 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : rch
  Commit : d6e1cab1edf5f432d2ce09ff8ee34fe14abaff5a
  Date   : Fri Jul 07 03:59:30 2017
  Subject: Enable QUIC v39 by default.

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : loading.desktop
  Metric       : timeToFirstMeaningfulPaint_avg/pcv1-warm/MLB
  Change       : 114.05% | 985.535777781 -> 1886.68511111

Revision             Result                  N
chromium@484780      985.536 +- 949.028      9       good
chromium@484818      1187.31 +- 1351.38      9       good
chromium@484823      1326.88 +- 1017.78      6       good
chromium@484825      1155.84 +- 873.692      6       good
chromium@484826      1891.35 +- 611.385      9       bad       <--
chromium@484828      1904.1 +- 425.757       14      bad
chromium@484837      1803.8 +- 942.075       14      bad
chromium@484855      1886.69 +- 736.962      9       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=MLB loading.desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973828812422214928


For feedback, file a bug with component Speed>Bisection
Cc: kouhei@chromium.org xunji...@chromium.org
+xunjieli on this really old bug. Helen, do you think this is worth investigating further? I'm not clear how a quic change could have affected old WPR. Adding test owner in case we decide to wontfix.
Status: WontFix (was: Untriaged)
I agree that this looks odd. The QUIC version bump shouldn't affect the loading metrics because QUIC paths weren't covered by WPR. 

Let's wontFix this. kouhei@: please re-open if further investigation is needed.

Sign in to add a comment