New issue
Advanced search Search tips

Issue 747815 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

10.8% regression in v8.runtimestats.browsing_desktop at 487411:487550

Project Member Reported by jarin@google.com, Jul 24 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Jul 24 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=747815

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=d88631abecd42c887d447929c9af0bc4e8fcf3c1885a7e02c84e54f007968d9d


Bot(s) for this bug's original alert(s):

linux-release
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Jul 24 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : V8-Only:duration_avg/browse_news/browse_news_hackernews

Revision             Result                  N
chromium@487410      963.886 +- 161.324      21      good
chromium@487550      982.656 +- 159.901      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.hackernews v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973205581417077264


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jul 24 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : V8-Only:duration_avg/browse_news/browse_news_hackernews

Revision             Result                  N
chromium@487410      871.177 +- 144.031      21      good
chromium@487550      873.42 +- 126.821       21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.hackernews v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8973205590037464688


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 6 by sheriffbot@chromium.org, Jul 24 2017

Labels: Hotlist-Google
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jul 30 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : Total:duration_avg/browse_social/browse_social_facebook_infinite_scroll

Revision             Result                  N
chromium@487410      6526.5 +- 3920.73       21      good
chromium@487550      6305.06 +- 716.624      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972692415170350816


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Jul 30 2017

Cc: fmalita@chromium.org
Owner: fmalita@chromium.org

=== Auto-CCing suspected CL author fmalita@chromium.org ===

Hi fmalita@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Florin Malita
  Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1
  Date   : Tue Jul 18 17:13:13 2017
  Subject: Enable Skia's integral-translate bilerp optimization

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : V8-Only:duration_avg/browse_social/browse_social_facebook_infinite_scroll
  Change       : 6.83% | 4807.10933333 -> 5135.2045

Revision             Result                  N
chromium@487410      4807.11 +- 107.388      6      good
chromium@487480      4831.46 +- 551.666      9      good
chromium@487498      4828.93 +- 376.666      9      good
chromium@487503      4780.92 +- 403.603      9      good
chromium@487505      4823.43 +- 270.203      9      good
chromium@487506      4812.03 +- 427.417      9      good
chromium@487507      5083.28 +- 806.553      9      bad       <--
chromium@487515      5129.79 +- 269.583      9      bad
chromium@487550      5135.2 +- 89.2919       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972692400841878704


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jul 30 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Florin Malita
  Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1
  Date   : Tue Jul 18 17:13:13 2017
  Subject: Enable Skia's integral-translate bilerp optimization

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : JavaScript:duration_avg/browse_social/browse_social_facebook_infinite_scroll
  Change       : 5.37% | 2067.61983333 -> 2178.68516667

Revision             Result                  N
chromium@487410      2067.62 +- 63.9053      6       good
chromium@487480      2068.25 +- 51.8151      6       good
chromium@487498      2059.75 +- 63.7764      6       good
chromium@487503      2064.64 +- 124.121      6       good
chromium@487505      2045.34 +- 129.248      14      good
chromium@487506      2053.42 +- 297.076      14      good
chromium@487507      2141.14 +- 325.199      14      bad       <--
chromium@487515      2166.27 +- 109.581      6       bad
chromium@487550      2178.69 +- 79.5018      6       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972692430341891424


For feedback, file a bug with component Speed>Bisection
Cc: mtklein@chromium.org reed@google.com
Labels: Needs-Bisect
Status: (was: Assigned)
I'm suspicious of the blame here:

1) https://chromium-review.googlesource.com/c/575675 should only affect raterization (no JS/v8 impact), and should be a strict improvement (disables image filtering in some cases).

2) the bisect identifies a 5-6% regression, but the graphs show a 60%+ regression (e.g. https://chromeperf.appspot.com/report?sid=df06f0ac7568adf3ef1472691c2bc30a5dbca6008df5c4fbc63b6d68cf368b75)

3) in local testing the numbers are quite noisy (definitely more than 5%).

Is there a way to re-run the blame bisect with a higher threshold?
Status: Assigned
Cc: sullivan@google.com
 Issue 751154  has been merged into this issue.
I see the bisect for  issue 751154  is also zooming in on https://chromium-review.googlesource.com/c/575675.  

I cannot repro the regression locally, and, more importantly, the actual tests don't even hit the code affected by the CL.

 I've verified with an instrumented build that neither canvas_xxxxx_pixels_per_second nor 
text_xxxxx_pixels_per_second are drawing any images, and are not reaching SkBitmapProcInfo::init() at all.

So something's off.  Not sure why the bisect keeps picking on https://chromium-review.googlesource.com/c/575675, but I'm pretty sure it cannot be responsible for these regressions.

=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : thread_times.tough_scrolling_cases
  Metric       : thread_raster_cpu_time_per_frame/text_30000_pixels_per_second
  Change       : 20.37% | 0.905910612795 -> 1.05918930715

Suspected Commit Range
  5 commits in range
  https://chromium.googlesource.com/chromium/src/+log/cebd3b079bbbe1716c0c4b5dc0ae27c03f5876a7..13b1999a87a545bcab93d66a2294758c3125a84b


Revision             Result                    N
chromium@487410      0.905911 +- 0.300258      14       good
chromium@487451      0.920002 +- 0.362076      21       good
chromium@487474      0.901417 +- 0.124938      9        good
chromium@487484      0.910623 +- 0.143609      14       good
chromium@487485      ---                       ---      build failure
chromium@487486      ---                       ---      build failure
chromium@487487      ---                       ---      build failure
chromium@487488      ---                       ---      build failure
chromium@487489      0.961421 +- 0.378532      14       bad
chromium@487494      0.973666 +- 0.217047      14       bad
chromium@487577      1.04894 +- 0.287142       9        bad
chromium@487743      1.00167 +- 0.153934       9        bad
chromium@488076      1.03938 +- 0.271104       9        bad
chromium@488741      1.05919 +- 0.189451       6        bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=text.30000.pixels.per.second thread_times.tough_scrolling_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972339980654200176


For feedback, file a bug with component Speed>Bisection
Cc: mythria@chromium.org
+mythria: any ideas how to help debug what could be going on here? See #13 and #17
I am not quite sure, the bisects on different metrics point to the same cl. This makes me suspect that we are doing more work (for ex: scrolling more) because of some side effect of the change pointed by the bisect. If it somehow changes the amount we scroll, it could impact the amount of javascript we execute and hence regress V8 metrics. I will trigger another bisect with a slightly different range. If that doesn't succeed or doesn't point us to anything different then I think we could close it as won't fix. 
I was only referring to regressions on V8 metrics when I said we can mark it as won't fix. I am not so sure about other regressions on thread_times.tough_scrolling_cases.
The thread_times bisect hit build failures, but it looks like a different range of CLs were narrowed in on. I split them into  bug 752457 .

mythria: as the owner of the benchmark, feel free to WontFix this based on results from the bisect since this bug is now just about v8.runtimestats.browsing_desktop.
sullivan@, mythria@ thanks for helping triage this.

Re. V8 metrics, handwaving around mythria's theory: the net effect of the CL is lower rasterization cost in some cases (when images are drawn with an integral translation only, we skip bilerp filtering); so if the tests are somehow synchronized on rasterization times, it sounds plausible that they'd be seeing more JS callbacks when frames take less time to raster.

FYI, https://chromium-review.googlesource.com/c/575675 was a fix for  issue 744674 , and its effect can be seen on those graphs: https://chromeperf.appspot.com/group_report?bug_id=744674

Come to think of it, many of those improved metrics appear related to scroll update latency - so mythria's explanation definitely makes sense: reduced latency -> increased frequency -> more V8 work.  Based on that, I now think the bisect is correct and we can close this as wontfix.

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Florin Malita
  Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1
  Date   : Tue Jul 18 17:13:13 2017
  Subject: Enable Skia's integral-translate bilerp optimization

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : v8.runtimestats.browsing_desktop
  Metric       : JavaScript:duration_avg/browse_social/browse_social_facebook_infinite_scroll
  Change       : 5.18% | 2054.71571429 -> 2178.05816667

Revision             Result                  N
chromium@487500      2054.72 +- 229.549      14      good
chromium@487504      2033.07 +- 111.48       9       good
chromium@487506      2045.37 +- 88.4205      9       good
chromium@487507      2108.64 +- 508.961      14      bad       <--
chromium@487513      2161.28 +- 200.657      9       bad
chromium@487525      2164.22 +- 177.437      9       bad
chromium@487550      2213.77 +- 123.371      6       bad
chromium@487600      2171.32 +- 184.056      9       bad
chromium@487700      2178.06 +- 96.34        6       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972197505818636800


For feedback, file a bug with component Speed>Bisection
Status: WontFix (was: Assigned)
Great, the bisect came back with the same result, now expected.  I think we're good here.
Cc: sullivan@chromium.org
 Issue 752457  has been merged into this issue.

Sign in to add a comment