New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 692965 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

13.3% regression in smoothness.tough_webgl_cases at 447793:447893

Project Member Reported by alexclarke@chromium.org, Feb 16 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=692965

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg2MDsoQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg2K6EtQoM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-gpu-ati
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Feb 16 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: winx64ati_perf_bisect
  Benchmark    : smoothness.tough_webgl_cases
  Metric       : mean_frame_time/http___kenrussell.github.io_webgl-animometer_Animometer_tests_3d_webgl.html
  Change       : 4.45% | 70.3926666667 -> 73.5245

Suspected Commit Range
  3 commits in range
  https://chromium.googlesource.com/chromium/src/+log/f2095b9da969dd7decf3224fc96ffe8ef5f5455b..36075614a07a4242d459810b956073be7428e56a


Revision             Result                   N
chromium@447792      70.3927 +- 4.56839       6        good
chromium@447843      64.347 +- 8.56205        14       good
chromium@447849      65.4483 +- 9.03721       9        good
chromium@447853      67.697 +- 11.6135        14       good
chromium@447854      ---                      ---      build failure
chromium@447855      ---                      ---      build failure
chromium@447856      71.0651 +- 4.66569       14       bad
chromium@447868      73.2063 +- 0.907959      6        bad
chromium@447893      73.5245 +- 1.56093       6        bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...kenrussell.github.io.webgl.animometer.Animometer.tests.3d.webgl.html smoothness.tough_webgl_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8987514463137686512

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5301943167090688


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Feb 16 2017

Cc: blink-w3c-test-autoroller@chromium.org
Owner: blink-w3c-test-autoroller@chromium.org

=== Auto-CCing suspected CL author blink-w3c-test-autoroller@chromium.org ===

Hi blink-w3c-test-autoroller@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : blink-w3c-test-autoroller
  Commit : efb2cef53f5492a5b5c69b20c79a3eea713ded38
  Date   : Thu Feb 02 21:26:13 2017
  Subject: Import wpt@269611dbf9944b2cee53d53fba81c23a681da729

Bisect Details
  Configuration: winx64ati_perf_bisect
  Benchmark    : smoothness.tough_webgl_cases
  Metric       : frame_times/http___kenrussell.github.io_webgl-animometer_Animometer_tests_3d_webgl.html
  Change       : 6.17% | 69.055297489 -> 73.3136903717

Revision             Result                  N
chromium@447792      69.0553 +- 5.38339      6       good
chromium@447843      68.3362 +- 14.791       14      good
chromium@447844      67.8897 +- 6.02555      6       good
chromium@447845      64.0284 +- 9.80966      14      bad       <--
chromium@447850      69.8594 +- 18.3184      14      bad
chromium@447856      73.5553 +- 1.24569      6       bad
chromium@447868      73.4995 +- 1.07622      6       bad
chromium@447893      73.3137 +- 1.81427      6       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...kenrussell.github.io.webgl.animometer.Animometer.tests.3d.webgl.html smoothness.tough_webgl_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8987507077220876256

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6387027194413056


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Feb 16 2017

Cc: tapted@chromium.org
Owner: tapted@chromium.org

=== Auto-CCing suspected CL author tapted@chromium.org ===

Hi tapted@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : tapted
  Commit : e91c25b113011f055428c46609e19ce6550b41ea
  Date   : Thu Feb 02 22:43:45 2017
  Subject: Allow permission bubbles to participate in key event dispatch as if they were a Browser.

Bisect Details
  Configuration: winx64ati_perf_bisect
  Benchmark    : smoothness.tough_webgl_cases
  Metric       : mean_frame_time/http___kenrussell.github.io_webgl-animometer_Animometer_tests_3d_webgl.html
  Change       : 10.88% | 65.9171666667 -> 73.0898333333

Revision             Result                   N
chromium@447600      65.9172 +- 5.37912       6       good
chromium@447750      69.7378 +- 8.12213       9       good
chromium@447825      71.9349 +- 2.0587        9       good
chromium@447863      64.4378 +- 7.48932       9       good
chromium@447864      68.5959 +- 9.39844       14      bad       <--
chromium@447865      70.8896 +- 3.606         9       bad
chromium@447866      73.0617 +- 0.83061       6       bad
chromium@447868      73.2728 +- 0.982381      6       bad
chromium@447873      72.3512 +- 1.1578        6       bad
chromium@447882      71.9702 +- 6.37057       9       bad
chromium@447900      73.0898 +- 1.30251       6       bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...kenrussell.github.io.webgl.animometer.Animometer.tests.3d.webgl.html smoothness.tough_webgl_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8987507065951359248

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5600452588077056


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: zmo@chromium.org kbr@chromium.org
+kbr, zmo (benchmark owners): looking at the graphs, it seems like this was really noise and not a real regression:
https://chromeperf.appspot.com/group_report?bug_id=692965

The bisect is getting confused by the noise. Should we WontFix, and also consider less sensitive alerting thresholds?

Comment 9 by kbr@chromium.org, Feb 17 2017

Cc: kainino@chromium.org
Components: Blink>WebGL
Yes please. How do we adjust the alerting thresholds for this benchmark?

Status: WontFix (was: Untriaged)
Closing based on #9.
Labels: Performance-Responsiveness

Sign in to add a comment