All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=699749

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg5PzxoAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxNSSvwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghOO8_QkM


Bot(s) for this bug's original alert(s):

android-nexus5X
chromium-rel-win7-dual
chromium-rel-win7-x64-dual

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8985650330966194240

=== Auto-CCing suspected CL author dominickn@chromium.org ===

Hi dominickn@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : dominickn
  Commit : 5ae55b8c8ed39e3a1019afba988b5df208f53420
  Date   : Wed Feb 22 09:00:01 2017
  Subject: Remove the 10 second delay before user input generates engagement.

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : smoothness.top_25_smooth
  Metric       : first_gesture_scroll_update_latency/Wikipedia (1 tab)
  Change       : 1.65% | 15.251 -> 15.5021666667

Revision             Result                    N
chromium@451909      15.251 +- 0.146369        6      good
chromium@451939      15.2309 +- 0.241977       9      good
chromium@451941      15.233 +- 0.344758        9      good
chromium@451942      15.476 +- 0.270987        6      bad       <--
chromium@451943      15.4366 +- 0.193995       9      bad
chromium@451947      15.432 +- 0.120192        6      bad
chromium@451954      15.4452 +- 0.0892235      6      bad
chromium@451968      15.4772 +- 0.272739       6      bad
chromium@452027      15.5022 +- 0.0940257      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Wikipedia..1.tab. smoothness.top_25_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8985650330966194240

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5233330452168704


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Hmm, is a 1.65% change here significant?

Are you referring to the bisect change? That's over enough samples using Mann-Whitney U to compare. The bisect is also just using Wikipedia, but if you look at all of the graphs that regressed (link in comment #2), you'll see that a bunch of different pagesets regressed.

Finally, yes, 1.5% is significant. If we let every 1.65% percent regression pass without justification, Chrome would be even more janky than it is today.

Options: Root cause and fix or justify with tradeoffs.

Thanks for confirming (it's unclear from the report what the actual scale of this regression is. Also it was filed as a P2).

Out of curiosity, does the testing trigger input events to be sent to the browser process? The blamed CL removes a delay from when the site engagement service starts listening and responding to input events (keyboard, mouse, scroll, etc.). If the testing doesn't trigger those events I'm curious as to why there's so much extra jank induced (the only extra cost should be triggering a timer and firing a small callback). If the testing does trigger those events then the impact seems fairly understandable.

In any case, the CL is question is pretty minor, so a possible remedy is just reverting it and leaving the 10 second delay in there. +benwells for thoughts.

I'm a bit confused - the graphs (which look pretty bad - e.g. 1.8 -> 2.1ms) and the bisect details summary (which looks a lot less bad) don't seem to correspond. Ben - can you explain more? Sorry if this is a dumb question.

+victor who is the benchmark owner for comment#6
+annie who is the owner of the alerts and bisect for comment#7

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8985569596327720816

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8985569538616880656

I kicked off a few more bisects, but I agree with #7 that this is strange--the first_gesture_scroll_update_latency metric in the perf dashboard chart goes from ~1.75ms to ~2ms, and in the bisect it goes from 15.251 -> 15.5021666667. That's a big difference in absolute values between the different bots with same hardware and config. Victor, any idea what could be going on here?

Traces from the perfbot:
Before regression: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_11-2017-02-22_02-24-33-46454.html
After regression: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_11-2017-02-22_08-50-20-98687.html

A trace from the bisector:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-03-08_16-18-14-41676.html

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : dominickn
  Commit : 5ae55b8c8ed39e3a1019afba988b5df208f53420
  Date   : Wed Feb 22 09:00:01 2017
  Subject: Remove the 10 second delay before user input generates engagement.

Bisect Details
  Configuration: win_x64_perf_bisect
  Benchmark    : smoothness.top_25_smooth
  Metric       : first_gesture_scroll_update_latency/Wikipedia (1 tab)
  Change       : 20.29% | 1.60716666667 -> 1.93333333333

Revision             Result                    N
chromium@451909      1.60717 +- 0.164331       6      good
chromium@451938      1.65317 +- 0.119201       6      good
chromium@451940      1.65783 +- 0.0694754      6      good
chromium@451941      1.69733 +- 0.0924193      6      good
chromium@451942      1.919 +- 0.380247         6      bad       <--
chromium@451946      1.911 +- 0.160873         6      bad
chromium@451953      1.91517 +- 0.251051       6      bad
chromium@451967      1.935 +- 0.144651         6      bad
chromium@452025      1.93333 +- 0.130818       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Wikipedia..1.tab. smoothness.top_25_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8985569538616880656

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6282406278987776


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Were there any more comments on the discrepancy between the two bisect runs?

Anyway, I'm planning on reverting the CL.

Revert landed.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982651551963052304