All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=654725

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgjefEvwoM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8999098389067331776

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@423958  17.1759  0.255614  18  good
chromium@424027  17.1487  0.284851  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 654725

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests smoothness.top_25_smooth
Test Metric: frame_times/http___answers.yahoo.com
Relative Change: 0.02%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6996
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999098389067331776


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5851285767585792

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8999065743361469552

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@423958  17.0878  0.112475  18  good
chromium@424027  17.1708  0.201138  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 654725

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests smoothness.top_25_smooth
Test Metric: frame_times/http___answers.yahoo.com
Relative Change: 0.22%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6999
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999065743361469552


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5869170145624064

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8997727724883865184

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7022
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

These regressions happened before M56 branch. M56 is now in stable. These regressions made it to the stable channel. Marking wontfix.

Woah, this is depressing.

Do we have statistics on how often this happens?

Back when perf sheriffs took ownership of their bugs, I don't remember this approach being taken. Is this a recent thing?

+benhenry, do you have stats on # of WontFixed perf bugs, which are too old? Did you institute a specific policy for WontFix of milestones in stable? If so, where is the announcement?

Unfortunately, most perf sheriffs never really took ownership of their own bugs, and the backlog was instead whittled down by a Friday bug bash of speed infra team. So we would do this triage little by little, and although there was never a set time period for WontFix, there have always been a large number of bugs we weren't able to get to the bottom of. Generally things were WontFixed when bisect stopped working due to going too far back in time (which isn't a set time, just a set of known breakages per platform). Thank you very much for being a good long-term owner of the bugs you filed, but we've never been able to get that to be the status quo.

In this case I think that this bug was closed too hastily; I think we should try a wider bisect. Re-opening and kicking off.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988397146654509568

=== Auto-CCing suspected CL author jbauman@chromium.org ===

Hi jbauman@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : jbauman
  Commit : 7ac9b1b7052cc76fa7b3fef51d2dc447f55991d3
  Date   : Fri Oct 07 16:53:36 2016
  Subject: Add 1 second delay before gestures in smoothness.top_25_smooth

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : smoothness.top_25_smooth
  Metric       : frame_times/http___answers.yahoo.com
  Change       : 7.34% | 16.0329105552 -> 17.2094851161

Revision             Result                    N
chromium@423885      16.0329 +- 0.294525       6      good
chromium@423886      17.2021 +- 0.699778       6      bad       <--
chromium@423887      17.1368 +- 0.298111       6      bad
chromium@423888      17.0422 +- 0.584275       6      bad
chromium@423891      17.0517 +- 0.0103518      6      bad
chromium@423897      17.0256 +- 0.797711       6      bad
chromium@423909      17.0542 +- 0.585608       6      bad
chromium@423933      17.1136 +- 0.550822       6      bad
chromium@423981      17.2406 +- 0.58421        6      bad
chromium@424076      17.2095 +- 0.546214       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...answers.yahoo.com smoothness.top_25_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8988397146654509568

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6250288354164736


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

This is my idea. I'll write something up and share with the team.

ericrk: are you planning on looking into this bug?

jbauman's change modified the benchmark itself. It seems understandable that a change in timing could introduce noise in the results (although I wouldn't expect too much). Given that this wasn't a code change but a metrics change, the change was ~6%, and only happened on 2 sub-pages, I think this is a won't-fix.