All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=767952

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=bc3c9ec9f5c6c779e8ce5bed544fdb3453bf62c56e38c7ce573b18286450d584


Bot(s) for this bug's original alert(s):

chromium-rel-win10
chromium-rel-win7-dual
chromium-rel-win7-x64-dual

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8967729815806649856

=== Auto-CCing suspected CL author hubbe@google.com ===

Hi hubbe@google.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Fredrik Hubinette
  Commit : cbd817fb8b417bbe16994b62f31e895cbee7e848
  Date   : Wed Sep 20 21:25:42 2017
  Subject: Benchmark page testing video load time when javascript thread is busy.

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : media.tough_video_cases
  Metric       : time_to_play/tulip2.mp4
  Change       : 36.73% | 97.3925 -> 133.160833333

Revision             Result                  N
chromium@503214      97.3925 +- 14.7041      6      good
chromium@503250      85.1267 +- 11.893       6      good
chromium@503251      85.4758 +- 7.21818      6      good
chromium@503252      124.388 +- 27.157       6      bad       <--
chromium@503253      125.609 +- 17.2823      6      bad
chromium@503255      125.617 +- 18.9414      6      bad
chromium@503259      124.957 +- 15.0668      6      bad
chromium@503268      123.471 +- 22.6347      6      bad
chromium@503285      133.161 +- 31.52        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8967729815806649856


For feedback, file a bug with component Speed>Bisection

 Issue 767951  has been merged into this issue.

How can there be a regression in a new benchmark?
Did I do something wrong when naming/tagging the benchmark?

There are two issues here: 

1. Despite lots of effort put to not alerting on summary metrics (https://bugs.chromium.org/p/chromium/issues/detail?id=721894#c6), it looks like we are still alerting on them. +simonhatch has the bug assigned to him.

2. The framework is also averaging out all the times we play tulip2.mp4. I need to figure out where it does this and fix it. See this: https://chromeperf.appspot.com/report?sid=a069b7ce8797a09d435bc39584752b93a8b918ae5130cb6a5315aa3ece131777&start_rev=497694&end_rev=504044

I looked into 2, and it looks like it only occurs for legacy benchmarks, not TBMv2 benchmarks. Since John is about to check in changes that will bring TBMv2 to parity with legacy benchmarks (https://codereview.chromium.org/3020433002/) (and deprecation of legacy media benchmarks will come right afterwards), I don't think trying to fix this is worth the effort.

1 is still an important problem to fix. +simonhatch@ is working on it https://github.com/catapult-project/catapult/issues/3903

2 is obsolete since legacy tough_video_cases is deleted. Duping 1 into  issue 721894