All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=626296

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsozatQkM


Bot(s) for this bug's original alert(s):

chromium-rel-mac10

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : ppapi: PPB_VpnProvider: Implement Resource Host
Author  : adrian.belgun
Commit description:
  
BUG=506490

Review-Url: https://codereview.chromium.org/1735473002
Cr-Commit-Position: refs/heads/master@{#403909}
Commit  : 8a1558d86fa478c65aba0b268129a9d8f2d3a7af
Date    : Wed Jul 06 15:21:13 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N    Good?
chromium@403908  98.4272  3.10031   710  good
chromium@403909  98.1899  1.76435   710  bad    <--
chromium@403910  98.4467  3.06431   473  bad
chromium@403912  107.802  1.25914   5    bad
chromium@403916  108.087  0.855968  5    bad
chromium@403923  107.278  1.34127   5    bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 626296

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.paint
Test Metric: large-table-background-change-with-visible-collapsed-borders/large-table-background-change-with-visible-collapsed-borders
Relative Change: 12.88%
Score: 0.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2185
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007800604731656864


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5887558253281280

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

sullivan@: This bisect result processing seems broken to me in this case! The mean clearly jumps from 98 to 107 between 403910:403912 but the bisect seem to be ignoring that and finding a . Am I missing something?

If my interpretation is correct the bug appears to be related to an Skia roll: https://chromium.googlesource.com/skia.git/+log/1e2a702c1959..4be9a30aed39


In anycase the regression seems to have recovered on its own so maybe we should close this issue?!

This is a freak result, not broken logic. an outlier data point (with a magnitude of ~125) in the first 5 tests of a preregression rev (403910)  made us mistakenly classify it as post regression. hence the huge number of re-tests.

I would just manually launch a bisect between 403910 and 403912 and that should point to the real culprit. Or just repeat the bisect, as it I think it very unlikely that it'll do the same thing again.

The culprit is most likely 911 or 912.

Deleted: figure_1.png 87.4 KB

	figure_1.png 87.4 KB View Download

The regressions have recovered, should we still put effort to find the culprit?

Roberto, not sure why you wanted this open, but making you owner. Feel free to close if that was a mistake.

The regression is definitely recovered. So I don't think we should spend more time on it.

I am not a paint expert but looking at the regression and recovery they both include a skia roll and not much else. So I guess it was a Skia regression which is now fixed.