All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=693049

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg2KKg_QkM


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8987496837037448016

=== Auto-CCing suspected CL author rsesek@chromium.org ===

Hi rsesek@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : rsesek
  Commit : b663b64c6237463da5417e040681f689a45ce3e1
  Date   : Tue Feb 14 01:47:01 2017
  Subject: [Mac] Do not strip out Headers from KeystoneRegistration.framework.

Bisect Details
  Configuration: mac_retina_perf_bisect
  Benchmark    : startup.cold.blank_page
  Metric       : first_non_empty_paint_time/first_non_empty_paint_time
  Change       : 7.80% | 632.142857143 -> 681.428571429

Revision             Result                  N
chromium@450094      632.143 +- 265.514      14      good
chromium@450183      616.556 +- 116.414      9       good
chromium@450189      614.778 +- 110.957      9       good
chromium@450192      616.929 +- 95.1784      14      good
chromium@450193      619.0 +- 138.946        14      good
chromium@450194      692.667 +- 158.449      9       bad       <--
chromium@450205      679.556 +- 156.8        9       bad
chromium@450227      701.833 +- 156.993      6       bad
chromium@450271      681.429 +- 160.578      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8987496837037448016

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5213874929795072


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8987492644123184736

I'm not convinced by that bisect.

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : rsesek
  Commit : b663b64c6237463da5417e040681f689a45ce3e1
  Date   : Tue Feb 14 01:47:01 2017
  Subject: [Mac] Do not strip out Headers from KeystoneRegistration.framework.

Bisect Details
  Configuration: mac_retina_perf_bisect
  Benchmark    : startup.cold.blank_page
  Metric       : first_non_empty_paint_time/first_non_empty_paint_time
  Change       : 7.66% | 627.928571429 -> 676.0

Revision             Result                  N
chromium@449900      627.929 +- 231.786      14      good
chromium@450100      616.111 +- 88.9769      9       good
chromium@450150      622.667 +- 175.829      9       good
chromium@450175      616.643 +- 143.949      14      good
chromium@450188      614.556 +- 102.412      9       good
chromium@450191      613.5 +- 115.739        14      good
chromium@450193      617.889 +- 105.446      9       good
chromium@450194      675.556 +- 92.4458      9       bad       <--
chromium@450200      692.444 +- 152.795      9       bad
chromium@450300      676.0 +- 125.292        14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.cold.blank_page

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8987492644123184736

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5495538348195840


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

That doesn't make a whole lot of sense. The system does not consult headers when loading a framework (I verified this with dtruss), so the CL should have no impact. And per issue 688076, this is a temporary thing until we get a new copy of Keystone without the Headers directory in it.

+erikchen, shrike: Any ideas what could be going on here? The perf waterfall had a regression on startup, just on the mac retinas, and bisect consistently reproduces the regression at r450194, but in #7 it looks like that CL should have no impact.

Looking at the graph, both ref and non-ref seem to climb at a slow pace, and it's not immediately clear that this regression can be pinpointed [e.g. looking back, there's another spike that reaches almost the same level]. 

Given that bisect is failing to produce a likely candidate, I'm inclined to ignore the problem [WontFix]. It's possible that there's a real regression, somewhere in this range, but the effort needed to identify that change with high confidence is very high, whereas we still have lots of low-hanging performance fruit we can work on.

Deleted: Screen Shot 2017-02-16 at 1.40.32 PM.png 63.6 KB

	Screen Shot 2017-02-16 at 1.40.32 PM.png 63.6 KB View Download

 Issue 692962  has been merged into this issue.

rsesek@ - in your cl description you say that stripping out the headers "invalidates the code signature on Keystone, which then prevents the Chrome Framework.framework from being validated." So when we stripped the headers previously, what were the ramifications? Did we have to run a separate pass to clean things up? Or did we ship a framework that could not be validated?

We shipped a framework that would not be validated. I'm just going to WontFix this because issue 693274 is being fixed imminently.