Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8971077168652981968

=== Auto-CCing suspected CL author kerrnel@chromium.org ===

Hi kerrnel@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Test failure found with culprit

Suspected Commit
  Author : Greg Kerr
  Commit : 9705b7aaabf4c81d4e9cb13174338df9bde61cc4
  Date   : Tue Aug 15 19:42:42 2017
  Subject: Enable Finch Trial Test for MacV2Sandbox.

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : v8.runtime_stats.top_25
  Metric       : ArraySlice:duration_avg/https___en.wikipedia.org_w_index.php?title_Barack_Obama_veaction_edit

Revision             Exit Code      N
chromium@494399      0 +- N/A       1      good
chromium@494465      0 +- N/A       1      good
chromium@494498      0 +- N/A       1      good
chromium@494500      0 +- N/A       1      good
chromium@494501      0 +- N/A       1      good
chromium@494502      1 +- N/A       1      bad       <--
chromium@494506      1 +- N/A       1      bad
chromium@494514      1 +- N/A       1      bad
chromium@494530      1 +- N/A       1      bad
chromium@494660      1 +- N/A       1      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https...en.wikipedia.org.w.index.php.title.Barack.Obama.veaction.edit v8.runtime_stats.top_25

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8971077168652981968


For feedback, file a bug with component Speed>Bisection

Investigating.

+cbruni@, do you know what it means that the exit code regressed? I ran the benchmark locally and it says "PASSED."

When I run this locally it says:

tools/perf/run_benchmark -v --browser=exact --browser-executable=out/Official/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https...en.wikipedia.org.w.index.php.title.Barack.Obama.veaction.edit v8.runtime_stats.top_25

[ RUN      ] /var/folders/fr/62fvjlx56c7c_q9stqgsbjcw009p4c/T/tmpizeJAQ.html
[       OK ] /var/folders/fr/62fvjlx56c7c_q9stqgsbjcw009p4c/T/tmpizeJAQ.html (6669 ms)
(WARNING) 2017-08-16 17:01:32,334 timeline_based_measurement._ComputeTimelineBasedMetrics:325  Processing resulting traces took 6.780 seconds
(INFO) 2017-08-16 17:01:32,547 browser.Close:268  Closing browser (pid=82002) ...
(INFO) 2017-08-16 17:01:35,817 browser.Close:283  Browser is closed.
[       OK ] https://en.wikipedia.org/w/index.php?title=Barack_Obama&veaction=edit (65854 ms)
(INFO) 2017-08-16 17:01:35,900 cloud_storage.Insert:377  Uploading /var/folders/fr/62fvjlx56c7c_q9stqgsbjcw009p4c/T/tmpizeJAQ.html to gs://chrome-telemetry-output/trace-file-id_0-2017-08-16_17-01-35-28125.html
View generated trace files online at https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-08-16_17-01-35-28125.html for story https://en.wikipedia.org/w/index.php?title=Barack_Obama&veaction=edit
[  PASSED  ] 1 test.

Uhmm, I'm not very familiar with the telemetry version of these benchmarks.
I'd assume the same as what you did from the local output: the tests passes.

Maybe there is some slightly different configuration running on the bots?
From the screenshot generated by the failing bot (https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_45-2017-08-16_16-54-0715168.png) it seems we just crash in the browser (so not just an issue with missing data), hence probable the "exit code regressed".

Have you tried running several repetitions? With --pageset-repeat=20 or so, you might be able to repro a flaky tests (although the bisect job looked pretty reliable).

Ah, can you somehow enable the finch experiment by default?

Although I thought we run the browser with --enable-benchmarking which should disable any noise generated by activating different finch experiments.

kernel@, note that your code is only causing crashes on Mac 10.11, so if you're not running that locally then the crashes won't happen.

Is it safe to revert this while you investigate? You can try out potential fixes by launching perf try jobs on Mac 10.11 with

  ./tools/perf/run_benchmark try mac-10-11 v8.runtime_stats.top_25 --story-filter=https...en.wikipedia.org.w.index.php.title.Barack.Obama.veaction.edit v8.runtime_stats.top_25

The "exit code regression" means that the Telemetry benchmark went from passing to failing. In this case, it's because Chrome started crashing.

https://paste.googleplex.com/4721478469681152 shows the relevant section from the logs where it fails.

Thanks for all that info, I was able to trivially reproduce it with that. This is a one line fix so I have a CL coming through the try bots now.

Okay, great. Can you verify via the try job that the fix works before submitting it?

I'm submitting the fix regardless because it definitely addresses a crash that I can see locally, but I will run the perf try job in parallel, sure. Thanks.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7e90a57df0f9434204a2bbf37366ca21aa8a9978

commit 7e90a57df0f9434204a2bbf37366ca21aa8a9978
Author: Greg Kerr <kerrnel@chromium.org>
Date: Thu Aug 17 18:40:59 2017

Mac Sandbox: Fix font servers in macOS 10.11.

A missing font server in the sandbox profile for macOS 10.11 causes
renderers to crash when visiting certain websites. This adds the
com.apple.FontObjectsServer to the profile.

Bug:  756145 
Change-Id: I02a34f31b23b385f0c69c544a3ec2116fa5d1d6b
Reviewed-on: https://chromium-review.googlesource.com/619387
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Greg Kerr <kerrnel@chromium.org>
Cr-Commit-Position: refs/heads/master@{#495246}
[modify] https://crrev.com/7e90a57df0f9434204a2bbf37366ca21aa8a9978/content/renderer/renderer_v2.sb

Perf try job appears to have succeeded:

https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/1702

 Issue 756623  has been merged into this issue.

 Issue 756027  has been merged into this issue.

 Issue 758199  has been merged into this issue.

 Issue 758188  has been merged into this issue.

 Issue 758215  has been merged into this issue.

 Issue 758233  has been merged into this issue.

 Issue 758576  has been merged into this issue.

 Issue 757993  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4fa660ddcdc1744e5f4380c336df5c8e2226642c

commit 4fa660ddcdc1744e5f4380c336df5c8e2226642c
Author: Greg Kerr <kerrnel@chromium.org>
Date: Thu Aug 24 20:48:27 2017

Revert "Enable Finch Trial Test for MacV2Sandbox."

This reverts commit 9705b7aaabf4c81d4e9cb13174338df9bde61cc4.

Reason for revert: This CL seems to be causing performance regressions.
 crbug.com/757993 

Original change's description:
> Enable Finch Trial Test for MacV2Sandbox.
>
> Cq-depend: CL:611386
> Bug: 749839
> Change-Id: I03057065380f5e49b899f1c9f9869e175d3223dc
> Reviewed-on: https://chromium-review.googlesource.com/612323
> Reviewed-by: Robert Sesek <rsesek@chromium.org>
> Reviewed-by: Steven Holte <holte@chromium.org>
> Commit-Queue: Greg Kerr <kerrnel@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#494502}

TBR=rsesek@chromium.org,holte@chromium.org

Bug:  756145 , 756623 
Change-Id: Iccada6b649f20f07ea4af4fdc995dc1a8426ded4
Reviewed-on: https://chromium-review.googlesource.com/633623
Reviewed-by: Greg Kerr <kerrnel@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Greg Kerr <kerrnel@chromium.org>
Cr-Commit-Position: refs/heads/master@{#497180}
[modify] https://crrev.com/4fa660ddcdc1744e5f4380c336df5c8e2226642c/testing/variations/fieldtrial_testing_config.json

 Issue 759028  has been merged into this issue.