All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=814155

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=bd3192b585180c9fd46e363690d6cd60c9a69f230243b6a4d1b03b5c046837ec


Bot(s) for this bug's original alert(s):

chromium-rel-mac12

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12d88987840000

📍 Found significant differences after each of 2 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/12d88987840000

Updating XTBs based on .GRDs from branch master by govind@chromium.org
https://chromium.googlesource.com/chromium/src/+/056d08eee04db3f48c53adb416c7044993ebceab

CQ: stop processing CLs. by tandrii@chromium.org
https://chromium.googlesource.com/chromium/src/+/6c234973451d66de1b7c46aca115671a883f521e

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

the file in touched is cq.cfg, it affects only commit queue.

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14c98a48440000

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/14c98a48440000

CQ: stop processing CLs. by tandrii@chromium.org
https://chromium.googlesource.com/chromium/src/+/6c234973451d66de1b7c46aca115671a883f521e

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

argh, apparently bisect is very insisting. Can it be that something in chromium compiles based on cq.cfg contents?!

Assigning to hablich, owner of the jetstream benchmark. hablich, it looks like the benchmark is too sensitive on Mac 10.12--the bisect run on the build with tandrii's CQ change consistently gives a different value than the previous build, even though it shouldn't affect compile at all. This means that bisect can't work correctly. Any ideas what to do about the noise level of the test?

Interesting. What do you mean the benchmark is to sensitive? The score changes by ~ 10 %, that is significant. Maybe the CQ config change removes a work that would run while the bot executes the benchmark? https://chromiumdash.appspot.com/commit/6c234973451d66de1b7c46aca115671a883f521e paints a quite interesting picture. Lots of improvements in the CL range.

Are the bots using cq.cfg?

hablich: None of the pinpoint bots are using cq.cfg, and the file isn't compiled into chromium, so there should be no way that this change can affect chrome's performance. We prebuild binaries at each revision, so it looks like compile churn is causing a significant difference between the two binaries on this test. This is why I think the benchmark is too sensitive, it's producing a false positive between these two versions which shouldn't have any different code. 

Surprising indeed.

mvstanton, any clue what's going on with Jetstream?

A few weeks before we checked in side-channel mitigations that affected JetStream (crbug.com/812205), that could be showing up here too.