The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d5dd606f9d167708bc3de0c9fcd6a7fd0390eb3a

commit d5dd606f9d167708bc3de0c9fcd6a7fd0390eb3a
Author: Ashley Enstad <ashleymarie@chromium.org>
Date: Thu Jan 18 17:55:01 2018

Disabling performance_browser_tests

performance_browser_tests are failing on Mac Air 10.11 Perf, Mac Pro
10.11 Perf, Mac Retina Perf, Win 7 ATI GPU Perf, Win 7 Intel GPU Perf,
Win 7 x64 Perf, and Win 8 Perf

Bug:  803455 
Change-Id: I92856fed9176452ab07191d13db1d7f1d692bb24
Reviewed-on: https://chromium-review.googlesource.com/873995
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ashley Enstad <ashleymarie@chromium.org>
Cr-Commit-Position: refs/heads/master@{#530194}
[modify] https://crrev.com/d5dd606f9d167708bc3de0c9fcd6a7fd0390eb3a/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/d5dd606f9d167708bc3de0c9fcd6a7fd0390eb3a/tools/perf/core/perf_data_generator.py

Out of curiosity, why would we disable a test that is telling us there's a problem? Why are we not reverting the change that broke the test instead?

If I understand the process correctly, we're trying to keep the waterfall green (all tests passing) since one failed test can hide other failures that come up while we figure out what broke. Ideally, we'd disable the failing test, investigate the failure to find out why it broke, revert the change that broke the test, and then re-enable the test.

If you have a hunch as to what change could have broken the test, don't hesitate to revert it :) I just haven't had a chance to investigate yet.

adding charliea@ to cc since he's working on a playbook for sheriffs that might cover questions like this or might have a better workflow in mind?

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/16bff9f0840000

(I support what Ashley said.)

Okay. I'm just really nervous because my team is making big changes to what is being tested, and we don't have this coverage while those changes land. :)

I think I should set up the perf_browser_tests to run in the CQ and/or on the main build waterfall. That would prevent breakage later on in the perf waterfall. I'll look into that...

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/16bff9f0840000

Support MockCertVerifier when network service is enabled
By jam@chromium.org · Tue Jan 09 03:27:34 2018
chromium @ 96721289b5bdc87d6c74880d474ea8089f09b504

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

jam@ is this CL revertable while you investigate why it's causing failures in the tests? At the moment, we have no performance browser test coverage because of the failures.

To #6: yes, please create a bug to add perf_browser_tests to run in the CQ and on the main build waterfall (it has to be an "and", so that if the test flaky, the main waterfall sheriff can disable it & avoid blocking CQ) :-)

Created bug 804191 to track enabling performance_browser_tests on the CQ and build waterfall.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/75db4df5e60a9a4f28c22ea0c58d243ffe1fc774

commit 75db4df5e60a9a4f28c22ea0c58d243ffe1fc774
Author: Yuri Wiitala <miu@chromium.org>
Date: Mon Jan 22 02:05:43 2018

Fix crash-on-start caused by recent Network Service browser tests change.

The crash was caused by https://chromium-review.googlesource.com/853301
because it added a call into base::CommandLine::ForCurrentProcess().
This change adds the missing instantiation of base::CommandLine in the
main() function for performance_browser_tests.

TBR=jam@chromium.org

Bug:  803455 ,  769401 
Change-Id: I8450b01559359d12f5258922d4b570e2d8b43647
Reviewed-on: https://chromium-review.googlesource.com/877823
Reviewed-by: Yuri Wiitala <miu@chromium.org>
Commit-Queue: Yuri Wiitala <miu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#530798}
[modify] https://crrev.com/75db4df5e60a9a4f28c22ea0c58d243ffe1fc774/chrome/test/base/browser_perf_tests_main.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1d5f7686b8165d381cb11a352bcbcd9389cd7837

commit 1d5f7686b8165d381cb11a352bcbcd9389cd7837
Author: Yuri Wiitala <miu@chromium.org>
Date: Mon Jan 22 12:29:34 2018

Revert "Disabling performance_browser_tests"

This reverts commit d5dd606f9d167708bc3de0c9fcd6a7fd0390eb3a.

Reason for revert: Landed fix. See: https://chromium-review.googlesource.com/c/chromium/src/+/877823

Original change's description:
> Disabling performance_browser_tests
> 
> performance_browser_tests are failing on Mac Air 10.11 Perf, Mac Pro
> 10.11 Perf, Mac Retina Perf, Win 7 ATI GPU Perf, Win 7 Intel GPU Perf,
> Win 7 x64 Perf, and Win 8 Perf
> 
> Bug:  803455 
> Change-Id: I92856fed9176452ab07191d13db1d7f1d692bb24
> Reviewed-on: https://chromium-review.googlesource.com/873995
> Reviewed-by: Ned Nguyen <nednguyen@google.com>
> Commit-Queue: Ashley Enstad <ashleymarie@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#530194}

TBR=miu@chromium.org,nednguyen@google.com,ashleymarie@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  803455 
Change-Id: Iae14b0e6c51adc98ff5b4e922e24979939b8edd5
Reviewed-on: https://chromium-review.googlesource.com/877229
Reviewed-by: Yuri Wiitala <miu@chromium.org>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#530844}
[modify] https://crrev.com/1d5f7686b8165d381cb11a352bcbcd9389cd7837/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/1d5f7686b8165d381cb11a352bcbcd9389cd7837/tools/perf/core/perf_data_generator.py

I stop-checked a couple chromium.perf bots and the binary is running just fine now.