Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996980702579846256

===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision         Exit Code  Std Dev  N  Good?
chromium@428539  0          N/A      2  good
chromium@428575  0          N/A      2  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 662170

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='crowd\.ogg' media.tough_video_cases
Test Metric: buffering_time/crowd.ogg
Relative Change: 0.00%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/870
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996980702579846256


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5899969133805568

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Ping

First seen: https://build.chromium.org/p/chromium.perf/builders/Mac%20HDD%20Perf%20%285%29/builds/3048

range: 428543 - 428574

Rerunning bisect...

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996245817771230896

=== Auto-CCing suspected CL author zmo@chromium.org ===

Hi zmo@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Lower ES3 capable requirement to GL 3.3 with extensions.
Author  : zmo
Commit description:
  
Still, we don't want to support ES3 on Mac if it's only 3.3.
We use blacklist to disable it.

This CL also implemented blacklist feature WebGL2.

BUG= 295792 
TEST=webgl2_conformance
R=kbr@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2460943002
Cr-Commit-Position: refs/heads/master@{#428564}
Commit  : 05f50f9bb7f2756bc7f291df332c997476a03043
Date    : Sat Oct 29 00:46:02 2016


===== TESTED REVISIONS =====
Revision         Exit Code  Std Dev  N  Good?
chromium@428543  N/A        N/A      5  good
chromium@428559  N/A        N/A      5  good
chromium@428563  N/A        N/A      5  good
chromium@428564  N/A        N/A      5  bad    <--
chromium@428565  N/A        N/A      5  bad
chromium@428567  N/A        N/A      5  bad
chromium@428574  N/A        N/A      5  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 662170

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=crowd1080.webm media.tough_video_cases
Test Metric: buffering_time/crowd1080.webm
Relative Change: 0.00%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/879
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996245817771230896


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5804132609294336

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996050706175211808

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Lower ES3 capable requirement to GL 3.3 with extensions.
Author  : zmo
Commit description:
  
Still, we don't want to support ES3 on Mac if it's only 3.3.
We use blacklist to disable it.

This CL also implemented blacklist feature WebGL2.

BUG= 295792 
TEST=webgl2_conformance
R=kbr@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2460943002
Cr-Commit-Position: refs/heads/master@{#428564}
Commit  : 05f50f9bb7f2756bc7f291df332c997476a03043
Date    : Sat Oct 29 00:46:02 2016


===== TESTED REVISIONS =====
Revision         Exit Code  Std Dev  N  Good?
chromium@428563  0          N/A      5  good
chromium@428564  1          N/A      5  bad    <--
chromium@428565  1          N/A      5  bad
chromium@428567  1          N/A      5  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 662170

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=crowd1080.webm media.tough_video_cases
Test Metric: buffering_time/crowd1080.webm
Relative Change: 0.00%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/880
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996050706175211808


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5836047169093632

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

I don't think my CL caused the perf regression.  It simply disable WebGL2 on that bot, but the underlying driver is still core profile 3.3, no behavior change there. Enabling/Disabling WebGL2 should have no effects on that video perf tests.

Un-assign myself and adding a few Mac gurus.

This is a test failure, not a performance regression. It pretty consistently fails at this CL.

I'm sorry but due to crucial feature work for M56 our team isn't going to be able to investigate this before this Thursday. This test runs fine on my NVIDIA MacBook Pro and given that it's an audio-only video file it's really hard to understand how this graphics-related change could have affected this test (esp. only this test, if other tests are still running OK on this hardware). If someone from the speed infra team can help us diagnose the failure mode on the bot we'll be happy to help.

Normally when a change breaks tests the default action should be to revert the change and then investigate later. Is the change that caused this failure crucial to Chrome?

Yes, it is, and it can not be reverted.

Can we disable the telemetry test on Mac until someone is able to investigate?

OK, let me take this back now I understand it's not a perf regression.

The root cause is we blacklist WebGL2 on GL version (< 4.1). We don't know that on browser startup because we haven't created a GL context to query the GL string (and since bots always do cold run with no user data cache, we can't query that string from caches either).  So on the browser side, we don't blacklist WebGL2.  However, once the GPU process launches, it collects the GL string and decides to blacklist WebGL2.  Based on this discrepancy, we take the safer path by disabling all GPU access, thus this test fails.

I'll get a fix up quickly.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c

commit 5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c
Author: zmo <zmo@chromium.org>
Date: Tue Nov 15 03:36:24 2016

Don't forbid GPU process if WebGL/WebGL2 is disabled on GPU but not browser.

BUG= 662170 
TEST=media.tough_video_cases,content_unittests
R=kbr@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2505473002
Cr-Commit-Position: refs/heads/master@{#432073}

[modify] https://crrev.com/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c/content/browser/gpu/gpu_data_manager_impl_private.cc
[modify] https://crrev.com/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c/content/browser/gpu/gpu_data_manager_impl_private_unittest.cc
[modify] https://crrev.com/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c/gpu/config/gpu_util.cc
[modify] https://crrev.com/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c/gpu/config/gpu_util.h
[modify] https://crrev.com/5881e9b3bc7c96dc8a0d5ec823078e3b4d80a85c/gpu/config/gpu_util_unittest.cc

The bot turned green already

Thanks!