All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=635512

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglpa2tgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglpu2pgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglt3UogoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglrmfuwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglvHlugoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgltSkoQkM


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus6
android-nexus7v2
android-nexus9

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004891883209382256

dcastagna@ the CL "cc: Add a flag to draw green borders on GL composited quads." is in this range, could it have caused this regression?

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3945
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004885218380562112

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3945
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

rmcilroy@: I doubt that's the cause. crrev.com/2198573002 shouldn't affect performance at all unless that flag is passed to chrome.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004815794283287664

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev    N   Good?
chromium@409892  16.7983  0.032562   18  good
chromium@409910  16.795   0.0266608  18  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 635512

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.simple_mobile_sites
Test Metric: frame_times/frame_times
Relative Change: 0.07%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2001
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004815794283287664


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5291413579759616

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004793807804282416

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2448
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004523558275041696

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004523548279349328

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3985
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3202
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004520112136764016

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004519938133524592

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3985
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3202
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004427584589459040

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004427430642941392

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2464
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2465
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004092956200088928

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3998
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004089032312137488

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3998
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004076700796085152

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2448
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9003996239983266800

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2488
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9003790607398702400

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2464
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Assigned to myself for investigation of repeated failed bisects.

Ping Roberto, any update on your investigation?

Perf sheriff ping

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998335140966361472

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3419
Failure reason: the build has failed.
Additional errors:
The metric was not found in the test output.
Either of the initial "good" or "bad" revisions failed to be tested or built.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995785378225766160

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Step('Compare samples') failed with return_code 1
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean  Std Dev  N  Good?
chromium@409854  N/A   N/A      0  good
chromium@409910  N/A   N/A      0  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 635512

Test Command: src/build/android/test_runner.py gtest --release -s cc_perftests --verbose
Test Metric: StructTraits deserialization/num runs in 2 seconds: DelegatedFrame_ManyQuads_100000_100000
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2762
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995785378225766160


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6441800145829888

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982576122300454400

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8977337033689490208

weiliangc@, it appears that the majority of these "regressions" are due to an increase in:

StructTraits deserialization / num runs in 2 seconds: *

An increase here feels like a good thing (more runs = faster runs = better?). Is it possible that the monitoring config for this perftest is just set up wrong and we need to say "higher is better"?

=== BISECT JOB RESULTS ===
Perf regression found but unable to continue

Bisect was stopped because a commit couldn't be classified as either
good or bad.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : smoothness.simple_mobile_sites
  Metric       : frame_times/frame_times


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests smoothness.simple_mobile_sites

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977337033689490208

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5320429440335872


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8977332757341040992

=== BISECT JOB RESULTS ===
Perf regression found but unable to continue

Bisect was stopped because a commit couldn't be classified as either
good or bad.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : smoothness.simple_mobile_sites
  Metric       : frame_times/frame_times


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests smoothness.simple_mobile_sites

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977332757341040992

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5320429440335872


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

WontFix-ing because these metrics dont' exist anymore.