All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=706866

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnPi46QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnLa06QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnPznoAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7NO5qAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnJ6E8ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnKq29QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7PfXqwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnID5uAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnPWOpgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnPi46QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnKrdgwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7LfNqQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7PfXqwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnOqMmgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnPWOpgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnKfOtgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7KPx7ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnNzDqQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnIKsqAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnN6vtQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnJrtlQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7Oyk2woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnNzD6QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnN7zpAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7MP09QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnKrdgwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnOiyigoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnICKyAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg7JuFkQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgnNXktgsM


Bot(s) for this bug's original alert(s):

android-one

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8983678408096573232

=== Auto-CCing suspected CL author ericrk@chromium.org ===

Hi ericrk@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : ericrk
  Commit : 2f952e4c39887d65126ca4cd67a897fc6d409fb9
  Date   : Wed Mar 29 22:15:32 2017
  Subject: Enable CrOS/Android DefaultEnableGpuRasterization field trial for bots

Bisect Details
  Configuration: android_one_perf_bisect
  Benchmark    : memory.top_10_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:cc:effective_size_avg/foreground/http_www_amazon_com_gp_aw_s_k_nexus
  Change       : 93.97% | 5511564.0 -> 10690788.0

Revision             Result               N
chromium@460532      5511564 +- 0.0       6      good
chromium@460542      5511564 +- 0.0       6      good
chromium@460547      5511564 +- 0.0       6      good
chromium@460548      5511564 +- 0.0       6      good
chromium@460549      10690788 +- 0.0      6      bad       <--
chromium@460550      10690788 +- 0.0      6      bad
chromium@460552      10690788 +- 0.0      6      bad
chromium@460572      10690788 +- 0.0      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8983678408096573232

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5697138786304000


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

 Issue 706923  has been merged into this issue.

The merge of  bug 706923  is because of the following bisect which is a big improvement of thread_raster_cpu_time_per_frame and happened around the time that we enabled spinvalidation. Bisect result indicates that the change is unrelated to spinvalidation. I split the bugs.

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : ericrk
  Commit : 2f952e4c39887d65126ca4cd67a897fc6d409fb9
  Date   : Wed Mar 29 22:15:32 2017
  Subject: Enable CrOS/Android DefaultEnableGpuRasterization field trial for bots

Bisect Details
  Configuration: android_one_perf_bisect
  Benchmark    : thread_times.simple_mobile_sites
  Metric       : thread_raster_cpu_time_per_frame/https___www.flickr.com_
  Change       : 89.04% | 3.51372135524 -> 0.385201552571

Revision             Result                     N
chromium@460532      3.51372 +- 0.177036        6      good
chromium@460542      3.47482 +- 0.0765304       6      good
chromium@460547      3.48152 +- 0.0929753       6      good
chromium@460548      3.5449 +- 0.171366         6      good
chromium@460549      0.389965 +- 0.0436638      6      bad       <-- (good actually)
chromium@460550      0.381884 +- 0.0172552      6      bad
chromium@460552      0.374826 +- 0.0211545      6      bad
chromium@460572      0.385202 +- 0.025873       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https...www.flickr.com. thread_times.simple_mobile_sites

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8983660914168803216

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5582631736967168


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

 Issue 708575  has been merged into this issue.

 Issue 708560  has been merged into this issue.

 Issue 708576  has been merged into this issue.

 Issue 708656  has been merged into this issue.

 Issue 708661  has been merged into this issue.

 Issue 708662  has been merged into this issue.

Investigating.

Looking at these regressions test-by-test to try to determine if real or not.

For Android-one regressions on system_health.memory_mobile, I don't think this is a real regression (we regress some categories, improve others, for an overall improvement). See graphs here:
https://chromeperf.appspot.com/report?sid=08d817e9a44fb68b0e072480638c2a3743ad32714539e8372c836de67981f769&rev=460551

Note that despite malloc and CC regressions, overall:
peak resident and proportional resident are unchanged.
private dirty and resident size show an improvement
Locked size shows a large improvement (15MB).

memory.top_10_mobile on andoir-one shows similar overall pattern:
https://chromeperf.appspot.com/report?sid=467204bf01499c525488aa9b7c69e787a7c065fac089e584ac03f4bc285406b2&rev=460549
private dirty size improves, peak resident, proportional resident, resident are unchanged.

memory.top_10_mobile_stress shows similar patterns overall:
no change or slight improvement across the four OS memory metrics.

Finally, the regression in GPU time is completely expected, as we use the GPU proc more. Results across procs look ok:
https://chromeperf.appspot.com/report?sid=fcc6b07416c06c784fb1d9d96f9f277a4a2ff5aedc4d9bcd15466c9a59077483&start_rev=456255&end_rev=463155

Could we just be moving memory around so they're showing up on different metrics with an end state that's better than where we started?

Yeah, I may not have been clear in #14 - but I believe that is what we've done. Overall, I think this is a slight improvement - memory is expected to have moved around.