All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=633200

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxoG4tAoM


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005528611150292896

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004248321953989392

=== Auto-CCing suspected CL author erikchen@chromium.org ===

Hi erikchen@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Allow Core Animation compositor to use some scaled filter effects.
Author  : erikchen
Commit description:
  
Filter scaling has no effect on filter effects that are not BLUR or DROP_SHADOW.

BUG= 581526 
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel

Review-Url: https://codereview.chromium.org/2188523006
Cr-Commit-Position: refs/heads/master@{#408286}
Commit  : 40f907eae01fd326837abcafff959df6594b7bf6
Date    : Thu Jul 28 00:10:38 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@408259  56432936  0.0      5  good
chromium@408273  60040693  8067189  5  good
chromium@408280  56432936  0.0      5  good
chromium@408284  60043970  8065361  5  good
chromium@408285  56432936  0.0      5  good
chromium@408286  86222344  0.0      5  bad    <--
chromium@408287  86228898  8973.89  5  bad
chromium@408315  86946517  1619300  5  bad

Bisect job ran on: mac_retina_perf_bisect
Bug ID: 633200

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop
Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:cc:effective_size_avg/load_tools_docs
Relative Change: 54.07%
Score: 99.8

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_retina_perf_bisect/builds/1570
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004248321953989392


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5884856039374848

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

The CL referenced was part of a code path that is no longer being used. There was a memory leak with that code path, but it is no longer relevant. Looking at the graphs, the mac one shows a drop back down shortly after the spike. The Windows graph hasn't recovered, but this logic is never run on Windows.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004076435515513536

OK, I'm doing another bisect.

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Allow Core Animation compositor to use some scaled filter effects.
Author  : erikchen
Commit description:
  
Filter scaling has no effect on filter effects that are not BLUR or DROP_SHADOW.

BUG= 581526 
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel

Review-Url: https://codereview.chromium.org/2188523006
Cr-Commit-Position: refs/heads/master@{#408286}
Commit  : 40f907eae01fd326837abcafff959df6594b7bf6
Date    : Thu Jul 28 00:10:38 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@408259  60040693  8067189  5  good
chromium@408273  60040693  8067189  5  good
chromium@408280  60043970  8065361  5  good
chromium@408284  60040693  8067189  5  good
chromium@408285  60043970  8065361  5  good
chromium@408286  86225621  7327.15  5  bad    <--
chromium@408287  86225621  7327.15  5  bad
chromium@408315  86222344  0.0      5  bad

Bisect job ran on: mac_retina_perf_bisect
Bug ID: 633200

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop
Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:cc:effective_size_avg/load_tools_docs
Relative Change: 43.61%
Score: 99.8

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_retina_perf_bisect/builds/1584
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004076435515513536


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5813911568252928

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Hmhh, the fact that bisect keeps pointing at your CL makes it hard to think that this happens coincidentally due to noise. 

nednguyen: Please see comment #5. A bisect needs to be run on Windows, not Mac.

I was going to file a new bug for the Windows regression, but I don't see it anymore on the telemetry graphs for this bug.

There is a major, unrecovered regression in the 407939 - 407983 range that needs investigation.

erik: my point stays the same. If you think that code isn't supposed to be triggered on Mac but the bisect bot reliably pointing at that CL as a cause of memory regression on Mac, it's possible that there is subtle bug that makes your claim incorrect. 

Do you think there is any chance that those code would be affecting Mac's memory number?

nednguyen: Maybe you should perform a bisect on Windows, rather than saying that there's a subtle bug that makes my claim incorrect?

As I mentioned in c#5, there was a memory leak in the original code path, so I could see this effecting Mac's memory number, if you look at the graphs, you see that the regression drops on Mac when this CL is reverted. The Windows regression has not changed though.

erikchen: What metric, story and bot was the regression in? Here are the aggregates (load_news, load_social, ...) for all windows platforms around the r407979-r407983 range: https://chromeperf.appspot.com/report?sid=48882f41300eeb42d3e91c17312bfd7155833f94c424745c5e338230e62c59e1&start_rev=407739&end_rev=408183

Are you suggesting that https://codereview.chromium.org/2188523006 was reverted? When (which revision)? I can't find the revert (a message is normally posted on the original patch's review).

The entire feature was turned off:
https://codereview.chromium.org/2193623004

Since then, it has been re-enabled and re-disabled twice. You can see these spikes in the Mac graph.

I looked into the regression on Mac. This is expected behavior, since my CLs enable the CA compositing path for Mac, which makes CALayers out of all the DrawQuads, which increases cc memory usage. 

I broke the Windows regression into a separate bug: https://bugs.chromium.org/p/chromium/issues/detail?id=638978