All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=749358

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=fae7a2e05c0b9daac43b74d0846a2cca28e0cd082e92ec9145e1561b0642d023


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina
chromium-rel-mac12

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972959338048176688

=== Auto-CCing suspected CL author treib@chromium.org ===

Hi treib@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Marc Treib
  Commit : 632bc455b5070bd3c09aeae61728fef3f83776da
  Date   : Tue Jul 18 09:25:10 2017
  Subject: Fieldtrial testing config for NTPCaptureThumbnail

Bisect Details
  Configuration: mac_10_12_perf_bisect
  Benchmark    : system_health.memory_desktop
  Metric       : memory:chrome:all_processes:reported_by_chrome:gpu:effective_size_avg/browse_search/browse_search_google
  Change       : 43.49% | 9644408.0 -> 13838776.0

Revision             Result               N
chromium@487419      9644408 +- 0.0       6      good
chromium@487423      9644408 +- 0.0       6      good
chromium@487424      13838776 +- 0.0      6      bad       <--
chromium@487425      13838776 +- 0.0      6      bad
chromium@487426      13838776 +- 0.0      6      bad
chromium@487432      13838776 +- 0.0      6      bad
chromium@487444      13838776 +- 0.0      6      bad
chromium@487468      13838776 +- 0.0      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.search.google system_health.memory_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972959338048176688


For feedback, file a bug with component Speed>Bisection

This is probably more or less the same as the previous issue 741856, there's some discussion there about probable causes.

Okay, I officially believe this is a false positive. Copying two theories from the other bug:
1) The memory increase is temporary (say, for a few hundred ms). The perf tests happen to measure just during that time, but averaged over all time, the extra memory use is negligible.
2) The extra buffers are a one-time cost - the very first CopyFromSurface call triggers their creation. In the perf tests, that happens to be my feature, but in real life, it's usually already happened before.

Adding a third:
3) Thumbnails are simply taken at different times - this particular test now happens to triggers a thumbnail capture, but that doesn't indicate an actual overall regression.

Evidence that it's not a real perf regression:
o In this latest iteration, the total number of thumbnails taken doesn't go up - if anything, it seems to go down slightly: https://uma.googleplex.com/p/chrome/variations/?sid=b6b621ec23fb5546426b44176fd7c74a  Thumbnails do get taken at different times, so apparently this test now triggers a thumbnail but didn't before.
o UMA shows no significant change in memory usage: https://uma.googleplex.com/p/chrome/variations/?sid=af3a442efc45f98bdca9bc1901ae9b1b  This was true even for a previous iteration, where the number of thumbnails taken was significantly larger.

Is this sufficient justification for WontFix'ing? If not, what's the path forward - how I can get sufficient evidence, or alternatively prove that the regression is real after all?

+erikchen on this bug, related to issue 741856

Assigning to test OWNER for decision. IMO, it's possible we may want to change the tests to measure memory at a different time. If not, it's also possible the measurements will become "random" (i.e., sometimes measuring during the snapshot, and sometime not). 

+primiano, +benhenry to help make a call.

Juan - is it possible to make the metrics collection more controlled or deterministic given this information? I'd hate to wontfix because of a hunch, because 3-4MiB is significant.

Ping! primiano/benhenry/perezju, how do we move forward here?

My vote is wontfix, but I want to make sure we're doing the right thing.

wontfix sgtm given the explanation in #6