I'm disabling the benchmark per perf bot sheriffing instructions (https://chromium.googlesource.com/chromium/src/+/master/tools/perf/docs/perf_bot_sheriffing.md): https://codereview.chromium.org/1910483005/

The disabling patch failed to get through the CQ (for unrelated reasons) and the frequency of the flakes reduced a lot, so I'm NOT going to disable the benchmark now, but I'll re-assign this bug to the benchmark owner instead. Please land the disabling patch (https://codereview.chromium.org/1910483005/) if the frequency flakes increases again.

We're still getting flakes on Mac 10.10 Perf (3). Here all the native crashes happen on ../../../chrome/test/data/perf/canvas_bench/many_images.html:

Standard output:
********************************************************************************
	2016-06-18 00:30:11.239 Google Chrome[12950:55306] NSWindow warning: adding an unknown subview: <FullSizeContentView: 0x7f99eaddab80>
	2016-06-18 00:30:11.239 Google Chrome[12950:55306] Call stack:
	(
	    "+callStackSymbols disabled for performance reasons"
	)
	[12960:30003:0618/003033:ERROR:ffmpeg_demuxer.cc(1495)] OnReadFrameDone result=-541478725 IsMaxMemoryUsageReached=0
	[12985:26143:0618/003302:ERROR:ffmpeg_demuxer.cc(1495)] OnReadFrameDone result=-541478725 IsMaxMemoryUsageReached=0
	[12985:26143:0618/003307:ERROR:ffmpeg_demuxer.cc(1495)] OnReadFrameDone result=-541478725 IsMaxMemoryUsageReached=0
********************************************************************************

Latest failing build: https://build.chromium.org/p/chromium.perf/builders/Mac%2010.10%20Perf%20%283%29/builds/3150

There were only 3 flakes in the past 20 builds, so, for the time being, I'm NOT going to disable the benchmark per perf bot sheriffing instructions (https://chromium.googlesource.com/chromium/src/+/master/tools/perf/docs/perf_bot_sheriffing.md).

junov@ (benchmark owner): Please decide what should be done next.

I am going to try downsizing the test.  It uses an extremely large working set of images, which may be creating an unreasonable amount of memory pressure. After all, this is a benchmark, not a robustness test.

Patch landed. It's a speculative fix, so let's let it bake for a day or two before declaring victory.

junov: Since this also affects the reference builds, I suspect we disable smoothness.tough_canvas_cases on mac-reference and re-enable it in a couple of weeks when the fix rolls into stable (assuming the issue is now fixed). Sounds good?

I don't understand the logic of that. What is the problem with leaving the test enabled?

The point is that we don't want benchmark failures to cause redness on the perf waterfall (http://build.chromium.org/p/chromium.perf/console). Since this issue won't be fixed in reference builds until your fix propagates all the way to Chrome Stable, the benchmark will be flaky on the waterfall (unless we disable it).

Looks like this test is still being very flaky on the reference build. However we're just about to update the reference builds to 52.0.2743.116 (https://codereview.chromium.org/2226243004/). Does that contain the fix? If yes, then we can just sit tight -- otherwise I think we should disable the test.

Ping!

I believe aiolos@ rolled the reference build last week.

It has been over a month since the last update. Seems fixed?

Marking as fixed. Re-open if needed.