[mac triage and, well I'm on the cc as well]

These are pixel tests. This could also be the skia roll. I don't think it's Chris's or my change.

Here are the baselines.

Deleted: small_cross_origin_expected.png 3.5 KB

	small_cross_origin_expected.png 3.5 KB View Download

Deleted: poster_tests_expected.png 24.5 KB

	poster_tests_expected.png 24.5 KB View Download

Here's a rebaseline done locally, but it doesn't actually fail on my machine (OSX 10.11, hidpi, r384591). The test has a tolerance of manhattan distance 20, I had to drop it to <5 to produce these.

SmallCrossOrigin and PosterTests fail at 23.

So rebaselining might shift the distance +/-5 to get within a distance of 20, or the tolerance may need to increase to ~25.

I'm still a pixel test n00b, so leaving it up to tommycli

Deleted: small_cross_origin_expected.png 8.5 KB

	small_cross_origin_expected.png 8.5 KB View Download

Deleted: poster_tests_expected.png 24.1 KB

	poster_tests_expected.png 24.1 KB View Download

Heh: I wonder what change (probably in skia) caused the pixels to change. I definitely can't see it with my eyes.

I will rebaseline locally (on Linux) and see if that fixes it. If not, I'll up the manhattan distance to 25.

It's a bit frustrating that the bots render it differently than locally so we have to guess.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c

commit 8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c
Author: tommycli <tommycli@chromium.org>
Date: Tue Apr 05 15:41:08 2016

Plugin Power Saver: Rebaseline pixel tests.

Some pixel tests broke recently, probably due to a Skia change.

This patch resets the reference image.

BUG=599484

Review URL: https://codereview.chromium.org/1856043002

Cr-Commit-Position: refs/heads/master@{#385176}

[modify] https://crrev.com/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c/chrome/browser/plugins/plugin_power_saver_browsertest.cc
[modify] https://crrev.com/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c/chrome/test/data/plugin_power_saver/large_cross_origin_obscured_expected.png
[modify] https://crrev.com/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c/chrome/test/data/plugin_power_saver/poster_tests_expected.png
[modify] https://crrev.com/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c/chrome/test/data/plugin_power_saver/small_cross_origin_expected.png
[modify] https://crrev.com/8492bf884d5b19a1384e1fd338c0bd5c68a9ab2c/chrome/test/data/plugin_power_saver/smaller_than_play_icon_expected.png

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3188721c5cb06cc130d1bd4206a11e9522bf969f

commit 3188721c5cb06cc130d1bd4206a11e9522bf969f
Author: tommycli <tommycli@chromium.org>
Date: Tue Apr 05 22:18:41 2016

Plugin Power Saver: Update pixel test tolerance and error behavior.

1. This increases the manhattan distance tolerance to 25 in BGR space.
2. This now reports all non-matching pixels instead of just the first.

BUG=599484

Review URL: https://codereview.chromium.org/1863483004

Cr-Commit-Position: refs/heads/master@{#385302}

[modify] https://crrev.com/3188721c5cb06cc130d1bd4206a11e9522bf969f/chrome/browser/plugins/plugin_power_saver_browsertest.cc

Still fails: https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac%20non-Oilpan%20%28dbg%29/builds/441

Looks like the threshold to get past this bot might need to be 30 instead of 25.

Is 30 abnormal? Is there a procedure for getting a dump of the pixel test from bot? (cf. I'm still a pixel test n00b)

Arg...

Okay. Based on the reported failed pixels (all in a line) and failure magnitude (not that large), I'm guessing that the image generated on this bot is offset by a half pixel (and thus anti-aliased differently).

To get the "actual" image, you can run the test locally with --rebaseline-pixel-tests and do "git diff" to see if it changed. Likewise I've logged onto the bots themselves, run the test locally with --rebaseline-pixel-tests and SFTPed the images off.

I have two questions:
 1) Is there a way for tests to dump extra binary data? Perhaps upload the "actual" image to a server somewhere?
 2) What is different about this Bot that causes failure while the other Mac bots continue to succeed?

Tommy

(I ask #2 because it would be a lot easier to diagnose if we could repro locally)

I want to disable flaky tests here: https://codereview.chromium.org/1868863003/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bce1b56007d9e70b8b926781b2a3deb340046b83

commit bce1b56007d9e70b8b926781b2a3deb340046b83
Author: loyso <loyso@chromium.org>
Date: Thu Apr 07 04:48:52 2016

Disable flaky browser tests on Mac.

BUG=599484
TBR=tommycli@chromium.org

Review URL: https://codereview.chromium.org/1868863003

Cr-Commit-Position: refs/heads/master@{#385664}

[modify] https://crrev.com/bce1b56007d9e70b8b926781b2a3deb340046b83/chrome/browser/plugins/plugin_power_saver_browsertest.cc

Linux tests flakiness observed in the following builds:
https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/45541
https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/45542
https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/45544

@tommycli is already aware of it, I'm leaving logs for the record.

Logs for reference:
[ RUN      ] PluginPowerSaverBrowserTest.PosterTests
Xlib:  extension "RANDR" missing on display ":9".
Xlib:  extension "RANDR" missing on display ":9".
[21291:21291:0829/102739:WARNING:audio_manager.cc(317)] Multiple instances of AudioManager detected
[21291:21291:0829/102739:WARNING:audio_manager.cc(278)] Multiple instances of AudioManager detected
[21291:21291:0829/102739:WARNING:password_store_factory.cc(248)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options.
BrowserTestBase received signal: Terminated. Backtrace:
#0 0x0000024dea2e base::debug::StackTrace::StackTrace()
#1 0x0000029ed499 content::(anonymous namespace)::DumpStackTraceSignalHandler()
#2 0x7faa6d24f0b0 <unknown>
#3 0x7faa6d300993 __poll
#4 0x7faa72776ff6 <unknown>
#5 0x7faa72777124 g_main_context_iteration
#6 0x0000024f6526 base::MessagePumpGlib::Run()
#7 0x00000250ceae base::RunLoop::Run()
#8 0x000002a129fc content::RunThisRunLoop()
#9 0x000002a1299a content::RunMessageLoop()
#10 0x000000c557a3 PluginPowerSaverBrowserTest::VerifySnapshot()
#11 0x000000c5616c PluginPowerSaverBrowserTest_PosterTests_Test::RunTestOnMainThread()
#12 0x00000256e0ff InProcessBrowserTest::RunTestOnMainThreadLoop()
#13 0x0000029ed209 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop()
#14 0x0000028796a0 ChromeBrowserMainParts::PreMainMessageLoopRunImpl()
#15 0x0000028786b5 ChromeBrowserMainParts::PreMainMessageLoopRun()
#16 0x000001d37665 content::BrowserMainLoop::PreMainMessageLoopRun()
#17 0x000001fd0327 content::StartupTaskRunner::RunAllTasksNow()
#18 0x000001d35b84 content::BrowserMainLoop::CreateStartupTasks()
#19 0x000001d3a5fd content::BrowserMainRunnerImpl::Initialize()
#20 0x000001d32f38 content::BrowserMain()
#21 0x0000024c5837 content::ContentMainRunnerImpl::Run()
#22 0x0000024c4320 content::ContentMain()
#23 0x0000029ecbe1 content::BrowserTestBase::SetUp()
#24 0x00000256c9e4 InProcessBrowserTest::SetUp()
#25 0x000002ed74b8 testing::Test::Run()
#26 0x000002ed8223 testing::TestInfo::Run()
#27 0x000002ed86b7 testing::TestCase::Run()
#28 0x000002edf6e7 testing::internal::UnitTestImpl::RunAllTests()
#29 0x000002edf33a testing::UnitTest::Run()
#30 0x000002578273 base::TestSuite::Run()
#31 0x0000024d3819 ChromeTestSuiteRunner::RunTestSuite()
#32 0x000002a0f8ef content::LaunchTests()
#33 0x0000024d1da1 main
#34 0x7faa6d23a7ed __libc_start_main
#35 0x0000005c7bad <unknown>
[1047/1047] PluginPowerSaverBrowserTest.PosterTests (TIMED OUT)

I'm looking into it. PluginPowerSaverBrowserTest::VerifySnapshot() can't tolerate any LayerTreeHost::SetNeedsCommit() call made within the first  content::RunMessageLoop() - it just hangs. Stay tuned for any updates.

InsertVisualStateCallback implies the response with MESSAGE_DELIVERY_POLICY_WITH_VISUAL_STATE:

  // If a commit was requested before the message was enqueued the message
  // will be delivered with the swap corresponding to that commit. Otherwise
  // the message will be sent immediately using regular IPC.
  // If the commit is aborted or optimized out the message is sent using regular
  // IPC.
  MESSAGE_DELIVERY_POLICY_WITH_VISUAL_STATE,

Investigating, why we don't receive it via "regular IPC".

VisualStateTest.CallbackDoesNotDeadlock is disabled and broken in ToT.

visual_state_browsertest.cc:123: Failure
Value of: GetCallbackCount()
  Actual: 0
Expected: 1

Here is the fix for c#12 reported by lazyboy@:
https://codereview.chromium.org/2313593002/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/260a8f74a7f1230c3252f2640ce7185981d8aef4

commit 260a8f74a7f1230c3252f2640ce7185981d8aef4
Author: loyso <loyso@chromium.org>
Date: Thu Sep 08 02:07:17 2016

Content Renderer FrameSwapMessageQueue: Fix DidActivate with DidNotSwap.

LayerTreeHostImpl::SwapBuffers may fail if frame.has_no_damage (SwapPromise::SWAP_FAILS).

IF FrameSwapMessageQueue::DidActivate happens during that frame
THEN
the outputSurface will NOT take care of the Drain+Send.
AND
QueueMessageSwapPromise::DidNotSwap will not send the
MESSAGE_DELIVERY_POLICY_WITH_VISUAL_STATE message back.

As a result, the callback inserted with RenderFrameHostImpl::InsertVisualStateCallback never triggers.
PluginPowerSaverBrowserTest::VerifySnapshot hangs forever.

BUG=599484

Review-Url: https://codereview.chromium.org/2313593002
Cr-Commit-Position: refs/heads/master@{#417162}

[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/frame_swap_message_queue.cc
[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/frame_swap_message_queue_unittest.cc
[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/queue_message_swap_promise_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/260a8f74a7f1230c3252f2640ce7185981d8aef4

commit 260a8f74a7f1230c3252f2640ce7185981d8aef4
Author: loyso <loyso@chromium.org>
Date: Thu Sep 08 02:07:17 2016

Content Renderer FrameSwapMessageQueue: Fix DidActivate with DidNotSwap.

LayerTreeHostImpl::SwapBuffers may fail if frame.has_no_damage (SwapPromise::SWAP_FAILS).

IF FrameSwapMessageQueue::DidActivate happens during that frame
THEN
the outputSurface will NOT take care of the Drain+Send.
AND
QueueMessageSwapPromise::DidNotSwap will not send the
MESSAGE_DELIVERY_POLICY_WITH_VISUAL_STATE message back.

As a result, the callback inserted with RenderFrameHostImpl::InsertVisualStateCallback never triggers.
PluginPowerSaverBrowserTest::VerifySnapshot hangs forever.

BUG=599484

Review-Url: https://codereview.chromium.org/2313593002
Cr-Commit-Position: refs/heads/master@{#417162}

[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/frame_swap_message_queue.cc
[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/frame_swap_message_queue_unittest.cc
[modify] https://crrev.com/260a8f74a7f1230c3252f2640ce7185981d8aef4/content/renderer/gpu/queue_message_swap_promise_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0983d3b2372bd46ec38d4ebc0b4fa06f286ff4d9

commit 0983d3b2372bd46ec38d4ebc0b4fa06f286ff4d9
Author: tommycli <tommycli@chromium.org>
Date: Tue Sep 27 22:08:07 2016

Revert of Disable flaky browser tests on Mac. (patchset #1 id:1 of https://codereview.chromium.org/1868863003/ )

Reason for revert:
Flakiness should have been fixed. See bug.

Original issue's description:
> Disable flaky browser tests on Mac.
>
> BUG=599484
> TBR=tommycli@chromium.org
>
> Committed: https://crrev.com/bce1b56007d9e70b8b926781b2a3deb340046b83
> Cr-Commit-Position: refs/heads/master@{#385664}

TBR=loyso@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=599484

Review-Url: https://codereview.chromium.org/2377743003
Cr-Commit-Position: refs/heads/master@{#421355}

[modify] https://crrev.com/0983d3b2372bd46ec38d4ebc0b4fa06f286ff4d9/chrome/browser/plugins/plugin_power_saver_browsertest.cc

These tests are deterministically failing on the 10.12 bot, but pass locally.

https://build.chromium.org/p/chromium.fyi/builders/Chromium%20Mac%2010.11%20Force%20Mac%20Toolchain/builds/14869

This is blocking our 10.12 rollout.

VisualStateTest.CallbackDoesNotDeadlock is still disabled and broken.

I can't repro the failures on my local 10.12.2 machine, but they do repro when I ssh into the vm. I've attached the failure logs. One of them appears to be related to pixel tests, the other does not.

https://build.chromium.org/p/chromium.fyi/builders/Chromium%20Mac%2010.11%20Force%20Mac%20Toolchain/builds/14951

Deleted: poster_test_failures.txt 19.9 KB

poster_test_failures.txt 19.9 KB View Download

Deleted: small_cross_origin_failures.txt 235 KB

small_cross_origin_failures.txt 235 KB View Download

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fde5cb085de250b5ac8b96ff8f95e1d54038ae2d

commit fde5cb085de250b5ac8b96ff8f95e1d54038ae2d
Author: erikchen <erikchen@chromium.org>
Date: Fri Dec 16 22:29:30 2016

Disable two plugin power saver tests on macOS.

The tests fail on macOS 10.12 and need to be investigated by the PPS team.

BUG=599484,  665691 

Review-Url: https://codereview.chromium.org/2585433002
Cr-Commit-Position: refs/heads/master@{#439218}

[modify] https://crrev.com/fde5cb085de250b5ac8b96ff8f95e1d54038ae2d/chrome/browser/plugins/plugin_power_saver_browsertest.cc