Seems to be failing 2/3 of the time.

First failure: https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac10.11%20%28dbg%29/builds/6029

Looks like a blinking cursor diff. Suspecting https://codereview.chromium.org/2515633002 (only 2 changes in the diff, this is the most likely).

+cc wangxianzhu,chrishtr in case the flake was caused by https://codereview.chromium.org/2517863002, the revert of "Don't early out recursion into PaintLayerChildren when computing composited bounds."

Reverted https://codereview.chromium.org/2521043002/ to see if it helps

Didn't help, unreverting.
->wangxianzhu who may know what's going on

This seems an unfixed issue of  bug 658287 . It seems that on debug bots, when the test finishes, the caret may have entered the invisible state. Do we have a way to force the caret not to blink?

Hmm, there is FrameSelection::setCaretBlinkingSuspended which is currently only used to suspend between mouse press/release. We could expose it via Internals.

There are other layout tests that use a caret and we haven't seen this kind of flakiness with them. I wonder whether it is the use of a reference test, the others I've seen are mainly repaint/pixel tests.

I guess this is related to the three runAfterLayoutAndPaint() calls which may take more than 0.5s on a debug build.

Would "input.blur(); input.focus();" before the test finishes work?

I'm building a Mac debug build, will try it. What happens at the 0.5s mark again, do we give up and take a pixel snapshot? In the end if we snapshot before all three runAfterLayoutAndPaint() calls have finished then we're not going to match the reference, I would think.

0.5s (or an more exact number) is the normal interval of caret blink timer. If we the test is slow, we may take a snapshot when the caret is off. I saw there is API to control the caret blink interval. Perhaps we can set it to a big number for layout tests.

Ah, it's not supposed to be blinking in layout tests already, see:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/layout/LayoutThemeDefault.cpp?q=caretBlinkInterval&sq=package:chromium&dr=CSs&l=255

but perhaps we've introduced a bug somehow and it is blinking again in some circumstances.

I see :)

Noticed that there is input.focus() in the reference, but no in the test. Perhaps autofocus is not reliable?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f7ef0041d8312a0d324237b722d1f4280731e69d

commit f7ef0041d8312a0d324237b722d1f4280731e69d
Author: piman <piman@chromium.org>
Date: Tue Nov 22 01:13:13 2016

Mark relayout-shifts-inner-editor flaky

BUG= 667403 
TBR=wangxianzhu

Review-Url: https://codereview.chromium.org/2519853003
Cr-Commit-Position: refs/heads/master@{#433726}

[modify] https://crrev.com/f7ef0041d8312a0d324237b722d1f4280731e69d/third_party/WebKit/LayoutTests/TestExpectations

I can repro failure locally by adding a 600ms delay before notifyDone. It's not due to use of a reference test -- if I remove -expected.html and generate pixel results with caret visible, and then apply 600ms delay and re-run as pixel test, I can repro same failure. It doesn't happen on Linux with above.

So, for some reason LayoutThemeMac.h extends LayoutTheme directly, whereas the other theme subclasses such as LayoutTheme{Linux,Win} extend LayoutThemeDefault. The caretBlinkInterval() override to set to zero is in LayoutThemeDefault, so only Mac doesn't get it.

Sending a small change to move the zero-for-tests to LayoutTheme::caretBlinkInterval().

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0210f10d9f5ac5b1f47f374a31dc8f70f33e5539

commit 0210f10d9f5ac5b1f47f374a31dc8f70f33e5539
Author: wkorman <wkorman@chromium.org>
Date: Tue Nov 22 03:51:57 2016

Move logic to turn off caret blinking for LayoutTests to LayoutTheme.

LayoutThemeMac extends LayoutTheme directly, whereas other themes
such as LayoutTheme{Win,Linux} extend LayoutThemeDefault. The logic
to turn off caret blinking in LayoutTests was placed in
LayoutThemeDefault::caretBlinkInterval() and so was ineffective on Mac.

BUG= 667403 

Review-Url: https://codereview.chromium.org/2525513002
Cr-Commit-Position: refs/heads/master@{#433780}

[modify] https://crrev.com/0210f10d9f5ac5b1f47f374a31dc8f70f33e5539/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/0210f10d9f5ac5b1f47f374a31dc8f70f33e5539/third_party/WebKit/Source/core/layout/LayoutTheme.cpp
[modify] https://crrev.com/0210f10d9f5ac5b1f47f374a31dc8f70f33e5539/third_party/WebKit/Source/core/layout/LayoutThemeDefault.cpp
[modify] https://crrev.com/0210f10d9f5ac5b1f47f374a31dc8f70f33e5539/third_party/WebKit/Source/core/layout/LayoutThemeDefault.h

Looks ok now at https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac10.11%20%28dbg%29