The issue seems to be that ClipboardTest/0.* leaves a window alive once their tests end, and then the later tests have trouble getting focus on the window.  

The code and tests for ClipboardTest don't seem to have been touched in a long time, so I'm not sure who should be made owner of this issue.

Sample output:
[1/543] ClipboardTest/0.ClearTest (99 ms)
[2/543] ClipboardTest/0.TextTest (94 ms)
[3/543] ClipboardTest/0.HTMLTest (96 ms)
[4/543] ClipboardTest/0.RTFTest (93 ms)
[5/543] ClipboardTest/0.TrickyHTMLTest (92 ms)
[6/543] ClipboardTest/0.UnicodeHTMLTest (91 ms)
[7/543] ClipboardTest/0.BookmarkTest (90 ms)
[8/543] ClipboardTest/0.MultiFormatTest (91 ms)
[9/543] ClipboardTest/0.URLTest (92 ms)
[10/543] ClipboardTest/0.SharedBitmapTest (102 ms)
[11/543] ClipboardTest/0.DataTest (90 ms)
[12/543] ClipboardTest/0.MultipleDataTest (90 ms)
[13/543] ClipboardTest/0.HyperlinkTest (90 ms)
[14/543] ClipboardTest/0.WebSmartPasteTest (92 ms)
[15/543] ClipboardTest/0.HtmlTest (87 ms)
[16/543] ClipboardTest/0.WriteEverything (89 ms)
[17/543] ClipboardTest/0.GetSequenceNumber (91 ms)
[18/543] ClipboardTest/0.WriteTextEmptyParams (90 ms)
[19/543] ClipboardTest/0.WriteURLEmptyParams (90 ms)
[20/543] ClipboardTest/0.WriteHTMLEmptyParams (91 ms)
[21/543] ClipboardTest/0.WriteRTFEmptyParams (92 ms)
[22/543] ClipboardTest/0.WriteBookmarkEmptyParams (90 ms)
[23/543] ClipboardTest/0.WriteHyperlinkEmptyParams (93 ms)
[24/543] ClipboardTest/0.WritePickledData (91 ms)
[25/543] ClipboardTest/0.WriteImageEmptyParams (90 ms)
[ RUN      ] ExtensionPointerLockTest.ExtensionPointerLockAccessFail

[2320:3212:0315/104029:WARNING:foreground_helper.cc(46)] Failed to send input; GetLastError(): 5

[2320:3212:0315/104029:ERROR:interactive_test_utils_win.cc(68)] ShowAndFocusNativeWindow failed. foreground window: 00000000, title: , path: 

e:\b\build\slave\win_gn\build\src\chrome\browser\apps\app_pointer_lock_interactive_uitest.cc(40): error: Value of: RunExtensionPointerLockTest("pointer_lock/no_permission")

  Actual: false

Expected: true

Can't focus window

[  FAILED  ] ExtensionPointerLockTest.ExtensionPointerLockAccessFail, where TypeParam =  and GetParam() =  (5211 ms)

Actually it looks like the clipboard tests don't need to gain focus of a window, so the rouge window is probably already around at the start of the test.

Since the rouge window seems to be created before the test starts, I'm going to pass this bug to the trooper since this seems like this might be an issue with the bots not successfully killing all windows from previous runs, and I don't really have a better idea of what to do with this bug :(

Taking a look...

I tried remote desktop-ing onto the bot and got "The trust relationship between this workstation and the primary domain failed."

friedman said to cc infra-labs, so here you go!

How about a machine name?

Detected 20 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

I'll just redeploy it... whats the machine name

It looks like this isn't just one machine failing... so far I've tested vm486-m4 and vm471-m4 and they both have said the same thing.

Releasing this bug since I'm no longer trooper, but labs should be taking a look at this...

The path e:\b\build\slave\win_gn\build.dead\f2d3d9c0562f46dcb2547b96ae84fdab\third_party\WebKit\LayoutTests\imported\web-platform-tests\html\semantics\scripting-1\the-template-element\additions-to-the-css-user-agent-style-sheet\css-user-agent-style-sheet-test-001-expected.html is too long.

Detected 14 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

Assume #10 is unrelated to this issue.

This is still happening so I'm assigning back to labs as per #9.

Also renaming bug and setting back to untriaged.

Detected 18 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

Detected 4 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

Appears this test is causing 10-20 flakes per day. I don't know how many times the test runs per day, but I assume that's not horrible so I'm lowering to pri-2. 

I clicked on a small set of flakes and they were all on different vm4xx-m1 VMs, so it's not just a single machine. Appears there is some test or application on the bots that leave always-on-top windows before interactive_ui_tests start, which breaks the tests.

cc:ing Pawel and sky@ as interactive_ui_tests owners. Do you have any Windows guru around (with bot access) that you can throw at this problem? :)

One thing that could be done is to look for top level windows (e.g. find any window with WS_EX_TOPMOST) before and after a task. 

1. When found before the task, the test would early exit. This should be done inside interactive_ui_tests itself upon startup.
2. When found after a task, it would mark the test as failed. This should be done inside the swarming bot (since it controls the host).

2. can be implemented in bot_config_public.py but right now the hook on_after_task() is NOT run within the task so doesn't permit to alter the task result. To do it from within the bot, a new took would be needed.

I looked at this log: https://build.chromium.org/p/tryserver.chromium.win/builders/win8_chromium_ng/builds/120213/steps/interactive_ui_tests%20%28with%20patch%29/logs/stdio . All failures appear to be happening because we can't bring the test to the front. These log entries indicate this is happening:

[3036:3464:0321/045701:WARNING:foreground_helper.cc(46)] Failed to send input; GetLastError(): 5
[3036:3464:0321/045701:ERROR:interactive_test_utils_win.cc(68)] ShowAndFocusNativeWindow failed. foreground window: 00000000, title: , path: 

An error code of 5 is ERROR_ACCESS_DENIED. I don't think this is because of an always on top window. Additionally there is code in chrome to close always on top windows before and after tests. That code generates log output if there are any always on top windows, and I don't see any.

The log output is from ForegroundHelper::SetForeground, which uses SendInput to bring a background chrome to the foreground. This only happens if chrome doesn't launch in the foreground, which can happen if another test ended up making chrome go to the background. It looks like we just started running these tests on win8. I'm guessing the security policy for using SendInput is different in win8, and we're getting the access denied.

I recommend *NOT* running interactive_ui_tests on win8 bots until this is sorted out. I'll revert one patch I see that changes this, but there may be others. I'm also changing the subject to indicate what I believe is going on.

Justin, I'm passing your way as I have a feeling you know who could best address this (and cc'ing scottmg and ananta).

Strange; we don't get any flakes on the windows 10 fyi bot, but then the tests haven't been enabled for a long time on there either.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fbd4ec4edc6087a29aa7e6bbde8c983ef2fa8439

commit fbd4ec4edc6087a29aa7e6bbde8c983ef2fa8439
Author: sky <sky@chromium.org>
Date: Mon Mar 21 17:53:33 2016

Revert of Add interactive_ui_tests to Win8 Aura config. (patchset #1 id:1 of https://codereview.chromium.org/1795853002/ )

Reason for revert:
interactive_ui_tests don't run well on win8 machines yet. See http://crbug.com/595071 . Reverting until sorted out.

Original issue's description:
> Add interactive_ui_tests to Win8 Aura config.
>
> BUG=373973, 594417 
>
> Committed: https://crrev.com/e040041ab6a37553a6feb5717472926c4fa4bc9d
> Cr-Commit-Position: refs/heads/master@{#381167}

TBR=dpranke@chromium.org,wfh@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=373973, 594417 ,595071

Review URL: https://codereview.chromium.org/1823583003

Cr-Commit-Position: refs/heads/master@{#382322}

[modify] https://crrev.com/fbd4ec4edc6087a29aa7e6bbde8c983ef2fa8439/testing/buildbot/chromium.win.json

Detected 25 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

Is this fixed now?

No. I reverted one place this test was recently added too. There are no doubt other places. That said, we want to run these tests on win8, so we need to fix the core issue.

jschuh@ Can you find someone to fix this? It's making the CQ flaky which slows everyone down. :(

I believe we've only recently started running interactive_ui_tests on newer windows machines. We should not run interactive_ui_tests on these machines until this is sorted out.

Will said he turned the tests off again. Personally, I'm fine ignoring anything other than Win7 and Win10.

Since this is assign, there is no need for the sheriff to be in the loop. Until the TryFlakes loops the sheriff back in.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3d084381a860ba3bb6755ad21caeb466527b4e30

commit 3d084381a860ba3bb6755ad21caeb466527b4e30
Author: Will Harris <wfh@chromium.org>
Date: Thu Mar 31 22:57:39 2016

Disable interactive_ui_tests on Win 10 test builder.

Interactive_ui_tests don't run well on win8+ machines yet.

This is breaking the Win 10 tests:

https://build.chromium.org/p/chromium.win/builders/Win10%20Tests%20x64/builds/29

BUG=595071
NOTRY=true
R=dpranke@chromium.org

Review URL: https://codereview.chromium.org/1852623002 .

Cr-Commit-Position: refs/heads/master@{#384427}

[modify] https://crrev.com/3d084381a860ba3bb6755ad21caeb466527b4e30/testing/buildbot/chromium.win.json

re:29 this does happen on Windows 10 as well, so this probably needs prioritizing.

Detected 4 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app.

this looks very related to  Issue 639350 , see Scott's latest CL there.

Detected 4 new flakes for test/step "interactive_ui_tests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLAsSBUZsYWtlIiFpbnRlcmFjdGl2ZV91aV90ZXN0cyAod2l0aCBwYXRjaCkM. This message was posted automatically by the chromium-try-flakes app.