Adding cc:
- robliao@ as Windows expert
- sky@ as BringBrowserWindowToFront in interactive ui tests according to https://chromium.googlesource.com/chromium/src/+blame/de351f393e5af7126425768903498b9c7180b951/chrome/test/base/interactive_ui_tests_main.cc

I'm not surprised. The code that attempts to bring the browser window to front is rather hacky.

This is a problem with the bot. I think this needs to goto a trooper. It is indicating that the top level windows is the EOSNotify window. Someone needs to log onto this machine and dismiss that window I think.

See also bug 825957

So the problem seems to be "Command line too long for RegisterApplicationRestart"

In line [1] RegisterApplicationRestart seems to be called with the full command line, while the documentation [2] states there is no need to specify the path to the executable. I wonder if replacing GetCommandLineString with GetArgumentsString will help to resolve the issue for now. If it does, it won't be a fix forever :\

I'm running the test locally now to figure out if it's going to work.

[1] https://cs.chromium.org/chromium/src/chrome/browser/chrome_browser_main_win.cc?q=chrome_browser_main_win.cc&sq=package:chromium&dr&l=625
[2] https://msdn.microsoft.com/en-us/library/windows/desktop/aa373347(v=vs.85).aspx

To confirm, I don't believe this is a problem with the bot. This seems to reproduce across many bots. I'm not sure what EOSNotify.exe is; I can try to find out though.

I was wrong. I caught a bot doing this; it's a windows update warning that this bot is out of date and needs to be updated before April 10th or something. So half of this is an infra issue. We've been having issues related to windows update lately, so maybe this is related? cc-ing some people.

The command argument list being too long is still a failure though, but seems to be less frequent.

Attached the error message.

Deleted: Screenshot from 2018-03-28 16-37-52.png 121 KB

	Screenshot from 2018-03-28 16-37-52.png 121 KB View Download

After looking closer at the error message, I believe command line length error is irrelevant here. The code that emits the message is for scheduling post-reboot autostarts, and reboots shouldn't happen in the tests; and it's just a warning, not an error.

And I think this is EOSNotify.exe. I did some google-ing, and didn't find anything specific, but I think it stands for End of Service notify? Running it manually on a bot didn't seem to do anything.

Sounds likely to be related, and would also explain why the other machines have been so insistent on updating themselves (Issue 819355).

I just ssh'd into win27-c1 (bot for Win10 Tests x64) and it's now on the new version of Windows (16299), which I don't believe we actually want for this builder (we want 15063).

Stephen, which bot did you catch like that, and what did you do with it?

They all seem to be machine provider machines. I ssh-ed into win10-0ab1efdf-us-west1-b-pnrp. When I saw that, I clicked "Not right now", and the tests ran ok after that.

To be clear, this is affecting CQ as well, which is what I was looking at.

I'm not sure if I touched win27-c1 at all; I might have, but I don't recall.

By CQ you mean win10_chromium_x64_rel_ng/win7_chromium_rel_ng, or other builders?

IIUC Machine Provider machines are spawned only for the duration of the task they run, which is why they're effectively fresh (non 16299) instances each time. How reproducible is this popup?

On second thought, this is highly related to Issue 819355, but it's possible Issue 818381 may address this immediate issue.

We're working on ver 1703 (Issue 818381). According to [1], that should help us here. It's unclear if it'll also address Issue 819355 though, even until Oct (crbug/819355#c44).

[1] https://support.microsoft.com/en-us/help/13853/windows-lifecycle-fact-sheet

I think it's just win10. win7 looks fine: https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/

So, there's no real short term fix. And from what I can tell, this is breaking most CLs that are going through CQ right now. I think we should remove win10 from the CQ for now :/ or maybe just disable this test suite on it?

 Issue 827089  has been merged into this issue.

Issue 825957 has been merged into this issue.

re #19: Alternatively:
- We could add code in our interactive ui tests  to TerminateProcess() the EOSNotify.exe process. Is it running with elevated privileges?
or
- We could only disable interactive ui tests. Other tests work fine.

I uploaded https://chromium-review.googlesource.com/c/chromium/src/+/986650, which should disable interactive ui tests on some platforms. 

Is it possible to run the bots on Windows 10 Server LTS? That may help address the support messages.

We've been requested specifically not to use server editions of Windows.

@26: What's the reasoning behind this?

I believe it's because the majority of our users are one.  There are differences between server windows and client windows that may skew test results.

+ someone on the windows team could probably provide more insight.

Alternatively, Windows 10 Enterprise LTSB/LTSC might also work here too. It's a client version of Windows intended for cases where frequent upgrades are undesirable.

does 1703 still pop the dialog in #8? if not, we should focus on the remaining work to update bots to 1703. This is tracked in 818381 and looks on track.

I'm surprised there isn't a gpo or something to disable these dialogs - there's certainly gpos to disable automatic updating if that is desired.

RS2 (1703) was canaried last night and is being rolled out today.  If all goes smooth all of the GCE bots should be on 1703 by EOD today.

FWIW our base image for RS2 is named "en_windows_10_enterprise_version_1703_updated_july_2017_x64_dvd_10925376.iso".  I think this is LTSB, right?

Our latest image also disables Windows Update via GPO.  From anecdotal experience it's a hit or miss whether or not the setting is respected.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5ae7e833c7a60d4c24353f3fd14dee55b44c5f36

commit 5ae7e833c7a60d4c24353f3fd14dee55b44c5f36
Author: Stephen Martinis <martiniss@chromium.org>
Date: Thu Mar 29 20:35:43 2018

Disable interactive ui tests

They're broken pretty badly on windows.

Bug:  826735 
No-Try: True
Change-Id: I65320b55b66d3780b542b8a0b521f72cee0f30f0
Reviewed-on: https://chromium-review.googlesource.com/986650
Reviewed-by: François Doray <fdoray@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546930}
[modify] https://crrev.com/5ae7e833c7a60d4c24353f3fd14dee55b44c5f36/testing/buildbot/chromium.win.json
[modify] https://crrev.com/5ae7e833c7a60d4c24353f3fd14dee55b44c5f36/testing/buildbot/test_suite_exceptions.pyl

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/094f08a9d67ce53085682825ea093cd5cc2e6982

commit 094f08a9d67ce53085682825ea093cd5cc2e6982
Author: Stephen Martinis <martiniss@chromium.org>
Date: Fri Mar 30 00:27:36 2018

Switch interactive_ui_tests to experiment

This won't block CQ, but will let us see if a potential fix works.

Bug:  826735 
Change-Id: I23c8e9a7b1df04206313df3fdbec58d733d17f7f
Reviewed-on: https://chromium-review.googlesource.com/986947
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#547042}
[modify] https://crrev.com/094f08a9d67ce53085682825ea093cd5cc2e6982/testing/buildbot/chromium.win.json
[modify] https://crrev.com/094f08a9d67ce53085682825ea093cd5cc2e6982/testing/buildbot/test_suite_exceptions.pyl

UI tests appear still to be failing in the experimental builds https://ci.chromium.org/buildbot/chromium.win/Win10%20Tests%20x64/

Issue 826223 has been merged into this issue.

Issue 826945 has been merged into this issue.

I may have spoken too quickly, as some of the bots were still being rolled out at the time.

Is this test still failing on the bots?

I don't think it's appropriate for me to own this. I'm not on call anymore.

It looks like this particular issue is no longer occurring. After the RS1 -> RS2 update, interactive_ui_tests is failing on BookmarkBarViewTest2.HideOnDesktopClick because the test needs updating to RS2, which is a dupe of Issue 828063. I'll mark this as fixed.

If this is fixed, should the exception added in r547042 be removed now?

Yeah, that sounds right. Stephen, can I volunteer you to roll back the change in your CL (https://chromium-review.googlesource.com/986947)? If someone else knows how to verify the correctness of the change after it lands that's fine too.

I'll re-enable it.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/94314156483094fb46539ebbab86699e7139f654

commit 94314156483094fb46539ebbab86699e7139f654
Author: Stephen Martinis <martiniss@chromium.org>
Date: Tue May 29 22:17:29 2018

Re-enable interactive_ui_tests

They seem to have been fixed and are running successfully.

Bug:  826735 
Change-Id: I34f7707986bf7517ed007364aa29144ac1056471
Reviewed-on: https://chromium-review.googlesource.com/1074205
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#562626}
[modify] https://crrev.com/94314156483094fb46539ebbab86699e7139f654/testing/buildbot/chromium.win.json
[modify] https://crrev.com/94314156483094fb46539ebbab86699e7139f654/testing/buildbot/test_suite_exceptions.pyl

Thanks, Stephen! I think this is done now but please reopen if there's more left.