I've seen the first flake before ( https://crbug.com/660952 ), and it seems has gone magically.

The second one (on lollipop) looks new. Anton, can you take a look?

The first failure happens to any test using mouse or keyboard events when something is obscuring the Chrome activity. To fix it we'd need to fix whatever is crashing on the device or showing some other bit of UI.

I suspect https://codereview.chromium.org/2470023002/ is the reason for the second failure.
Perhaps the test page gets the connected state change too soon for the test page to catch it?

+zhaobin

Looking into the second failure.

Re the patch in #2: in Blink we are resolving the PresentationRequest promise [1] and dispatching the state change event to "connected" [2] as two separate tasks. Despite this, the state change has already occurred by the time the statements chained to the promise is run, since [1] isn't synchronous with the 
chained code being executed.

This makes it a bit tricky to always ensure the initial PresentationConnection state is always connecting when the start() promise is resolved. In fact I am not sure if it is possible in general.

For now I think it would be acceptable to change the assertion of the test to either connected or connecting.

Anton, Mounir: WDYT?

[1] https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/presentation/PresentationConnectionCallbacks.cpp?q=f:webkit+f:presentation+onsuccess&sq=package:chromium&dr=CSs&l=36
[2] https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/presentation/PresentationConnection.cpp?dr&sq=package:chromium&rcl=1478881111&l=418

It seems, we send two mojo messages from PresentationServiceImpl synchronously, right? [1]

I assume Mojo preserves the order. If the messages are processed separately on the renderer message loop, but in the same order, we shouldn't have a problem - there'd be a message loop cycle between the promise being resolved and the state being changed to 'connected'.

It is possible the two Mojo messages are buffered and processed on the same call stack, then the promise is resolved and then the state changes to 'connected' immediately but again in the right order. AFAIK, both promise resolving and event dispatching in Blink are synchronous so we shouldn't have a problem either. However, if promise resolving is async, the event would be processed earlier. Is this what's really happening?

I'm fine with fixing the test by changing the expected state, esp. short term, but I think it would be better if we could fix the order for the developers (what if they count on the order of 'connecting' and then 'connected' event for their pages working?).

[1] https://cs.chromium.org/chromium/src/content/browser/presentation/presentation_service_impl.cc?rcl=0&l=307

Anton: from what I can infer, that's what's happening. The PresentationRequest resolver is resolved in a task on the renderer message loop first, then the state change event is dispatched in a later task on that same message loop. V8's ScriptPromiseResolver implementation has a conditional that implies that the promise does not get settled right away in all cases [1] - in cases where the test was failing, maybe we are hitting that case?

[1] https://cs.chromium.org/chromium/src/third_party/WebKit/Source/bindings/core/v8/ScriptPromiseResolver.h?sq=package:chromium&dr=CSs&rcl=1478895861&l=120

Hm, this condition (active DOM objects suspended) might be hit when the
page is in the background or something like that IIRC. Mounir might know
more.

On my side, just to remind that the Android test is a Java implementation of the native browsertest. See:
https://cs.chromium.org/chromium/src/chrome/test/media_router/media_router_integration_browsertest.cc?rcl=0&l=408

The test steps/behaviors on desktop and Android should be consistent.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8b0e9501ddf062423fd35f1ef00ba39085614dcc

commit 8b0e9501ddf062423fd35f1ef00ba39085614dcc
Author: zhaobin <zhaobin@chromium.org>
Date: Mon Nov 14 22:42:51 2016

[Presentation API] Fix integration test - initial state of connection can be 'connecting' or 'connected'

presentationRequest.start().then(function(connection) {
  // connection state could be 'connecting' or 'connected' here
  });

We created two async tasks on page: 1. resolve start() and invoke callback in then(); 2. change connection state to 'connected'. It seems that execution order of these two tasks is random. Will allow both state for now.

More details in  crbug.com/664629 

BUG= 664629 

Review-Url: https://codereview.chromium.org/2492393002
Cr-Commit-Position: refs/heads/master@{#431926}

[modify] https://crrev.com/8b0e9501ddf062423fd35f1ef00ba39085614dcc/chrome/test/media_router/resources/common.js