I think I have found the issue when it comes to invoking the telemetry API.

The webview we issue commands to is not deterministic when we have multiple screens (e.g. MIMO and regular screen) so sometimes we issue commands to the MIMO sometimes to the TV page.

This would explain why we only have issues on bluestreak test setups. I will look into tomorrow how to resolve the issue.

I think we managed to get down to the bottom of this.

The telemetry API has initially designed for hangouts which operated on all our test setups on a single window (read: screen). With mimo going live and having dual screen setups + mimo (possibly multiple mimos in the future) we need to update the telemetry API to accommodate multi-window support. 

The current problem is that http://cs/chromeos_public/src/third_party/autotest/files/client/common_lib/cros/kiosk_utils.py?l=31&rcl=4416ff5127e1af348f05fcf05a39f0b70133b356 returns only window/webview context, and it is not even deterministic. That causes our tests to send telemetry commands sometimes to the MIMO, sometimes to the main window.

Also, the MIMO window does not expose most telemetry commands - that will be addresses today by a separate cl/ on the front end side.

For context, multi window support was added in https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/524633 and we need to expand on that logic.

Harpreet, I am digging through the code base and notice some oddities.

Since we want to add new objects represeting the e.g. MIMO, woudl it make sense to make a facade for it and expose it through a new cfm_xmlrpc_server object?
I seems strange we're adding our APIs to the multimedia one directly.

I've tried to expose the mimo property on the cfm facade, but the adapter in multimedia allow only one level of delegation. What I mean by this: we can atm we can do cfmFacade.isInMeeting(), but not cfmFacade.main_scree.isInMeeting() from within a test. We're hitting http://cs/chromeos_public/src/third_party/autotest/files/client/cros/multimedia/multimedia_xmlrpc_server.py?l=85&rcl=4442d6fb2c7fa950a49aad4c38402003dc4eb484

I think it would make sense to add a new cfm rpc server for the future - whatever we have we will keep running/maintaing but not add new features.
The further I look at the infra here and the growing complexity of the project I believe it is in our best interest to define clearly how we interact with the cfm and its peripherals in our tests (through componentized APIs). As for writing tests we can always implement helper methods that would make them more generic and support running on different setups.

wdyt?

I don't see any benefit of creating a separate cfm_xmlrpc_server. Existing facades under multimedia provide other needed wiring for us to use the browser as well. It might be best to just add another facade for MIMO here itself if you want the ability to interact with both the MIMO and TV API's via the same test. In addition to this, you can make getting the webview context more deterministic by iterating through all available contexts and checking if the given context has MIMO or TV apis and returning the desired context.
https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/client/common_lib/cros/kiosk_utils.py?l=15-44


I would also suggest you to go through the existing CFM tests as well as the lab setup (go/cfm-lab-setup) to get an idea of how the current setup works.

Also, our long term plan should not be to have extensive number of end to end integration tests using these API's but rather to have few important such tests and split the rest into ChromeOS/Chrome specific, WebRtc specific, PA specific tests. We can talk about this more.

Just for clarity, the approach I mentioned would allow us to make calls like cfmFacade.isInMeeting() and mimoFacade.isInMeeting() from within a test. So I am not clear about any added benefit for having multi level api delegation as you suggested but I am open to any changes you want to try out.

I was just under the impression that we want to have the mimo and main_scree (tv Ui) properties under the cfm facade available which technically is not feasible as I found out yesterday - there is nothing blocking/speaking against adding a new mimo facade (and will not have to refactor existing tests).

Doing so we can simply treat the existing cfm facade as the TV UI facade which should in any setup. Caveat: the OOB procedure will evetually be reediness so we might have to do some extra work when it comes to testing OOB w/ or wo/ mimo (minor though as it should just be test code).

Thanks for the comments re xmlrpc_server, I was not aware that we use it for things besides the cfm facade so it makes sense to keep it where it is.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/2ee011ca40f182acd7c85f3776138064678c67ed

commit 2ee011ca40f182acd7c85f3776138064678c67ed
Author: Denis Tosic <dtosic@chromium.org>
Date: Fri Aug 25 19:09:02 2017

Pick webviews based on the 'screen' url param

cfm_facade_native is a simple API wrapper that issues commands
to a webview. The webview picking logic is not working properly
if there are multiple webviews available (e.g. MIMO + main screen)
and any one of them could be returned.

This change adds the screen param to the constructor to be able
to create cfm facades for specific webviews. It also updates the
necessary logic in kiosk_utils.py, points the _cfm_proxy (in
cfm_facade_adapter) to use the main screen and exposes a new mimo
facade in the xmlproxy that will be used in future changes/tests.

This change should not affect existing tests as all tests that use
the cfm_facade_adapator.cfm since commands are now always sent to
the main window.

TEST=local setup
BUG= chromium:756166 

Change-Id: Idb8b7f490ed17c09dc6e01f0ff79a4bf17edff6c
Reviewed-on: https://chromium-review.googlesource.com/628540
Commit-Ready: Claes Malmnäs <malmnas@google.com>
Commit-Ready: Denis Tosic <dtosic@chromium.org>
Tested-by: Claes Malmnäs <malmnas@google.com>
Tested-by: Denis Tosic <dtosic@chromium.org>
Reviewed-by: Claes Malmnäs <malmnas@google.com>
Reviewed-by: Denis Tosic <dtosic@chromium.org>

[modify] https://crrev.com/2ee011ca40f182acd7c85f3776138064678c67ed/server/cros/multimedia/cfm_facade_adapter.py
[modify] https://crrev.com/2ee011ca40f182acd7c85f3776138064678c67ed/client/common_lib/cros/kiosk_utils.py
[modify] https://crrev.com/2ee011ca40f182acd7c85f3776138064678c67ed/client/site_tests/enterprise_RemoraRequisition/enterprise_RemoraRequisition.py
[modify] https://crrev.com/2ee011ca40f182acd7c85f3776138064678c67ed/client/cros/multimedia/cfm_facade_native.py
[modify] https://crrev.com/2ee011ca40f182acd7c85f3776138064678c67ed/client/cros/multimedia/multimedia_xmlrpc_server.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/53a0d85dc911c6d9b852a421f9dcfe5e677b94cd

commit 53a0d85dc911c6d9b852a421f9dcfe5e677b94cd
Author: Denis Tosic <dtosic@chromium.org>
Date: Tue Sep 19 12:25:10 2017

Expose the main and mimo screen properties

This will be used in cfm autotests to interact with the
corresponding screen. The API exposed on these properties
is the native facade bound to them in
/client/cros/multimedia/multimedia_xmlrpc_server.py.

BUG= chromium:756166 
TEST=none

Change-Id: I6cbdd3af7b50d02f7f609fc1259048af4ed81fee
Reviewed-on: https://chromium-review.googlesource.com/640950
Commit-Ready: Denis Tosic <dtosic@chromium.org>
Tested-by: Denis Tosic <dtosic@chromium.org>
Reviewed-by: Harpreet Grewal <harpreet@chromium.org>
Reviewed-by: Min Zhuo <mzhuo@chromium.org>

[modify] https://crrev.com/53a0d85dc911c6d9b852a421f9dcfe5e677b94cd/server/cros/multimedia/cfm_facade_adapter.py

This has been resolved, please open new bugs for new issues.