Thanks Caleb for filing this bug.

Yes, I am writing a telemetry based test for power consumption performance in videos, and automating fullscreen mode is important.

Originally I hoped for something similar to '--autoplay-policy=no-user-gesture-required' but failed to find any.

So now the path forward is through a 'real' click via DevTools. The js click() won't work.

Does the gpu_benchmarking_extension (used by many Telemetry-based tests) provide what's needed?
https://cs.chromium.org/chromium/src/content/renderer/gpu/gpu_benchmarking_extension.cc

If not, let's please add it.

No, tap goes through gpu_benchmarking_extension, but click does not, and gpu_benchmarking_extension doesn't have a click function.

OK. Then let's investigate either adding a Tab.click() to Telemetry that's plumbed through DevTools, or adding click() to gpu_benchmarking_extension. The decision probably depends on where you want to invoke it from – the Python-based test harness, or the web page itself.

+1 to using Devtools's Input.dispatchMouseEvent. I think the fact that we still rely on gpu_benchmarking_extension for synthetic gesture is a technical debt. Future gestures should be based on Devtools 

+Pavel/Sammy who might have more historical context

I agree, we've been wanting to remove gpu_benchmarking_extension for quite some time now. If we do this in DevTools then we could easily support this in things like puppeteer too.

I'm confused about #6.

Here's my understanding of the system architecture:

Telemetry uses DevTools.
Puppeteer uses DevTools.
ChromeDriver uses DevTools.

If you update Telemetry to use the proper DevTools API for click() then it will help zmo@ write a Telemetry benchmark that can fullscreen a video, but I don't think it would help you use Puppeteer. Presumably Puppeteer already uses the proper Input.dispatchMouseEvent API for clicks since the DevTools team implemented Puppeteer recently.

@einbinder

My understanding is that DevTools protocol has sufficient APIs for all kinds of input Telemetry might need and both Puppeteer and ChromeDriver are already using those. Might be subtle details in the ChromeDriver land, hence +Joel.

I am willing to take a shot at wiring Telemetry Click through DevTools if someone can point me to some examples.

Is the ChromeDriver code I linked to not a helpful example? Otherwise you could find the Puppeteer source code and use that. The API it will use is called "Input.dispatchMouseEvent", so you can search for that.

Ah, I missed the link. Thanks.

I got a CL working - thanks Pavel for helping me out. Will upload for review soon.

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/61bedb99ba87a7f855b56450626a29b6f581b363

commit 61bedb99ba87a7f855b56450626a29b6f581b363
Author: Zhenyao Mo <zmo@chromium.org>
Date: Wed Oct 03 01:49:41 2018

Wire action_runner's Click through DevTools

Bug: chromium:885912
Change-Id: I08b004dcd6c6c6a5147f1bd58713e2237f0bc2f3
TEST=run_telemetry_tests action_runner_unittest
R=nednguyen@google.com
Reviewed-on: https://chromium-review.googlesource.com/c/1256928
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>

[delete] https://crrev.com/2dd914402ebcc44ecb60034e5c7b9bfa3d0c9d89/telemetry/telemetry/internal/actions/javascript_click.py
[delete] https://crrev.com/2dd914402ebcc44ecb60034e5c7b9bfa3d0c9d89/telemetry/telemetry/internal/actions/mouse_click_unittest.py
[add] https://crrev.com/61bedb99ba87a7f855b56450626a29b6f581b363/telemetry/telemetry/internal/actions/click.py
[modify] https://crrev.com/61bedb99ba87a7f855b56450626a29b6f581b363/telemetry/telemetry/internal/actions/action_runner.py
[modify] https://crrev.com/61bedb99ba87a7f855b56450626a29b6f581b363/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py
[delete] https://crrev.com/2dd914402ebcc44ecb60034e5c7b9bfa3d0c9d89/telemetry/telemetry/internal/actions/mouse_click.js
[modify] https://crrev.com/61bedb99ba87a7f855b56450626a29b6f581b363/telemetry/telemetry/internal/browser/web_contents.py
[delete] https://crrev.com/2dd914402ebcc44ecb60034e5c7b9bfa3d0c9d89/telemetry/telemetry/internal/actions/mouse_click.py

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/5949d2dbe4356e515740bdcf5e75a205528a4d99

commit 5949d2dbe4356e515740bdcf5e75a205528a4d99
Author: Zhenyao Mo <zmo@chromium.org>
Date: Wed Oct 03 18:00:53 2018

Revert "Wire action_runner's Click through DevTools"

This reverts commit 61bedb99ba87a7f855b56450626a29b6f581b363.

Reason for revert: Seems to cause telemetry_perf_unittests failures on Android, Win, Linux bots. See https://chromium-review.googlesource.com/c/catapult/+/1256928

Original change's description:
> Wire action_runner's Click through DevTools
> 
> Bug: chromium:885912
> Change-Id: I08b004dcd6c6c6a5147f1bd58713e2237f0bc2f3
> TEST=run_telemetry_tests action_runner_unittest
> R=​nednguyen@google.com
> Reviewed-on: https://chromium-review.googlesource.com/c/1256928
> Commit-Queue: Zhenyao Mo <zmo@chromium.org>
> Reviewed-by: Kenneth Russell <kbr@chromium.org>
> Reviewed-by: Ned Nguyen <nednguyen@google.com>

TBR=zmo@chromium.org,kbr@chromium.org,pfeldman@chromium.org,nednguyen@google.com

Change-Id: I89228685700b2d53a42ebaabaa6a4e73a3984b0f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:885912
Reviewed-on: https://chromium-review.googlesource.com/1259323
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>

[add] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/actions/javascript_click.py
[add] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/actions/mouse_click_unittest.py
[delete] https://crrev.com/929a7f4eba49faea05967904a06f5260472c652a/telemetry/telemetry/internal/actions/click.py
[modify] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/actions/action_runner.py
[modify] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py
[add] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/actions/mouse_click.js
[modify] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/browser/web_contents.py
[add] https://crrev.com/5949d2dbe4356e515740bdcf5e75a205528a4d99/telemetry/telemetry/internal/actions/mouse_click.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8fbe05fb32cf430baa5ff7f13704f6bee7abeab4

commit 8fbe05fb32cf430baa5ff7f13704f6bee7abeab4
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Thu Oct 04 03:52:41 2018

Roll src/third_party/catapult 2dd914402ebc..0543f082e2f4 (7 commits)

https://chromium.googlesource.com/catapult.git/+log/2dd914402ebc..0543f082e2f4


git log 2dd914402ebc..0543f082e2f4 --date=short --no-merges --format='%ad %ae %s'
2018-10-03 khmel@chromium.org Fix flakiness of tests, that require GAIA login.
2018-10-03 kylechar@chromium.org Support macOS 10.14 Mojave.
2018-10-03 zmo@chromium.org Revert "Wire action_runner's Click through DevTools"
2018-10-03 chiniforooshan@chromium.org Telemetry: clean up legacy surface flinger metrics
2018-10-03 pasko@chromium.org androidStartupMetric: skip the first start
2018-10-03 benjhayden@chromium.org Fix flaky cp-toast.autoclose-only-last test.
2018-10-03 zmo@chromium.org Wire action_runner's Click through DevTools


Created with:
  gclient setdep -r src/third_party/catapult@0543f082e2f4

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:879353 , chromium:890951 ,chromium:885912, chromium:890757 , chromium:881384 ,chromium:885912
TBR=sullivan@chromium.org

Change-Id: I6e2ae7a29df4dcab18c0cde1e852ee159a3f12cf
Reviewed-on: https://chromium-review.googlesource.com/c/1260146
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#596495}
[modify] https://crrev.com/8fbe05fb32cf430baa5ff7f13704f6bee7abeab4/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/c771409079e4206986826ac6d152026e4635c534

commit c771409079e4206986826ac6d152026e4635c534
Author: Zhenyao Mo <zmo@chromium.org>
Date: Fri Oct 12 22:03:32 2018

Add userGesture bit for click action.

Bug: chromium:885912
Change-Id: I72acaf373f11d93bbaf57d385dcbe8f903c04fb0
TEST=run_telemetry_tests action_runner_unittest
R=nednguyen@google.com,pfeldman@chromium.org,kbr@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/c/1278270
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>

[modify] https://crrev.com/c771409079e4206986826ac6d152026e4635c534/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py
[modify] https://crrev.com/c771409079e4206986826ac6d152026e4635c534/telemetry/telemetry/internal/actions/page_action.py
[modify] https://crrev.com/c771409079e4206986826ac6d152026e4635c534/telemetry/telemetry/internal/actions/javascript_click.py
[modify] https://crrev.com/c771409079e4206986826ac6d152026e4635c534/telemetry/telemetry/internal/browser/web_contents.py
[modify] https://crrev.com/c771409079e4206986826ac6d152026e4635c534/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/02f960678270ba73d4616bf551c8c4148f261678

commit 02f960678270ba73d4616bf551c8c4148f261678
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Sat Oct 13 00:41:47 2018

Roll src/third_party/catapult 99b88b0a2d58..1de6f5099019 (2 commits)

https://chromium.googlesource.com/catapult.git/+log/99b88b0a2d58..1de6f5099019


git log 99b88b0a2d58..1de6f5099019 --date=short --no-merges --format='%ad %ae %s'
2018-10-12 benjhayden@chromium.org Add comment to KeyValueCacheRequest.
2018-10-12 zmo@chromium.org Add userGesture bit for click action.


Created with:
  gclient setdep -r src/third_party/catapult@1de6f5099019

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:885912
TBR=sullivan@chromium.org

Change-Id: Iaa6799ebfbf3db1032eb58e145392c4aa62717a6
Reviewed-on: https://chromium-review.googlesource.com/c/1279099
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#599444}
[modify] https://crrev.com/02f960678270ba73d4616bf551c8c4148f261678/DEPS