New issue
Advanced search Search tips

Issue 454849 link

Starred by 16 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug

Blocked on:
issue 502824


Show other hotlists

Hotlists containing this issue:
Gamepad


Sign in to add a comment

Can never play audio/video with gamepad input on Android

Reported by ashlaa...@gmail.com, Feb 3 2015

Issue description

Example URL:
http://www.scirra.com/labs/bugs/playoninput/

Steps to reproduce the problem:
Using a desktop browser, which works:
1. Visit the URL in a desktop browser.
2. Notice audio starts playing immediately without any user input.

Using touch input on mobile, which works eventually:
3. Visit the URL in Chrome for Android.
4. Notice no audio starts playing, since due to  issue 178297 , a user input is required to start playing audio.
5. Touch the screen. A moment later audio should start playing (since it issues the play() call in a touchstart event, which is allowed)

Using a gamepad on mobile, which never works:
6. Visit the URL in Chrome for Android (or reload if still on it).
7. Connect a gamepad (e.g. Xbox 360 controller) to an Android 4.4+ device. I tested on a Moto X running Android 5.
8. Press gamepad buttons.
9. Note the page recognises the input, but the music never plays.

What is the expected behavior?
HTML5 games controlled exclusively by gamepad input, without any touch activity, should still be able to play audio/video.

What went wrong?
The gamepad API has no input events to issue play() calls in - it simply updates the state of input arrays every frame. Even so, pressing gamepad buttons does not unblock the restriction that audio or video playback cannot begin outside of a user input event.

Did this work before? No 

Is it a problem with Flash or HTML5? HTML5

Does this work in other browsers? N/A 

Chrome version: 40.0.2214.93  Channel: stable
OS Version: 5.0
Flash Version: n/a

Any HTML5 game in Chrome for Android controlled exclusively by gamepad input will never be able to play audio or video.

My preferred resolution would be to resolve  issue 178297  by allowing playback at any time on mobile. As the comment thread describes, this limitation is pointless considering that the Web Audio API can play audio at any time, and only hinders legitimate use cases. The workaround is to play the music with Web Audio, but this means fully downloading and decoding the entire track before playback starts, and this is made worse by the fact audio decoding is very slow on Android ( issue 424174 ).

Even if the restriction is kept, then audio/video playback should be allowed at any time while any gamepad input is being provided, so that the use case of a gamepad-controlled HTML5 game in Chrome for Android can still play audio and video.

 

Comment 1 by sats...@gmail.com, Feb 3 2015

Hi,
I just saw your post and you are stating that "this limitation is pointless considering that the Web Audio API can play audio at any time, and only hinders legitimate use cases. The workaround is to play the music with Web Audio, but this means fully downloading and decoding the entire track before playback starts, and this is made worse by the fact audio decoding is very slow on Android"
Do you have any example with a very small audio that autostarts on android with Web Audio Api?
Thank you!
I don't see that that has anything to do with this bug? Maybe ask on StackOverflow?
Labels: -Cr-Internals-Media Cr-IO-Gamepad
Owner: qin...@chromium.org
Status: Assigned
I am able to repro on S5/KOT49H
Labels: -Pri-2 Pri-3
Components: -IO>Gamepad Blink>GamepadAPI

Comment 7 by scheib@chromium.org, Jul 11 2017

Cc: mustaq@chromium.org rbyers@chromium.org iclell...@chromium.org mlamouri@chromium.org
Components: Blink>Media>Audio Internals>Media>UI Programs>Alignment Blink>Input
Owner: ----
Status: Untriaged (was: Assigned)
I was unable to repro cleanly with Chrome 59 Android, in that I couldn't get audio to play at all on Android even with touch events. 

I see that  issue 178297  (Android Chrome does not allow applications to play HTML5 audio without an explicit action by the user) was closed recently WontFix, but that exceptions are made.

Marking this Untriaged until we can find the contributors who were/are actively working on these interventions.
scheib@, are the events fired by the gamepage somehow translated to keyboard/mouse events or are they special events that maybe the user gesture code in Blink should take into account?
mlamouri@ gamepad events are specific events and don't go through the normal input event pipeline.

I believe there might be a chrome ipc channel or mojo channel that backs the events and then they end up here:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/gamepad/NavigatorGamepad.cpp?sq=package:chromium&l=153
Status: Available (was: Untriaged)
Yeah when gamepad is being used for triggering links, it should totally take a user-gesture the same as keyboard/mouse.  Eg. you should also be able to open pop-ups or go fullscreen with the gamepad.  This is likely an easy fix I think.

scheib@: Note that tapping (touchend/click) also ALWAYS has a user-gesture.  The touch issues you're referring to are just about potential scrolling scenarios (touchstart, touchmove).  If you're seeing cases where touchscreen tapping doesn't allow audio to play then that's definitely a separate bug that's news to me.
Blockedon: 502824
Owner: mattreynolds@chromium.org
Status: Started (was: Available)
I couldn't repro on Android, Chrome 60.0.3112.116. The Audio input test says "Is playing: yes" and responds to touch inputs but I don't get any output.

The NavigatorGamepad events referenced in #9 are gamepadconnected and gamepaddisconnected events, not the button/axis input events you would want for a user gesture. Gamepad input is polling-based and doesn't generate any input events. However, connection events are only dispatched to the renderer if we've already received an input event. It should be sufficient to simulate a user gesture when NavigatorGamepad first dispatches gamepadconnected.

Unfortunately, we are not very consistent about issuing gamepadconnected. I have a CL in progress which should enable us to fix this issue.  crbug.com/502824 
Labels: UserActivation

Comment 14 by a...@scirra.com, May 15 2018

Note that since Chrome 66 transferred the same restrictions to desktop, gamepad-controlled web games on desktop are now permanently muted, with no way to unmute via the gamepad. The user must reach for their keyboard or mouse to provide an input event to unmute audio, making it an awkward experience to play browser games with a gamepad. See issue 840866.
The issue should be fixed in M68 (current canary/dev) when User Activation v2 is enabled through chrome://flags, see  Issue 381596 .  Can someone double-check please?
Labels: Needs-Feedback
I can't verify this with the provided test case. On both desktop Chrome and Android, "Is playing: yes" is always shown. Audio starts to play if a user activation is received, but only after switching away from the tab and switching back. E.g., on Chrome 68 for Linux, I get audio only if I click and then switch tabs:

1. Navigate to http://www.scirra.com/labs/bugs/playoninput/ ("Trying to play audio" is logged under "Events:")
2. Tap/click on the page ("On touch input" is logged. No audio.)
3. Switch to a different tab (No audio.)
4. Switch back to the playoninput tab. (Audio starts to play.)

The reverse order (switch tabs, then click) doesn't work. The tab switch must happen after the click.

1. Navigate to http://www.scirra.com/labs/bugs/playoninput/ ("Trying to play audio" is logged under "Events:")
2. Switch to a different tab (No audio.)
3. Switch back to the playoninput tab (No audio.)
4. Tap/click on the page ("On touch input" is logged. No audio.)
5. Switch to a different tab (No audio.)
6. Switch back to the playoninput tab. (Audio starts to play.)

I observed the same behavior on Chrome 66 for Android. I tried with and without UserActivationV2, but it made no difference.

It's difficult to see what the test case is doing because the relevant code is minified. ash@, would you mind providing a cleaner test case?
Also, here is the relevant commit that should enable this behavior on M68 with UserActivationV2:

https://chromium-review.googlesource.com/c/chromium/src/+/1029117

Treat gamepad button presses as user activations

Gamepad button presses do not generate input events, which means APIs
like the Fullscreen API which require a user activation may not
function correctly when the user is interacting primarily through gamepad
inputs. This CL checks the gamepad button state each time gamepads are
polled and signals that the frame has received an activation if at
least one button is pressed.

To reduce the likelihood that inadvertent inputs may generate a user
activation, a button press is only counted if the button value is at
least 75% of the maximum value. This prevents triggers and other analog
inputs from causing user activation from a light touch. Analog axes are
not considered for user activation.

This change requires the UserActivationV2 flag to be enabled to have any
effect.

BUG= 381596 
It looks like issue 856290 could fix this if the new gamepad events count as user gestures.

Sign in to add a comment