Galaxy S6 Web Audio Buffer currentTime interval too long
Reported by
j...@with.in,
Jun 5 2017
|
||
Issue descriptionExample URL: https://with.in/watch/under-neon-lights/ Steps to reproduce the problem: 1. Go to https://with.in/watch/under-neon-lights/ 2. Click "Start in 2D" Button once experience loads 3. See that animation playback is choppy despite rendering at 60FPS What is the expected behavior? I expect the animation to playback smoothly. This is the case on the latest version of Chrome Stable on my Pixel as well as my desktop on all browsers. What went wrong? Under Neon Lights relies on the Web Audio API's Audio Buffer interface to infer the timing of the song. Specifically, the experience relies on the `currentTime` getter. I suspect that currentTime isn't granular enough for syncing up animations at 30FPS or even 24FPS. Again, to reiterate this is not the case on my Google Pixel or any browser on my PC ( Chrome, FF, Edge ). The animation is smooth. Did this work before? No Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? N/A Chrome version: 58.0.3029.83 Channel: stable OS Version: 7 Flash Version: Contents of chrome://gpu:
,
Jun 5 2017
The Galaxy S6 does not support Android's low latency audio (when I last checked). In this case WebAudio uses AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER = 1440 frames. At 48 kHz, this works out to be 30 ms. This makes context.currentTime get updated in a really bursty manner. So, currentTime gets updated very quickly 11 times, and then waits for about 30ms before getting updated again. I'm assuming this is what you're seeing. On desktop, the buffer size is much smaller (10 ms or less, 5 ms on OSX). I don't have Pixel to test with, but I'm pretty sure it supports Android's low-latency audio so the buffer size is probably about 256, like OSX. If you get a dev builds of Chrome, you can examine audioContext.baseLatency to see what kind of latency you're getting. If you can, please let me know what value is returned.
,
Jun 5 2017
Thanks rtoy@ that totally sounds like the issue. I'll look into that and report back on the value. Thanks!
,
Jun 6 2017
Sorry for the delay. This is what the console reads for my Galaxy S6: ctx.baseLatency 0.08016666666666666
,
Jun 6 2017
Thanks! So, yeah, the low-latency path is not available, so you get a huge latency which affects the timing of currentTime. I think there are two ways forward here. 1. Use the new AudioContext.getOutputTimeStamp() method to get a timestamp that relates currentTime with performanceNow time. This is intended to allow developers to synchronize currentTime with performance time. You will have to do your own smoothing and what not to account for the funny currentTime jitter behavior that occurs for such huge latency values. If you do this, I would very much like to know how well it works (or doesn't!). 2. Use the AudioContext latencyHint value to specify the exact latency you want. Unfortunately, this isn't yet implemented in Chrome (but is coming soon). Even if it were available today, I would not recommend using this unless you know exactly what devices you'll be using and how it behaves on those devices.
,
Jun 6 2017
Oh wow, that is so cool. Sounds like the prior is the perfect feature! Thanks for the intel on that. Is that part of the WebAudio spec ( i.e do you know off hand if that also runs in Safari, FF, and Edge? )
,
Jun 6 2017
Yes, it's part of the spec. I unfortunately don't know if any one else has implemented this yet. getOutputTimestamp arrived in Chrome 57. If you have issues with getOutputTimestamp, please file bugs. Nevertheless, I would love to hear your experience with it. Don't know of any one using this feature yet.
,
Jun 6 2017
Awesome. Will do! Thanks rtoy@
,
Jun 6 2017
Also, do you know is there someone at Samsung I could reach out to in order to allow Chrome to have lower latency on a check like this?
,
Jun 6 2017
So upon further poking the "Samsung Internet" app doesn't have the issue that Chrome does... Upon inspecting the context there is no read-property baseLatency, but the currentTime has a much lower latency than in Chrome making for smooth looking animation. Maybe there's something new in Android 7.0 that allows Chrome to have low latency as well?
,
Jun 6 2017
Unfortunately, I don't know of anyone at Samsung that could help with this. As for the Samsung Internet browser, it could have access to things that Chrome doesn't. I do know that many Samsung device do have really great latency, but I don't know which devices they are or whether they're exposed via the Android low-latency API. I think Android 7 has a Pro Audio API, but my understanding is that if a device supports that API, it must also support the low-latency API. Sorry I can't be more helpful.
,
Jul 14 2017
Hey rtoy@, Sorry for the delay on this. I've revisited the issue and I made a mistake. The image that I attached in Comment 4 (https://bugs.chromium.org/p/chromium/issues/detail?id=729615#c4) is ACTUALLY the baseLatency of a Google Pixel. The attached image here shows the baseLatency for Chrome 59 (Stable) on my Galaxy S6 on the left. The window on the right is the baseLatency for the Samsung Internet Browser. As you can see in the image, Chrome's baseLatency on the Pixel (0.08016666666666666) is 2x smaller than on the Galaxy S6 (0.16033333333333333). In addition, the Samsung Browser doesn't support the baseLatency parameter, but in testing I'll reiterate that it runs smoothly. Finally, I've tested all this on a newly bought Galaxy S8 with an SD835 processor which I'm told by Oculus' Carmel Browser Team that it's most similar to the Pixel in hardware architecture. The Galaxy S8 also returns a baseLatency value of 0.16033333333333333. All of this leads me to believe that this is actually a Chrome discrepancy, not a performance issue nor a hardware discrepancy ( desktop vs mobile, pixel vs s6 ). I've currently fixed https://with.in/watch/under-neon-lights/ to not rely on a WebAudio Buffer's `currentTime` and just using requestAnimationFrame deltas I'm able to get a smooth animation playback. I'd like to reiterate that I think there's a possible enhancement in Chrome to make sure that the baseLatency is more consistent across mobile devices. While this fixes my specific issue, it would be more appropriate to rely on a WebAudio Buffer's currentTime to ensure that audio and video never get out of sync. Thanks for taking the time to address this!
,
Aug 25 2017
Sorry for the delay. I don't know anything about the Samsung Browser, but it's certainly possible for them to take advantage of all kinds of Samsung-specific features. For chrome, we can't do much if the Galaxy S6 doesn't report that it supports Android's low-latency path. While it may actually work, there's no way for use to tell. Finally, it's not really possible for WebAudio to keep sync with video. They run on different clocks so they will get out of sync. It's up to the developer to keep the video in sync with the audio. (Skipping a frame of video normally doesn't cause a noticeable effect, but skipping a frame of audio is.) You'll have to use the audio timestamp along with performance Now time to keep these in sync yourself.
,
Aug 25 2017
Closing as WontFix (WAI) Chrome uses the appropriate buffer sizes as reported by the Android audio system. |
||
►
Sign in to add a comment |
||
Comment 1 by dalecur...@chromium.org
, Jun 5 2017