elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
Reported by
dominik....@gmail.com,
Aug 21
|
||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36 Steps to reproduce the problem: 1. Create iframe 2. Use some WebVR framework to enter VR mode 3. After entering VR mode try to target some element in the iframe via elementFromPoint / elementsFromPoint function http://jsfiddle.net/afuybnrt/ What is the expected behavior? Return an array of elements that are intersected on specific point in VR mode, like in normal mode What went wrong? Function elementFromPoint / elementsFromPoint returns in VR mode only root element of the i-frame, which is <html>, therefore these functions are unusable on i-frames when in VR mode. Did this work before? Yes Not sure, last stable version we used was about year old Chrome version: 68.0.3440.106 Channel: stable OS Version: 10.0 Flash Version: Works well on Safari and Firefox
,
Aug 23
,
Aug 23
The issue seems to be related to VR mode. Hence, adding label TE-Hardware-Dependency. Thanks...!!
,
Aug 27
Could you add OS macOS and Android, please? This is happening in all Chrome versions. Thank you.
,
Sep 25
This is a serious issue for us, what's required so the work on this issue is begun? Thank you.
,
Sep 26
Seems to be related to compositing in VR
,
Sep 26
Chrome's WebVR implementation only returns devices on Android, and Safari does not support WebVR, so I assume this is using the polyfill. dominik.kadlcek: Does "VR mode" mean the JavaScript-based "Cardboard mode" supported by the polyfill? Do you have the WebVR flag enabled in Chrome? (I'm assuming not, which would rule out the WebVR implementation as the cause.)
,
Sep 27
I thought VR mode in a-frame uses polyfill when needed, but I do have the WebVR flag enabled. And on Android, it is running in the WebVR mode. Is it somehow possible to downgrade Chrome to older versions so I can check which version it was running properly? Thank you.
,
Sep 27
On Android, you will know you are using the native WebVR implementation (using GVR) if a) you see the Daydream Device On flow or b) clicking the gear in the upper right corner takes you to the native VR settings page. If you disable the flag, do you get the same results? When using the WebVR implementation on Android, the page/DOM is no longer rendered (i.e., the rAF stops). That could explain a change in behavior, but this does not apply to other platforms, so if you are seeing it on those, it is probably a non-VR issue. There are instructions for downloading old Chromium builds at https://www.chromium.org/getting-involved/download-chromium#TOC-Not-as-easy-steps:
,
Oct 8
,
Oct 16
It seems to be native WebVR. When disabled flag, still the same results. I have checked several versions, it works until chrome 65 (including the 65). Since Chrome 66 it doesn't seem to be working.
,
Oct 19
I wonder, can't this issue somehow relate to this issue? https://bugs.chromium.org/p/chromium/issues/detail?id=843215 Thank you.
,
Oct 22
The NextAction date has arrived: 2018-10-22
,
Oct 22
Comment #12 seems to be on the right track, and may well explain what is happening. The WebVR team needs to look at this. But it may well also be OOPIF.
,
Oct 22
This is not a WebVR issue - AFrame is a javascript framework that can sit on top of WebVR or the polyfill. This bug as described in #1 is talking about the behavior with the polyfill. I see the same behavior as described on linux, which doesn't support webvr. For jsfiddle pages, it appears that the a-frame content is hosted in another iframe, and feature policy prevents that iframe from accessing webvr - so we go through polyfill on android even if webvr flags are enabled. When the button to enter VR mode is clicked, a canvas element in the page is made fullscreen. This means that the iframe we are trying to call elementFromPoint on isn't in the tree we are making fullscreen. I don't know what is the spec-compliant behavior here, since the iframe isn't visible at this point. Adding rakina@, who fixed the bug mentioned in #12. Updating title to avoid confusion about vr mode.
,
Oct 22
,
Oct 22
,
Oct 25
Hi, can you add some example calls to elementFromPoint/elementsFromPoint and the expected output? Not too familiar on how it should work in WebVR...
,
Oct 25
webvr isn't really involved other than a-frame (a javascript library in use on the sample page) typically uses the webvr api. However, in this case a-frame isn't able to use webvr so its polyfilling it and using fullscreen instead. go to http://jsfiddle.net/afuybnrt/ in debug tools, switch to debug the innermost iframe (it has a string about 404), and run elementFromPoint(100,100) from console to see 4 elements. click on the google cardboard viewer icon in the lower right iframe. this makes a canvas element fullscreen. run elementFromPoint(100,100) from console while the canvas is fullscreen. now see 1 element in results (html element) The expectation in #1 is that we should see multiple elements still.
,
Oct 25
Thanks for explaining! I don't think my change that was linked in #12 is the cause of this, it's mostly just affecting Shadow DOM retargeting. It seems there was some changes in Hit-testing code that takes place in a similar time, so let me add Layout to the components.. I'm OOO currently but I'll also try debugging to confirm in around a week.
,
Nov 14
Thank you for the investigation. Did you find out any news?
,
Nov 14
Is this a fullscreen bug, or just VR?
,
Nov 14
Fullscreen. No Chrome VR code is running in the repro - there is a javascript library running that pretends to implement the VR API by using fullscreen and reading sensor data.
,
Nov 14
,
Nov 15
,
Nov 19
I bisected this to: https://chromium.googlesource.com/chromium/src/+/818e182ca It is about propagating inert attribute into subframes. So this appears to be working according to the spec. In that since a element is going into the top layer the rest of the document and subframes become inert and therefor nothing participates in hit testing.. And elementsFromPoints returnings the document element. +domenic/foolip to correct me if I am wrong. But this seems to be correct so I'm guessing this is a WontFix. reporter@ do you have a use case where hit testing an iframe should work?
,
Nov 19
,
Nov 20
How's possible that an iframe is taken different way in full screen mode than in regular? If I'm able to target the elementsFromPoint on regular mode, why am I unable when in fullscreen mode -- shouldn't it act the same? This is a critical issue that affects our product, please add back the behavior allowing to get proper element(s) from elementsFromPoint called on iframe when in fullscreen mode.
,
Nov 20
Is there a reason you cannot put the canvas in a div and that iframe inside the div and fullscreen the div instead of the canvas?
,
Nov 20
Or fullscreening the document (as well will work instead of the canvas element)
,
Nov 21
Right, this seems to be working as intended. When something else is fullscreened on top of the iframe, elementsFromPoint() will not return the iframe. This follows from the spec for elementsFromPoint(): > For each layout box in the viewport, in paint order, starting with the topmost box, *****that would be a target for hit testing***** at coordinates x,y even if nothing would be overlapping it, when applying the transforms that apply to the descendants of the viewport, append the associated element to sequence. from https://drafts.csswg.org/cssom-view/#dom-document-elementsfrompoint, emphasis mine When fullscreen is on top of something, nothing besides the fullscreen element (and its subtree) are a target for hit-testing. That said, this may not be an intended consequence by the spec writers, so folks might want to discuss that further. In either case, it would be good to add a test and ensure that we are interoperable across browsers, one way or another.
,
Nov 21
Here is an example of it working. I added a click listener to the a-scene so you need to just click it and not the glasses to go fullscreen. http://jsfiddle.net/dtapuska/L938kzqt/ |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by schenney@chromium.org
, Aug 21