New issue
Advanced search Search tips

Issue 876226 link

Starred by 8 users

Issue metadata

Status: WontFix
Owner:
Closed: Nov 21
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android , Windows , Chrome , Mac
Pri: 2
Type: Bug-Regression



Sign in to add a comment

elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element

Reported by dominik....@gmail.com, Aug 21

Issue description

UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

Steps to reproduce the problem:
1. Create iframe
2. Use some WebVR framework to enter VR mode
3. After entering VR mode try to target some element in the iframe via elementFromPoint / elementsFromPoint function 

http://jsfiddle.net/afuybnrt/

What is the expected behavior?
Return an array of elements that are intersected on specific point in VR mode, like in normal mode

What went wrong?
Function elementFromPoint / elementsFromPoint returns in VR mode only root element of the i-frame, which is <html>, therefore these functions are unusable on i-frames when in VR mode.

Did this work before? Yes Not sure, last stable version we used was about year old

Chrome version: 68.0.3440.106  Channel: stable
OS Version: 10.0
Flash Version: 

Works well on Safari and Firefox
 
Components: -Blink Blink>WebXR>VR
Labels: Needs-Triage-M68 Needs-Bisect
Labels: Triaged-ET TE-Hardware-Dependency
The issue seems to be related to VR mode. Hence, adding label TE-Hardware-Dependency.

Thanks...!!
Could you add OS macOS and Android, please? This is happening in all Chrome versions. Thank you.
This is a serious issue for us, what's required so the work on this issue is begun?

Thank you.
Cc: paint-dev@chromium.org
Components: Blink>Compositing
Status: Untriaged (was: Unconfirmed)
Seems to be related to compositing in VR
Labels: OS-Android OS-Mac
Chrome's WebVR implementation only returns devices on Android, and Safari does not support WebVR, so I assume this is using the polyfill.

dominik.kadlcek: Does "VR mode" mean the JavaScript-based "Cardboard mode" supported by the polyfill? Do you have the WebVR flag enabled in Chrome? (I'm assuming not, which would rule out the WebVR implementation as the cause.)
I thought VR mode in a-frame uses polyfill when needed, but I do have the WebVR flag enabled. And on Android, it is running in the WebVR mode.

Is it somehow possible to downgrade Chrome to older versions so I can check which version it was running properly?

Thank you.
On Android, you will know you are using the native WebVR implementation (using GVR) if a) you see the Daydream Device On flow or b) clicking the gear in the upper right corner takes you to the native VR settings page.

If you disable the flag, do you get the same results?

When using the WebVR implementation on Android, the page/DOM is no longer rendered (i.e., the rAF stops). That could explain a change in behavior, but this does not apply to other platforms, so if you are seeing it on those, it is probably a non-VR issue.

There are instructions for downloading old Chromium builds at https://www.chromium.org/getting-involved/download-chromium#TOC-Not-as-easy-steps:
Labels: Needs-Feedback
NextAction: 2018-10-22
It seems to be native WebVR.

When disabled flag, still the same results.

I have checked several versions, it works until chrome 65 (including the 65).

Since Chrome 66 it doesn't seem to be working.
I wonder, can't this issue somehow relate to this issue? https://bugs.chromium.org/p/chromium/issues/detail?id=843215

Thank you.
The NextAction date has arrived: 2018-10-22
Components: -Blink>Compositing
Labels: -Needs-Feedback
NextAction: ----
Comment #12 seems to be on the right track, and may well explain what is happening. The WebVR team needs to look at this.

But it may well also be OOPIF.
Cc: rakina@chromium.org billorr@chromium.org
Summary: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element (was: elementFromPoint / elementsFromPoint called on i-frame when on VR Mode returns only root element)
This is not a WebVR issue - AFrame is a javascript framework that can sit on top of WebVR or the polyfill.  This bug as described in #1 is talking about the behavior with the polyfill.  I see the same behavior as described on linux, which doesn't support webvr.

For jsfiddle pages, it appears that the a-frame content is hosted in another iframe, and feature policy prevents that iframe from accessing webvr - so we go through polyfill on android even if webvr flags are enabled.

When the button to enter VR mode is clicked, a canvas element in the page is made fullscreen.  This means that the iframe we are trying to call elementFromPoint on isn't in the tree we are making fullscreen.  I don't know what is the spec-compliant behavior here, since the iframe isn't visible  at this point.

Adding rakina@, who fixed the bug mentioned in #12.

Updating title to avoid confusion about vr mode.

Components: Blink>DOM
Status: Available (was: Untriaged)
Hi, can you add some example calls to elementFromPoint/elementsFromPoint and the expected output? Not too familiar on how it should work in WebVR...
webvr isn't really involved other than a-frame (a javascript library in use on the sample page) typically uses the webvr api.  However, in this case a-frame isn't able to  use webvr so its polyfilling it and using fullscreen instead.

go to http://jsfiddle.net/afuybnrt/
in debug tools, switch to debug the innermost iframe (it has a string about 404), and run elementFromPoint(100,100) from console to see 4 elements.
click on the google cardboard viewer icon in the lower right iframe.  this makes a canvas element fullscreen.
run elementFromPoint(100,100) from console while the canvas is fullscreen.

now see 1 element in results (html element)

The expectation in #1 is that we should see multiple elements still.


Components: Blink>Layout
Thanks for explaining! I don't think my change that was linked in #12 is the cause of this, it's mostly just affecting Shadow DOM retargeting. It seems there was some changes in Hit-testing code that takes place in a similar time, so let me add Layout to the components.. 

I'm OOO currently but I'll also try debugging to confirm in around a week.
Thank you for the investigation. Did you find out any news?
Is this a fullscreen bug, or just VR?
Fullscreen.  No Chrome VR code is running in the repro - there is a javascript library running that pretends to implement the VR API by using fullscreen and reading sensor data.
Components: -Blink>DOM -Blink>WebXR>VR Blink>Fullscreen
Status: Untriaged (was: Available)
Owner: dtapu...@chromium.org
Status: Assigned (was: Untriaged)
Cc: domenic@chromium.org foolip@chromium.org
Labels: OS-Chrome OS-Linux
I bisected this to: https://chromium.googlesource.com/chromium/src/+/818e182ca

It is about propagating inert attribute into subframes.

So this appears to be working according to the spec. In that since a element is going into the top layer the rest of the document and subframes become inert and therefor nothing participates in hit testing.. And elementsFromPoints returnings the document element.

+domenic/foolip to correct me if I am wrong. But this seems to be correct so I'm guessing this is a WontFix.

reporter@ do you have a use case where hit testing an iframe should work?
Labels: -Needs-Bisect -TE-Hardware-Dependency
How's possible that an iframe is taken different way in full screen mode than in regular? If I'm able to target the elementsFromPoint on regular mode, why am I unable when in fullscreen mode -- shouldn't it act the same?

This is a critical issue that affects our product, please add back the behavior allowing to get proper element(s) from elementsFromPoint called on iframe when in fullscreen mode.
Is there a reason you cannot put the canvas in a div and that iframe inside the div and fullscreen the div instead of the canvas?
Or fullscreening the document (as well will work instead of the canvas element)
Right, this seems to be working as intended. When something else is fullscreened on top of the iframe, elementsFromPoint() will not return the iframe. This follows from the spec for elementsFromPoint():

> For each layout box in the viewport, in paint order, starting with the topmost box, *****that would be a target for hit testing***** at coordinates x,y even if nothing would be overlapping it, when applying the transforms that apply to the descendants of the viewport, append the associated element to sequence.

from https://drafts.csswg.org/cssom-view/#dom-document-elementsfrompoint, emphasis mine

When fullscreen is on top of something, nothing besides the fullscreen element (and its subtree) are a target for hit-testing.

That said, this may not be an intended consequence by the spec writers, so folks might want to discuss that further. In either case, it would be good to add a test and ensure that we are interoperable across browsers, one way or another.
Status: WontFix (was: Assigned)
Here is an example of it working. I added a click listener to the a-scene so you need to just click it and not the glasses to go fullscreen.

http://jsfiddle.net/dtapuska/L938kzqt/

Sign in to add a comment