BTW: It does work Firefox 60.0.2 and Safari 11.1.1 on macOS 10.13.5.

This is related to Siteisolation, since the same wasn't reproducible when siteisolation is disabled.

Thanks for the report and for adding the site isolation label.  Confirmed that I can also repro with Mac Canary 69.0.3464.0, and that this starts working when I turn site isolation off (which can be done by setting chrome://flags/#site-isolation-trial-opt-out to "Opt out").

Ken, I think you fixed some CSS problems with OOPIFs in the past (e.g., in  issue 823433 ).  Would you be able to take a look at how this one is different?

Interesting. The mouse events are getting correctly targeted to the video, but the coordinates are wrong. This happens both with and without enable-viz-hit-test-draw-quad, even though the browser coordinate conversion works differently on those paths.

Thanks for the report!  Sounds like https://threejs.org/examples/#css3d_youtube is the page where this was discovered (based on https://github.com/mrdoob/three.js/issues/14320).

The bug is partly due to cross-site iframes being put into a different process as part of Site Isolation, which is a new security feature to help mitigate Spectre/Meltdown.  More information about that feature here:
https://www.chromium.org/Home/chromium-security/site-isolation

We'll try to get this coordinates problem fixed quickly if possible.  Thanks!

Quick update: Ken is still investigating the coordinate issue to figure out why the CSS transform is affecting things.

Just to update, the problem is hit testing into iframes with 3D transforms applied and the transform-style:preserve-3d directive applied. There is some complexity missing from the browser hit testing code for both the viz and quad-based versions because the iframe and its container have coordinate spaces with different planes. Simply applying the transform to the point isn't enough.

+Chris

This sounds like it is going to be difficult to fix on the browser side.

Sadrul suggested what I think is a partial fix to the problem, which would be to have the InputTargetClient interface return the transformed point during a slow path hit test, rather than just the target widget. In that case we would have to continue using slow path hit testing whenever an OOPIF has a perspective transform applied in order to use the renderer's hit testing logic to change coordinate spaces properly.

The advantage of that approach is that it is an easy way to fix this bug. The downside is that it won't work in cases where the browser is transforming points from the transformed OOPIF to screen space, such as for select element popup menus, or context menus. Those would continue to appear in the wrong position.

Another approach is to send the transform for the target, instead of the location, back to the browser from blink. This would allow the browser to update the locations correctly for the other things.

I have a WIP that partially fixes the problem, along the lines of comment 11.
https://chromium-review.googlesource.com/c/chromium/src/+/1118612

It plumbs the transformed point back from the renderer process during an asynchronous hit test and uses that, rather than doing the conversion in the browser process. A problem there is that we still have to know to force an async hit test, and the only way I can see right now is to use async for all input events whenever *any* element has a transform that isn't 2D axis aligned with the page.

It seems like gfx::Transform::Preserves2dAxisAlignment() corresponds to when the transformation will or will not fail. I'm not certain that is correct, but it looked right in the test matrices I could come up with. Testing any of the following did not seem to work: non-invertibility, Transform::IsFlat(), Transform::HasPerspective().

The CL as it currently is (PS3) doesn't work with Viz hit testing enabled. That needs a parallel fix to what is change in surface_hittest.cc.

More significantly, there are still some weird things when it tries to transform events that don't/can't use asynchronous hit testing. This includes cases where we are targeting an event with mouse capture active, or generating extra MouseMove events for the purpose of MouseEnter/MouseLeave across iframes.

It would still be better if we could correctly transform points in the browser. At this point it looks like when we have a transform with a different normal from the page, then applying the inverse transform to a point doesn't correctly map the point into the element's coordinate space. However, I'm not clear on how to do that.

Another possible report of this in https://bugs.chromium.org/p/chromium/issues/detail?id=659642#c78.  From earlier today, it sounded like Ken was making progress on the fix.

+trchen and vollick

While I'm trying to land a fix for hit testing when these transforms are applied to out-of-process iframes, it won't solve the more general problem here. All I am doing is deferring to the renderer process for correct hit testing, rather than teaching the browser process to deal with non-flat matrices. So there will be other related bugs, such as positioning select element menus.

For a simplified test I have been using this CSS applied to an OOPIF:
iframe {
  transform-style: preserve-3d;
  transform: matrix3d(1,  0,  0,    0,
                      0,  1,  0,    0,
                      1,  0,  1,    0,
                      0,  0, -1000, 1);
}

I would prefer to have gfx::Transform provide some way to convert coordinates correctly, but I don't know exactly what that would require. Ian, Tien-Ren or chrishtr@, any thoughts on this?

Do we already have the fully accumulated screen matrix for OOPIFs in the browser process? I believe we do, because the browser process needs to be able to draw a renderer frame onto the screen.

The problem you're trying to solve is to map a point in screen coordinate to the local space of an OOPIF. This can be done by mapping a point through the inverse of the 2D homogeneous mapping. Which can be done by this:

screen_matrix.FlattenTo2d();
screen_matrix.TransformPointReverse(&point);

A had originally tried flattening the transformation matrix before mapping the point coordinates. That didn't seem to change the broken behavior. My approach was just to do that where we call TransformPoint in surface_hittest.cc. Is the suggestion in comment 16 somehow different from that?

I think that should just work. Will need more debug dumps to determine what went wrong exactly. My guess is that it maybe related to contents scale, but it is a weak confident guess.

Maybe we need both a simple short term fix (ask blink main when the situation is hard) and a longer better fix. 

Ideally the longer better fix now but it seems challenging to find the cycles immediately. I'd think that the right approach would ray cast to find the containing the FrameSink? There are some subtle issues: e.g. what to do with quads whose normals point into the page? (If we backface culled, they couldn't get events right? :-)

There seems to be multiple code paths involving hit testing. I tried to add some debug dump in surface_hittest.cc but nothing came out. Fixing HitTestDataProviderDrawQuad::GetHitTestData() worked for me locally. Try: https://chromium-review.googlesource.com/c/chromium/src/+/1150841

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a5ac92116fa34882c2a325518c2d1117fcc5e43a

commit a5ac92116fa34882c2a325518c2d1117fcc5e43a
Author: Ken Buchanan <kenrb@chromium.org>
Date: Thu Aug 09 02:01:04 2018

Flatten transforms when used for browser hit testing

Currently, when the browser process hit testing systems map points into
coordinates spaces with non-flat transforms, they are resulting in
incorrect targeting.

This patch flattens all transforms that are to be used for surface-
based hit testing, as well as the two current modes of Viz hit testing:
enable-viz-hit-testing-drawquad and enable-viz-hit-test-surfacelayer.

BUG= 854247 

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;master.tryserver.blink:linux_trusty_blink_rel
Change-Id: I3302b5b81a808e2be0e88d77efda35c35d287462
Reviewed-on: https://chromium-review.googlesource.com/1151962
Commit-Queue: Ken Buchanan <kenrb@chromium.org>
Reviewed-by: Tien-Ren Chen <trchen@chromium.org>
Reviewed-by: Fady Samuel <fsamuel@chromium.org>
Reviewed-by: Alex Moshchuk <alexmos@chromium.org>
Cr-Commit-Position: refs/heads/master@{#581742}
[modify] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/cc/trees/layer_tree_host_impl.cc
[modify] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/components/viz/client/hit_test_data_provider_draw_quad.cc
[modify] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/components/viz/service/surfaces/surface_hittest.cc
[modify] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/components/viz/service/surfaces/surface_hittest_unittest.cc
[modify] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/content/browser/site_per_process_hit_test_browsertest.cc
[add] https://crrev.com/a5ac92116fa34882c2a325518c2d1117fcc5e43a/content/test/data/frame_tree/page_with_non_flat_transformed_frame.html

Non-flat transforms without perspective should now behave properly.

On second thought, I can leave this open for the other CL, which adds a workaround fix for the problem of iframes with perspective transforms. I had thought there was a different bug for that but there isn't.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/44d7e2f6594c4772500473ff3dfdb9c9b3d80906

commit 44d7e2f6594c4772500473ff3dfdb9c9b3d80906
Author: Ken Buchanan <kenrb@chromium.org>
Date: Thu Aug 23 14:18:05 2018

Use renderer coordinate conversion in slow path hit testing

When the browser uses InputTargetClient to perform a slow path hit test
to an out-of-process iframe, it currently uses
TransformPointToCoordSpaceForView to convert the event's target point
into the OOPIF's coordinate space. This does not work correctly for
iframes with perspective transforms applied or otherwise non-invertible
transform matrices.

This CL causes InputTargetClient to return the transformed point from
the renderer process, which already covers such transforms. The browser
uses that point for subsequent hit tests or passes it back to the
renderer with the input event.

It also changes HitTestQuery and SurfaceHitTest to always defer to
slow path hit testing whenever any element with such a transform is
encountered, because it can't determine accurately whether the point
is within the bounds of the transformed element.

Bug:  854247 
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel
Change-Id: Ic9b69afe418bccdadab45480fc58b8942fccd09f
Reviewed-on: https://chromium-review.googlesource.com/1118612
Reviewed-by: Philip Rogers <pdr@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Alex Moshchuk <alexmos@chromium.org>
Commit-Queue: Ken Buchanan <kenrb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#585481}
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/components/viz/host/hit_test/hit_test_query.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/components/viz/service/surfaces/surface_hittest.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/browser/renderer_host/render_widget_targeter.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/browser/renderer_host/render_widget_targeter.h
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/browser/site_per_process_hit_test_browsertest.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/input/input_target_client_impl.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/input/render_widget_input_handler.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/input/render_widget_input_handler.h
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/render_widget.cc
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/render_widget.h
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/renderer/render_widget_browsertest.cc
[add] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/content/test/data/frame_tree/page_with_perspective_transformed_frame.html
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/services/viz/public/interfaces/hit_test/input_target_client.mojom
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/third_party/blink/public/web/web_hit_test_result.h
[modify] https://crrev.com/44d7e2f6594c4772500473ff3dfdb9c9b3d80906/third_party/blink/renderer/core/exported/web_hit_test_result.cc