Confirmed, this bug doesn't happen in Safari either.

@Senorblanco, can you route this?

An example which makes the problem more apparent can be seen here:
http://cs.sru.edu/%7Eddailey/svg/distortGrid0.svg Compare both speed and
image quality between Chrome and Firefox.

I'm not sure if we're outside compatibility here. I'll need to read the specs to be sure.

The speed difference is likely because Firefox on Windows uses Direct2D IIRC, and the filters are accelerated.

This will be fixed when GPU Rasterization is launched on Windows. (You can preview it today by setting "GPU Rasterization" to "Enabled" in chrome://flags.)

I assume the image fidelity issue is a matter of only taking on sample from the "color" input. If so, senorblanco is right that we're not "outside compatibility" here (i.e how the sampling is to be performed - number of samples et.c - is not specified. The spec says:

"When applying this filter, the source pixel location will often lie  
 between several source pixels.

 Note: Depending on the speed of the available interpolents, this choice 
 may be affected by the image-rendering property setting."

(https://drafts.fxtf.org/filters/#feDisplacementMapElement; That note was added after the filter part of the spec was broken out, IIRC.)

Fun fact: The viewer used to render the "reference" for the (only) feDisplacementMap test in the testsuite [1] did use bilinear interpolation (presumably 4-tap).

[1] https://www.w3.org/Graphics/SVG/Test/20110816/harness/htmlObjectApproved/filters-displace-01-f.html

Since our implementation is within spec, I'm going to mark this as WontFix.

Our result on the repro and on the spec test case looks the same as Firefox on Mac (which also seems to be doing 1-tap).

Does it make sense to pass this on to the SVG WG? It seems like the lack of
clarity of the image after deformation is unacceptable from an
artistic/visual perspective. If I'm understanding it correctly, then the
filters folks with the Working Group, perhaps should think again about how
the interpolation should be done. Or when you write "This will be fixed
when GPU Rasterization is launched on Windows. " does that mean the
interpolation will automatically improve?
Perhaps the Working Group wrote the spec correctly and are just giving
everyone a chance to get up to speed? Thanks for all your research here
(and the fun fact!).

Sorry, the "this will be fixed w/GPU rasterization" comment was with respect to speed, not quality.

Thanks again. In case anyone's interested, I raised the issue with SVG-WG:
https://lists.w3.org/Archives/Public/www-svg/2016Aug/0011.html.

Any feedback on this should go to the "FX Taskforce" since they are the "group" managing the filters spec now[1]. Hopefully someone redirects it there from www-svg though...)

[1] https://drafts.fxtf.org/filters/ see "Feedback" in the header.