NB: f():
    float(x * 215245U ) / float(0x3d0900U)  shows the bug as well (same on ints)
but float(x * 215245U ) / float(0x3d08ffU)  doesn't  ( same on ints )

Setting flags as suggested.

Submitter: can you please try to drill down a little deeper and figure out more precisely what seems to be going wrong either with the integer multiplications or int-to-float conversions? float and ceil are not just completely broken in ANGLE's shader translator. Something more subtle is going on.

Problem seems really to be in floor ( float( i * 215245 ) / float(0x3d0900) )
Bug is there for divider >= 0x3d0900 , no bug for less values.
( 0x3d0900 is decimal 4000000 , an awkward limit ).
Somehow big (u)int to float conversion strangely makes the final floor  failing.

I'll test more on Windows tomorrow (only linux at home).

here is a simpler version: https://www.shadertoy.com/view/ld3fWM

    O = vec4 ( U.y <   floor(  1e-6 * float ( int(U.x*4e5) ) )  );    // bug                                        
 // O = vec4 ( U.y <   floor(  4e-6 * float ( int(U.x*1e5) ) )  );    // nobug                                                

Might it be that in HLSL (or in transpilation) int->float translation is incorrect for ints larger than a float mantissa ? But how could this make floor ignored ?

I'll forward this bug to Microsoft. I don't think we'll make a workaround in ANGLE.

Automated test case here:

https://chromium-review.googlesource.com/c/angle/angle/+/1041959

HLSL pixel shader source generated by ANGLE attached.

Deleted: nested_floor.hlsl 1.7 KB

nested_floor.hlsl 1.7 KB Download

thanks !

Wow, you even drilled one step deeper : it was not tight to int/float conversion at all. 

I'm very curious what cause can do that, and when it might have impacted other math expressions.
Bu how are you sure it's D3D fault, and not the transpilation ? (isn't Angle in charge of the GLSL->HLSL transpilation ? or is it D3D's job ?)

I checked the generated HLSL pixel shader source. It looks correct. This can be done in various ways - partial source is available all the way from JS with WEBGL_debug_shaders extension, and complete source by attaching a debugger to ANGLE and inspecting the source passed to D3D.

The following revision refers to this bug:
  https://chromium.googlesource.com/angle/angle/+/03bb2231a821b2815a1ca472db47164af1350c6e

commit 03bb2231a821b2815a1ca472db47164af1350c6e
Author: Olli Etuaho <oetuaho@nvidia.com>
Date: Thu May 03 15:31:15 2018

Add test for D3D11 nested floor() bug

The test fails on D3D11, which seems to ignore floor() calls if
they're nested in a particular way with magic multipliers inside.

BUG=chromium:838885
TEST=angle_end2end_tests

Change-Id: I2c8749defa923553bfee69149da15f8e4ddf5096
Reviewed-on: https://chromium-review.googlesource.com/1041959
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>

[modify] https://crrev.com/03bb2231a821b2815a1ca472db47164af1350c6e/src/tests/gl_tests/GLSLTest.cpp

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ffd84f4151496ace08f3c022da67b210efe64098

commit ffd84f4151496ace08f3c022da67b210efe64098
Author: angle-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <angle-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu May 03 18:34:04 2018

Roll src/third_party/angle/ e708f004b..03bb2231a (1 commit)

https://chromium.googlesource.com/angle/angle.git/+log/e708f004bce5..03bb2231a821

$ git log e708f004b..03bb2231a --date=short --no-merges --format='%ad %ae %s'
2018-05-03 oetuaho Add test for D3D11 nested floor() bug

Created with:
  roll-dep src/third_party/angle
BUG=chromium:838885


The AutoRoll server is located here: https://angle-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
TBR=cwallez@chromium.org

Change-Id: Ieb3d226ee13b6f9f8595063974b44c341877d2b7
Reviewed-on: https://chromium-review.googlesource.com/1042530
Commit-Queue: angle-chromium-autoroll <angle-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: angle-chromium-autoroll <angle-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#555824}
[modify] https://crrev.com/ffd84f4151496ace08f3c022da67b210efe64098/DEPS

The HLSL compiler team has fixed this bug. The fix is targeting the next release of Windows and the Windows SDK.

Thank you Rafael! That's great news.