Note that the demo link I posted is a slightly modified copy of the demo in this git repo - https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer - which is a fork of the popular mapbox-gl-js renderer.

I had to copy the demo as I have applied the workaround (mentioned above) to the code in the git repo, but here I need to show what it looks like without the workaround.

Tried testing the issue on reported chrome version #69.0.3497.100 and latest chrome version #71.0.3576.0 using Mac OS 10.12.6 by following below steps.

Steps:
=====
1.Launched chrome.
2.Navigated to "chrome://settings", disabled hardware acceleration.
3.Navigated "https://s3-eu-west-1.amazonaws.com/bug-demo/google.html".
4.Observed no glitches on the web page.
Note: Tried with enabling hardware acceleration, unable to observe any glitches.

Attached screencast for reference.
@reporter: Could you please review the attached screencast and let us know if anything is being missed here.
Thanks.!

Deleted: 894021.mp4 3.5 MB

	894021.mp4 3.5 MB View Download

It looks like that was recorded on a Mac where there is no fallback CPU-based WebGL, and indeed that's what seems to happen (there is nothing at all on top of the base google maps). 

go to chrome://gpu to see the status of webgl

Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

(just to be clear, you need to be on windows)

Able to reproduce the issue on reported chrome version #69.0.3497.100 and on latest chrome #71.0.3578.0 using Windows 10( as mentioned in comment#4) by following steps in comment#0. The following are the observations noted while triaging the issue.

Observations:
===========
1.Able to reproduce the  issue 67 .0.3396.0, after that when tried on the builds from M-68 to 69.0.3495.0(M-69) a different behaviour is observed(as per screencast).
2.From M-68 to 69.0.3495.0(M-69) after disabling hardware acceleration in settings, no glitches were found on the browser page and the webpage displayed is different from the good behaviour(as per screencast).

Attached screencast for reference.
@reporter: Could you please review attached screencast and let us know if anything is being missed here. If possible request you to provide any other file/URL which reproduces the issue consistently so that it would be really helpful for further triaging of the issue and attach gpu details by navigating to chrome://gpu.
Thanks.!

Deleted: 894021(2).mp4 6.3 MB

	894021(2).mp4 6.3 MB View Download

It looks like software webgl is still not working (which has nothing to do with this bug).  Pleas go to chrome://gpu to see if software rendering is available. 

Also please note the following:

source:  https://developers.google.com/web/updates/2012/02/SwiftShader-brings-software-3D-rendering-to-Chrome

SwiftShader is a software 3D renderer used in Chrome 18 that lets you use CSS 3D and WebGL even on blacklisted GPUs. SwiftShader is available only on Windows and kicks in when you visit a site that uses 3D graphics features.

The first time you run Chrome, it checks if your GPU is blacklisted. In the unfortunate case that it is, Chrome downloads and installs the SwiftShader component in the background. After the component is installed, you can view 3D content. If you visit a 3D site before the component has finished installing, you may need to close and re-open the tab to view the site.

The performance of SwiftShader should be good enough to view simple 3D content.

To force-enable SwiftShader for testing purposes, start Chrome from the command line with the --blacklist-accelerated-compositing and --blacklist-webgl flags.

---

I may be able to provide a screencast of my own, but please see the original screenshots as they demonstrate the behaviour.

Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Submitter: I can reproduce the problem with Chrome 71.0.3578.0 (Official Build) canary (64-bit) (cohort: Clang-64) by launching the browser from the command line with --disable-gpu, which causes the browser to fall back to SwiftShader.

Unfortunately it's unclear what's going on in your application and what's going wrong.

Could you please shrink down your test case? We would really like to get to the bottom of this, but don't have the time to reduce your application to a minimized test case.

Thanks in advance for your help.

Forgot to attach the screenshot from my Windows workstation.

Deleted: MapboxSwiftShader.png 1.1 MB

	MapboxSwiftShader.png 1.1 MB View Download

The issue reproduces with --disable-gpu and --use-gl=swiftshader-webgl, but not with --use-gl=swiftshader. I've also tested with a revision of SwiftShader that's over a year old, and the results are the same.

This makes it a little less likely that it's a SwiftShader bug. Could anyone try to bisect? Happy to take a deeper look if it indicates a SwiftShader change as the culprit.

Test case is good at r500800, bad at r598222, both with --disable-gpu. Will run the bisect tomorrow.

There are at least two bugs.

1) Somewhere between r557492 and r561088 the WebGL content stopped rendering with --disable-gpu.

2) In this regression range WebGL started rendering again, but with the corrupted rendering mentioned above:

You are probably looking for a change made after 576151 (known good), but no later than 576159 (first known bad).
CHANGELOG URL:
  https://chromium.googlesource.com/chromium/src/+log/1b277b01fb2b57f68b1c7d9a63586274f1d40b8d..aabc8d68ec9ca3f787acc3f478229132f38136a6

The CL causing the regression was almost certainly:

Make WebGL use CanvasResource for submitting to OffscreenCanvas.
https://chromium.googlesource.com/chromium/src/+/00d64e2ea02ea16beb67f92ee971871402185eae

That CL has caused other bugs as well. Will have to see what new code path it is causing to be taken.

Additionally, per-revision bisects aren't working for this test; --disable-gpu is causing SwiftShader to crash the GPU process. Only the Chromium continuous builds work, and they have less precision.

Will run another bisect to track down the cause of (1). Command line I've tried:

python bisect_builds.py -a win64 -g 500800 -b 598222 --use-local-cache -- --no-first-run --disable-gpu https://s3-eu-west-1.amazonaws.com/bug-demo/google.html

This one doesn't work (with the Google-internal per-revision-bisect tool):

python bisect_builds.py -a win64 -o -g 500800 -b 598222 --use-local-cache -- --no-first-run --disable-gpu https://s3-eu-west-1.amazonaws.com/bug-demo/google.html

The change which caused the WebGL content to disappear when SwiftShader's in use is in this range:

You are probably looking for a change made after 557631 (known good), but no later than 557662 (first known bad).
CHANGELOG URL:
  https://chromium.googlesource.com/chromium/src/+log/9e191d88ec9e62953c56973e190b42aa0621f3d9..ed11385d70dda5f5b41a080bce5cb1fc854217b4

with the most likely cause being:

Check gpu acceleration before using CanvasResourceProvider for webgl in canvas
https://chromium.googlesource.com/chromium/src/+/5d46ba06f6c54ca5f0865717e554421d434b0eb8

Both changes - which caused WebGL to disappear, and to reappear but with the corrupted rendering - are related to the CanvasResourceProvider and the code paths which are taken when determining whether GPU acceleration is in use.

Continuing to investigate. Something's going wrong during a CopyTextureCHROMIUM call; trying to figure out whether the destination texture is not being properly initialized.

The problem doesn't seem to be in the CopyTextureCHROMIUM code path. I see https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/commit/a5b1261349dca6d55b7f85aaeed845f3b6d289c2 which seems to have worked around this bug.

I'm having difficulty rebuilding the project so I can try to reduce the test case. Have followed the instructions at https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/blob/master/debug/basic/readme.md and think I rebuilt everything and switched to my Google Maps JS API key, but there are the following errors in the console:

Access to XMLHttpRequest at 'https://s3-eu-west-1.amazonaws.com/tiles-for-mapbox-rendering-demo/14/8184/5497' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Submitter, is there any way you can provide a smaller test case than this entire application?
 

Thanks for looking into this.

I did have a go at making a simpler test case, which I was able to do for the earlier not-rendering-at-all bug, but I didn't manage to in this case (time constraints being the main issue). However, there is actually a simpler demo in the github src that I could have used instead (which I have now made available here - https://s3-eu-west-1.amazonaws.com/bug-demo/index.html, though unfortunately I wasn't able to verify the bug is present as I don't have access to a windows machine right now). 

As you seem to have discovered, you can also find the demos here (and you need to undo the relevant workaround commit)...
https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/tree/master/debug/basic

In terms of running it locally, it seems to work for me, so hopefully CORS is setup correctly. If not, you can just download one or two tiles for use locally. See line 53 in debug/basic/defaults.js.

In terms of a very quick overview of what is going, on there is a standard canvas, that is out-of-dom, which has the webgl context and various operations are done on that using mapbox's renderer. Sections of that canvas are then copied to visible canvases using drawImage. If you call showCanvasForDebug (https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/blob/master/src/basic/renderer.js#L440) you can bring the out-of-dom canvas into the dom and show it on the screen for debugging.
The main paint logic is initiated here - https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/blob/master/src/basic/renderer.js#L390. And the drawImage call is here https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer/blob/master/src/basic/renderer.js#L401.

As I sort of explained before, it seem like there are two separate buffers for the webgl canvas (i.e. the src canvas) since when it is visible on screen (with showCanvasForDebug) everything looks ok there, but when using drawImage from that canvas you get something that looks different (as though gl.clear was never called).

Hope that was a little helpful.  

I suppose I could try simplifying this demo a bit rather than starting from scratch, but unfortunately I can't spend a lot of time on it right now.

Manu thanks

Thanks for the tips. It would have been helpful had the test case been more self-contained.

I was able to get it running locally by tracing the network requests as you suggested, manually downloading the 6 Mapbox tiles, putting them under debug/basic (zip archive attached), and building in that directory per the instructions.

The problem's in WebGLRenderingContextBase::PaintRenderingResultsToCanvas, which is called from HTMLCanvasElement::GetSourceImageForCanvas, BaseRenderingContext2D::drawImage. If the host's ResourceProvider is discarded and recreated every frame, the rendering is correct. Still trying to figure out where along the copying paths the destination isn't being cleared correctly. The WebGL context's contents appear to be fine.

Deleted: tiles-for-mapbox-rendering-demo.zip 240 KB

tiles-for-mapbox-rendering-demo.zip 240 KB Download

baslomon@, fserb@: could I please ask for some help? I'm having difficulty figuring out a few things:

- Exactly where and how Skia is allocating the OpenGL graphics resources associated with its GPU-hosted SkSurface. Thinking they may be an EGL Pbuffer and that that may have different behavior on SwiftShader than on other platforms

- Why the CanvasResourceProvider in WebGLRenderingContextBase::CopyRenderingResultsFromDrawingBuffer is unaccelerated the first time it's called and accelerated the second, and whether that makes a difference here.

- What resource fundamentally contains the incorrect rendering results. I've tried forcing a new snapshot_paint_image_content_id_ in CanvasResourceProvider::MakeImageSnapshot, eliminating fCachedImage in SkSurface_Base::refCachedImage, and am trying more hacks - none of these had the same effect of recreating the CanvasResourceProvider every frame.

The test case is here (sorry, Google internal only):
https://drive.google.com/open?id=1042mNcTNB6r14Trda37XNpoAceTE0fbS

Unzip, serve up the directory via "python -m SimpleHTTPServer" or node's http-server, run Chrome with --disable-gpu, and navigate to e.g. localhost:8080/google.html . When the roads appear they should be messed up.

+fmalita as well

+brianosman,jvanverth for Ganesh questions.

This bug also reproduces on Linux with --disable-gpu. That probably makes it easier to debug.

Here are my patches instrumenting chromium/src and third_party/skia with debug logging to figure out what's going on.

Deleted: src.patch 19.6 KB

src.patch 19.6 KB Download

Deleted: skia.patch 3.7 KB

skia.patch 3.7 KB Download

Have this test case running on Linux in single-process mode for easier debugging with a modifiable build of https://github.com/landtechnologies/Mapbox-vector-tiles-basic-js-renderer . Will continue to see why CanvasResourceProvider::Create is getting created in two different modes for drawing WebGL to 2D canvases.

I didn't have luck yet creating a small test case from scratch demonstrating a difference between SwiftShader and using a real GPU (attempt attached, should be dropped in to sdk/tests/conformance/rendering of a checkout of https://github.com/KhronosGroup/WebGL/ ).

Any help still appreciated in (1) creating a smaller test case and (2) figuring out what's getting corrupted while drawing WebGL to a 2D canvas. The bug's pretty clearly in CanvasResourceProvider.

Deleted: draw-webgl-to-canvas-2d-repeatedly.html 2.9 KB

draw-webgl-to-canvas-2d-repeatedly.html 2.9 KB View Download

The bug is: when software compositing is enabled, this call to cc::PaintCanvas::drawImage:
https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/modules/webgl/webgl_rendering_context_base.cc#1622

isn't clearing out the previous results in the destination PaintCanvas. This might be per spec of PaintCanvas and/or SkCanvas; still investigating.

https://www.khronos.org/registry/webgl/sdk/tests/conformance/rendering/draw-webgl-to-canvas-2d-repeatedly.html is a reduced test case from https://github.com/KhronosGroup/WebGL/pull/2741 . It reproduces with either --disable-gpu (SwiftShader) or --disable-gpu-compositing, so clearly this code path is just wrong.

Looking at what happens in SkiaPaintCanvas::drawImage, it seems like that's going to use a default SkPaint, which will have kSrcOver as the blend mode. If there's alpha in the texture, that's going to blend with whatever's already there. Not sure if it's easy to wire in, but paint.setBlendMode(SkBlendMode::kSrc) would be a good test. (It's not clear why this would depend on SwiftShader vs. real GPU, though).

Thanks brianosman@. Came to the same conclusion. https://chromium-review.googlesource.com/1330890 is incoming fixing this bug.

The code path in WebGL's context is triggered only when software compositing is used, which happens when SwiftShader's turned on (via --disable-gpu, at least), or with --disable-gpu-compositing on desktop platforms. Added a test which runs in normal mode, SwiftShader, and disable-gpu-compositing which catches the bug.

Marking R-B-S for M71. This has been broken for a while, but there have been multiple reports that SwiftShader wasn't working quite correctly in WebGL applications, and this is one cause. The fix is small, safe, and well-understood, and should be merged back to beta.

Submitter: please test your web app with Chrome Beta and, ideally, Chrome Canary on a regular basis, and file any bugs you find. You can email me the bug IDs and I'll help ensure they get triaged. If we can catch issues like this earlier, we can ensure Chrome Stable will never regress. Thanks.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/93925aac015056c95114b744e3fc27594be67a68

commit 93925aac015056c95114b744e3fc27594be67a68
Author: Kenneth Russell <kbr@chromium.org>
Date: Sat Nov 10 03:40:10 2018

Fix draws of WebGL canvases to 2D contexts with software compositing.

This code path was using the wrong blend mode when drawing the WebGL
content to the intermediate cc::PaintImage which is then drawn to the
2D context.

Added a pixel test for this which fails before the fix, and passes
afterward. A WebGL conformance test has also been written which will
be rolled in later, but it would not catch the bug unless the tests
were run with SwiftShader - which they aren't currently on Chromium's
waterfall.

Bug:  894021 
Change-Id: I77a0d4877641c545159bc43b11483b80ffc74c72
Reviewed-on: https://chromium-review.googlesource.com/c/1330890
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: James Darpinian <jdarpinian@chromium.org>
Cr-Commit-Position: refs/heads/master@{#607096}
[add] https://crrev.com/93925aac015056c95114b744e3fc27594be67a68/content/test/data/gpu/pixel_repeated_webgl_to_2d.html
[modify] https://crrev.com/93925aac015056c95114b744e3fc27594be67a68/content/test/gpu/gpu_tests/pixel_expectations.py
[modify] https://crrev.com/93925aac015056c95114b744e3fc27594be67a68/content/test/gpu/gpu_tests/pixel_test_pages.py
[modify] https://crrev.com/93925aac015056c95114b744e3fc27594be67a68/third_party/blink/renderer/modules/webgl/webgl_rendering_context_base.cc

Thank you kbr@ for getting it fixed!

I haven't been able to verify the fix yet, but I will do so when I can (though I do trust your fix is good!).

Testing our app regularly on beta/canary definitely sounds like a good idea. hopefully we can make that happen.

All the best, and thanks for the great work!

Thanks @the1manson.

Verified fixed in 72.0.3608.0 (Official Build) canary (64-bit) (cohort: Clang-64) on Windows. Ran with --disable-gpu and navigated to https://s3-eu-west-1.amazonaws.com/bug-demo/google.html ; renders correctly now.

Requesting merge to M71. This is a small and well understood fix which is affecting some number of customers using WebGL on servers, and it would be better to get the fix out sooner rather than later.

This bug requires manual review: M71 has already been promoted to the beta branch, so this requires manual review
Please contact the milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Approving merge to M71 branch 3578 based on comment #36. Pls merge ASAP so we can pick it up for this week beta. Thank you.

Merged to M71 as https://chromium.googlesource.com/chromium/src/+/89f02a1fc6017dcfd518c6452f60ded64b75d394 .

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/89f02a1fc6017dcfd518c6452f60ded64b75d394

commit 89f02a1fc6017dcfd518c6452f60ded64b75d394
Author: Kenneth Russell <kbr@chromium.org>
Date: Mon Nov 12 22:15:19 2018

Fix draws of WebGL canvases to 2D contexts with software compositing.

This code path was using the wrong blend mode when drawing the WebGL
content to the intermediate cc::PaintImage which is then drawn to the
2D context.

Added a pixel test for this which fails before the fix, and passes
afterward. A WebGL conformance test has also been written which will
be rolled in later, but it would not catch the bug unless the tests
were run with SwiftShader - which they aren't currently on Chromium's
waterfall.

TBR=kbr@chromium.org

(cherry picked from commit 93925aac015056c95114b744e3fc27594be67a68)

Bug:  894021 
Change-Id: I77a0d4877641c545159bc43b11483b80ffc74c72
Reviewed-on: https://chromium-review.googlesource.com/c/1330890
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: James Darpinian <jdarpinian@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#607096}
Reviewed-on: https://chromium-review.googlesource.com/c/1332262
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#641}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}
[add] https://crrev.com/89f02a1fc6017dcfd518c6452f60ded64b75d394/content/test/data/gpu/pixel_repeated_webgl_to_2d.html
[modify] https://crrev.com/89f02a1fc6017dcfd518c6452f60ded64b75d394/content/test/gpu/gpu_tests/pixel_expectations.py
[modify] https://crrev.com/89f02a1fc6017dcfd518c6452f60ded64b75d394/content/test/gpu/gpu_tests/pixel_test_pages.py
[modify] https://crrev.com/89f02a1fc6017dcfd518c6452f60ded64b75d394/third_party/blink/renderer/modules/webgl/webgl_rendering_context_base.cc

The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/89f02a1fc6017dcfd518c6452f60ded64b75d394

Commit: 89f02a1fc6017dcfd518c6452f60ded64b75d394
Author: kbr@chromium.org
Commiter: kbr@chromium.org
Date: 2018-11-12 22:15:19 +0000 UTC

Fix draws of WebGL canvases to 2D contexts with software compositing.

This code path was using the wrong blend mode when drawing the WebGL
content to the intermediate cc::PaintImage which is then drawn to the
2D context.

Added a pixel test for this which fails before the fix, and passes
afterward. A WebGL conformance test has also been written which will
be rolled in later, but it would not catch the bug unless the tests
were run with SwiftShader - which they aren't currently on Chromium's
waterfall.

TBR=kbr@chromium.org

(cherry picked from commit 93925aac015056c95114b744e3fc27594be67a68)

Bug:  894021 
Change-Id: I77a0d4877641c545159bc43b11483b80ffc74c72
Reviewed-on: https://chromium-review.googlesource.com/c/1330890
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: James Darpinian <jdarpinian@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#607096}
Reviewed-on: https://chromium-review.googlesource.com/c/1332262
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#641}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}