Windows Debug build is unusably slow on test case from bug |
||||
Issue descriptionChrome Version: ToT OS: Win10 What steps will reproduce the problem? (1) Build Chrome on Windows in Debug mode using the following GN args: use_goma = true is_debug = true is_component_build = false enable_nacl = false dcheck_always_on = true ffmpeg_branding = "Chrome" proprietary_codecs = true v8_enable_debugging_features = false (2) Download the test case from https://bugs.chromium.org/p/chromium/issues/detail?id=894021#c22 and serve it up locally (Google internal only, sorry) (3) Navigate to localhost:8080/google.html (note: --disable-gpu is *not* necessary - this happens even without SwiftShader) What is the expected result? Expect the map to be displayed with roads overlaid. What happens instead? The map operates extremely slowly - it looks like 1 frame every few seconds. The satellite tiles of the map are displayed reasonably quickly but the icon overlays and vector roads don't display reliably. I've waited 1-2 minutes for them to display and they do so only intermittently. Have tried gathering a profile in Visual Studio and it shows nothing useful; checking the box to view "external sources" doesn't complete in a reasonable timeframe, so it's impossible to know where the renderer process is spending its time. Traces (two are attached) may implicate microtask execution; have seen one trace where this appeared to take 5+ seconds. Have attempted to turn off any of V8's debugging checks via the GN arg v8_enable_debugging_features=false but that didn't speed things up. A debug build would be extremely useful in order to diagnose Issue 894021 but I'm continuing to add logging for it and also attempting to debug with optimized builds, so am not blocking that bug on this one. Any help getting this test to operate reasonably well in a build where I can reliably set breakpoints and step through Blink's and Skia's code (which isn't working well with an optimized build - seeing all sorts of weird issues where Visual Studio gets confused about the location of the program counter) would be greatly appreciated. Thanks in advance.
,
Nov 8
Here are two traces. The second one (slow-map-2) shows a lot of slow operations in the compositor (LayerTreeHost::UpdateLayers::BuildPropertyTrees, ProxyMain::BeginMainFrame::Commit) as well as some slow JavaScript execution (V8.Execute). Some slow resource loading happens earlier inside microtask execution. CC'ing more people for any help. Note that I routinely build with is_debug=false dcheck_always_on=true, and that performs OK, but in this case I really need is_debug=true to work and not perform pathologically slowly.
,
Nov 8
@adamk I haven't set that. Let me do so and rebuild; will report back.
,
Nov 8
Er, setting it to true. Got that backwards. v8_optimized_debug = true I thought we did that by default for Chromium builds, but my gn-fu is not as strong as my gyp-fu was.
,
Nov 8
Yes, it looks like it's on by default in Chromium Debug builds. Changing it did not force a rebuild.
,
Nov 9
The debug build on Linux seems OK and reproduces the problem. I'll probably continue debugging there. Downgrading to P2, though someone really should look at the Windows debug build because the performance there seems unworkably slow.
,
Nov 9
Any recommended technique for serving it up locally? python -m SimpleHTTPServer isn't working and if I navigate to the google.html file I don't get the overlays.
,
Nov 9
Ah, sorry, I hardwired a port to 8080 in one of the configuration files. Can you install node's http-server and try that? It's more reliable than Python's anyway.
,
Nov 10
TL;DR set this gn arg:
enable_iterator_debugging=false
That will make debug builds complete about 50 s faster and will make this test case run orders of magnitude faster.
kbr@ gave me this batch file:
http.bat:
python.exe -c "import BaseHTTPServer as bhs, SimpleHTTPServer as shs; bhs.HTTPServer(('127.0.0.1', %1), shs.SimpleHTTPRequestHandler).serve_forever()"
and when I ran that with "http.bat 8080" I could then connect to localhost:8080/google.html.
With that I could reproduce the bug. The renderer process was trying to use all 20/40 cores/threads on my machine. I grabbed an ETW trace with UIforETW and it showed that roughly 98.3% of CPU time was being spent acquiring and releasing critical sections. A single critical section perhaps. The lock(s) were being acquired by msvcp140d.dll so I guessed that this was probably a quirk of Microsoft's iterator debugging system. So I turned it off and it looks like this has solved things.
My Chrome-build symbols refused to load for some reason so I don't have as much data as I would like, but the solution seems to work.
The build speedup is because generating snapshot_blob.bin is greatly slowed (from ~12 s to 60-70 s) by iterator debugging. I may change the default setting for iterator debugging to false because I don't remember when it last found a bug.
,
Nov 10
Thank you Bruce for tracking this down. *Please* change the default for enable_iterator_debugging to false. The Windows debug build is unusably slow right now - especially compared to other platforms. Also could we please document that build flag somewhere, and provide a win_debugging.md in src/docs ?
,
Nov 10
,
Nov 10
I filed crbug.com/904034 to track changing the default value of this flag.
,
Nov 10
|
||||
►
Sign in to add a comment |
||||
Comment 1 by adamk@chromium.org
, Nov 8