zmo@: unfortunately I think this is one of your two CLs that landed in that range:

Hook up driver bug workaround's platform dependent disabled extensions.
https://chromium-review.googlesource.com/614780

Switch GpuDataManager to use GpuFeatureInfo returned from GPU process
https://chromium-review.googlesource.com/615750

Of the two I think the one most likely to have caused this is https://chromium-review.googlesource.com/614780 (that was the trickier one). Talking with zmo@ now about this.

https://chromium-review.googlesource.com/614780 is the culprit.

g_driver_glx.InitializeExtensionBindings() call gxf::GetXDisplay() and glXQueryExtensonsString().

This used to happen before gfx::InitializeThreadedX11(). Now we move it after GPU driver bug workarounds decisions. By doing that, we triggered a sandbox threads violation on a Linux NVidia bot. So we need to create a display and call a glX function before gfx::InitializeThreadedX11() as a temporary hack.

The affected platform is NVidia Quadro P400 with driver 378.13.

We can't reproduce on NVidia Quadro 600 with driver 367.57.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2

commit 8f2b7886b0a8d4416b1c0891804ea8f6128cbab2
Author: Zhenyao Mo <zmo@chromium.org>
Date: Fri Aug 18 23:36:27 2017

[Reland] Hook up driver bug workaround's platform dependent disabled extensions.

Originally reviewed on https://chromium-review.googlesource.com/614780

BUG= 744658 , 756885 
TEST=gl_unittests
R=piman@chromium.org,kbr@chromium.org

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: Idf356d0718c50b3721f38059f2c7f2ae1392a1a2
Reviewed-on: https://chromium-review.googlesource.com/621885
Reviewed-by: Antoine Labour <piman@chromium.org>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#495745}
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/gpu/ipc/service/gpu_init.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/egl_api_unittest.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/generate_bindings.py
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_api_unittest.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_bindings_autogen_egl.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_bindings_autogen_gl.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_bindings_autogen_glx.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_bindings_autogen_osmesa.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_bindings_autogen_wgl.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_context.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_context.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_egl_api_implementation.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_egl_api_implementation.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_enums_implementation_autogen.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_gl_api_implementation.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_gl_api_implementation.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_glx_api_implementation.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_glx_api_implementation.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_egl.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_egl.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_glx.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_glx.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_wgl.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_surface_wgl.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_wgl_api_implementation.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/gl_wgl_api_implementation.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/glx_api_unittest.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory.h
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_android.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_fuchsia.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_mac.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_ozone.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_win.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/init/gl_factory_x11.cc
[modify] https://crrev.com/8f2b7886b0a8d4416b1c0891804ea8f6128cbab2/ui/gl/wgl_api_unittest.cc

Took a look at libX11 threading. What I found was that XInitThreads does a couple of high-level things:
1- create a couple of global locks for things like the error functions or some internal state. These locks are always used by libX11 after XInitThreads was called
2- create a hook to setup per-Display lock (for XLockDisplay and friends, including internal locking within X11 functions). Any Display created after XInitThreads will have its lock functional and used, but any Display created before XInitThreads will not have its lock setup, meaning XLockDisplay is a noop.

So my high-level reading is that as the code is, the main display (returned by gfx::GetXDisplay, before we call XInitThreads) will never lock its per-Display lock, but other displays (e.g. for the SGIVideoSyncVSyncProvider) will - FWIW. The global locks will still be taken either way.

So that means that, while we are bending the rules, provided we only use the main display on the main thread, we should actually be "safe".


I suspect the 378.13 NVIDIA driver detects the presence of the global libX11 lock when initializing GLX (on the first glX call on that display) and decide whether to create extra threads if it is present. I will see if I can reproduce this locally.

Data point: could not repro with driver 375.66 on Quadro K2200.

+jln/jorgelo is SECCOMP_FILTER_FLAG_TSYNC an option here?

It definitely would be. We haven't used TSYNC so far in Chrome but it would alleviate most of these threading issues.

Yes, TYSNC might help, bit it's definitely a bit of a hack and it's quite dangerous. By nature, you want the sandbox to be on as early as possible and having a number of threads running code while you enable the sandbox can go wrong in many ways, including breaking security, or creating crashes in ways that would be difficult to reproduce.

I explained more about the dangers here: https://bugs.chromium.org/p/chromium/issues/detail?id=438893#c1

Also, even if everything is "ok" today (the other threads don't acess resources in a way that would interact with the sandbox), it could regress easily without us ever knowing about it: new threads could be created before we enable the sandbox, they may do "wrong things" and we won't know when that happens because there will be no CHECK() to tell us.

If there is any reasonable way to avoid TSYNC, we should avoid it.

(Note: technically we always use TSYNC when available. However we do refuse to start the sandbox if there is more than one thread in the current process)

@#9: right now the hacks we have to resort to to prevent the driver from creating threads before we have a chance to turn on the sandbox are pretty bad (see #5) - they're a violation of the API rules (and mutual exclusion - I don't like playing with that) and the same argument relative to things "working" now vs the future is valid too. So rock, meet hard place.
Worst being that we need to make the decision to use this workaround before we can even get any info about the driver/GPU - IOW, it's always on right now - so every driver suffers from this risk (as opposed to only the ones that would create threads).

I repro'ed on 378.13, same GPU (Quadro K2200). I've had crashes though, so I'm not sure that driver is very stable. I will also check more recent drivers.

For the record, here are the stacks that creates internal threads in the driver before we start the sandbox (there are 2):

#0  0x00007ffa9328cac0 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:54
#1  0x00007ffaacafe22a in create_thread (pd=pd@entry=0x7ffa82bf1700, attr=attr@entry=0x7ffc8f2d5f00, stopped_start=<optimized out>, 
    stopped_start@entry=false, stackaddr=<optimized out>, thread_ran=0x7ffc8f2d5eff) at ../sysdeps/unix/sysv/linux/createthread.c:102
#2  0x00007ffaacaffc53 in __pthread_create_2_1 (newthread=<optimized out>, attr=<optimized out>, start_routine=<optimized out>, arg=<optimized out>) at pthread_create.c:679
#3  0x00007ffa84ca1ebc in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#4  0x00007ffaacb06739 in __pthread_once_slow (once_control=0x7ffa84f27d68, init_routine=0x7ffa84ca1e70) at pthread_once.c:116
#5  0x00007ffa84c9fb33 in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#6  0x00007ffa839bccf2 in  () at /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.378.13
#7  0x00007ffa83c6064b in  () at /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.378.13
#8  0x00007ffa84c7901f in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#9  0x00007ffa84c3ebf9 in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#10 0x00007ffaacd2486a in call_init (l=0x36b7433b1f20, argc=argc@entry=13, argv=argv@entry=0x7ffc8f2dbda8, env=env@entry=0x36b743228b20) at dl-init.c:58
#11 0x00007ffaacd249bb in call_init (env=0x36b743228b20, argv=0x7ffc8f2dbda8, argc=13, l=<optimized out>) at dl-init.c:30
#12 0x00007ffaacd249bb in _dl_init (main_map=main_map@entry=0x36b7433b1f20, argc=13, argv=0x7ffc8f2dbda8, env=0x36b743228b20) at dl-init.c:120
#13 0x00007ffaacd28f38 in dl_open_worker (a=a@entry=0x7ffc8f2d7d90) at dl-open.c:575
#14 0x00007ffaacd24754 in _dl_catch_error (objname=objname@entry=0x7ffc8f2d7d80, errstring=errstring@entry=0x7ffc8f2d7d88, mallocedp=mallocedp@entry=0x7ffc8f2d7d7f, operate=operate@entry=0x7ffaacd28b50 <dl_open_worker>, args=args@entry=0x7ffc8f2d7d90) at dl-error.c:187
#15 0x00007ffaacd286e9 in _dl_open (file=0x36b74317b3e0 "libGLX_nvidia.so.0", mode=-2147483647, caller_dlopen=0x7ffa8522046c, nsid=-2, argc=<optimized out>, argv=<optimized out>, env=0x36b743228b20)
    at dl-open.c:660
#16 0x00007ffa95f86ee9 in dlopen_doit (a=a@entry=0x7ffc8f2d7fc0) at dlopen.c:66
#17 0x00007ffaacd24754 in _dl_catch_error (objname=0x36b74329f390, errstring=0x36b74329f398, mallocedp=0x36b74329f388, operate=0x7ffa95f86e90 <dlopen_doit>, args=0x7ffc8f2d7fc0) at dl-error.c:187
#18 0x00007ffa95f87531 in _dlerror_run (operate=operate@entry=0x7ffa95f86e90 <dlopen_doit>, args=args@entry=0x7ffc8f2d7fc0) at dlerror.c:163
#19 0x00007ffa95f86f82 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#20 0x00007ffa8522046c in  () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#21 0x00007ffa8522127c in  () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#22 0x00007ffa85219fe8 in glXQueryExtensionsString () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#23 0x00007ffaa33019b5 in (anonymous namespace)::GLXApiBase::glXQueryExtensionsStringFn(Display*, int) (this=0x36b74326ce20, dpy=0x36b743277820, screen=0) at ../../ui/gl/gl_bindings_autogen_glx.cc:410
#24 0x00007ffaa3313145 in (anonymous namespace)::RealGLXApi::glXQueryExtensionsStringFn(Display*, int) (this=0x36b74326ce20, dpy=0x36b743277820, screen=0) at ../../ui/gl/gl_glx_api_implementation.cc:94
#25 0x00007ffaa3315ec6 in (anonymous namespace)::GLSurfaceGLX::GetGLXExtensions() () at ../../ui/gl/gl_surface_glx.cc:488
#26 0x00007ffaa3315e81 in (anonymous namespace)::GLSurfaceGLX::HasGLXExtension(char const*) (name=0x7ffaa333af1b "GLX_EXT_visual_rating") at ../../ui/gl/gl_surface_glx.cc:493
#27 0x00007ffaa331e57b in (anonymous namespace)::GLVisualPickerGLX::GLVisualPickerGLX() (this=0x36b7431fedf0) at ../../ui/gl/gl_visual_picker_glx.cc:149
#28 0x00007ffaa331efd0 in (anonymous namespace)::DefaultSingletonTraits<gl::GLVisualPickerGLX>::New() () at ../../base/memory/singleton.h:54
#29 0x00007ffaa331ed46 in (anonymous namespace)::Singleton<gl::GLVisualPickerGLX, base::DefaultSingletonTraits<gl::GLVisualPickerGLX>, gl::GLVisualPickerGLX>::get() () at ../../base/memory/singleton.h:258
#30 0x00007ffaa331cd19 in (anonymous namespace)::GLVisualPickerGLX::GetInstance() () at ../../ui/gl/gl_visual_picker_glx.cc:30
#31 0x00007ffaa33156ea in (anonymous namespace)::GLSurfaceGLX::InitializeOneOff() () at ../../ui/gl/gl_surface_glx.cc:432
#32 0x00007ffa91c5f823 in (anonymous namespace)::(anonymous namespace)::InitializeGLOneOffPlatform() () at ../../ui/gl/init/gl_initializer_x11.cc:152
#33 0x00007ffa91c580bd in (anonymous namespace)::(anonymous namespace)::InitializeGLOneOffImplementation(enum (anonymous namespace)::GLImplementation, bool, bool, bool, bool) (impl=(anonymous namespace)::kGLImplementationDesktopGL, fallback_to_software_gl=false, gpu_service_logging=false, disable_gl_drawing=false, init_extensions=false) at ../../ui/gl/init/gl_factory.cc:81
#34 0x00007ffa91c57ee2 in (anonymous namespace)::(anonymous namespace)::(anonymous namespace)::InitializeGLOneOffHelper(bool) (init_extensions=false) at ../../ui/gl/init/gl_factory.cc:58
#35 0x00007ffa91c5801b in (anonymous namespace)::(anonymous namespace)::InitializeGLNoExtensionsOneOff() () at ../../ui/gl/init/gl_factory.cc:72
#36 0x00007ffaa3846db8 in (anonymous namespace)::GpuInit::InitializeAndStartSandbox((anonymous namespace)::CommandLine const&, bool) (this=0x7ffc8f2dac50, command_line=..., in_process_gpu=false)
    at ../../gpu/ipc/service/gpu_init.cc:243
#37 0x00007ffaa4a83b4d in (anonymous namespace)::GpuMain((anonymous namespace)::MainFunctionParams const&) (parameters=...) at ../../content/gpu/gpu_main.cc:271
#38 0x00007ffaa7827e54 in (anonymous namespace)::RunNamedProcessTypeMain((anonymous namespace)::(anonymous namespace)::string const&, (anonymous namespace)::MainFunctionParams const&, (anonymous namespace)::ContentMainDelegate*) (process_type=..., main_function_params=..., delegate=0x7ffc8f2dbbd8) at ../../content/app/content_main_runner.cc:426
#39 0x00007ffaa782ab5e in (anonymous namespace)::ContentMainRunnerImpl::Run() (this=0x36b7432955c0) at ../../content/app/content_main_runner.cc:709
#40 0x00007ffaa782562d in (anonymous namespace)::ContentServiceManagerMainDelegate::RunEmbedderProcess() (this=0x7ffc8f2dbb60) at ../../content/app/content_service_manager_main_delegate.cc:51
#41 0x00007ffaacda1d05 in (anonymous namespace)::Main((anonymous namespace)::MainParams const&) (params=...) at ../../services/service_manager/embedder/main.cc:469
#42 0x00007ffaa7826ccf in (anonymous namespace)::ContentMain((anonymous namespace)::ContentMainParams const&) (params=...) at ../../content/app/content_main.cc:19
#43 0x00005592d96b63ae in ChromeMain(int, char const**) (argc=13, argv=0x7ffc8f2dbda8) at ../../chrome/app/chrome_main.cc:122
#44 0x00005592d96b62c2 in main(int, char const**) (argc=13, argv=0x7ffc8f2dbda8) at ../../chrome/app/chrome_exe_main_aura.cc:17


#0  0x00007ffa9328cac0 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:54
#1  0x00007ffaacafe22a in create_thread (pd=pd@entry=0x7ffa82bf1700, attr=attr@entry=0x7ffc8f2d5fa0, stopped_start=<optimized out>, 
    stopped_start@entry=false, stackaddr=<optimized out>, thread_ran=0x7ffc8f2d5f9f) at ../sysdeps/unix/sysv/linux/createthread.c:102
#2  0x00007ffaacaffc53 in __pthread_create_2_1 (newthread=<optimized out>, attr=<optimized out>, start_routine=<optimized out>, arg=<optimized out>) at pthread_create.c:679
#3  0x00007ffa84ca10c2 in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#4  0x00007ffa839bca6f in  () at /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.378.13
#5  0x00007ffa83c6064b in  () at /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.378.13
#6  0x00007ffa84c7901f in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#7  0x00007ffa84c3ebf9 in  () at /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#8  0x00007ffaacd2486a in call_init (l=0x36b7433b1f20, argc=argc@entry=13, argv=argv@entry=0x7ffc8f2dbda8, env=env@entry=0x36b743228b20) at dl-init.c:58
#9  0x00007ffaacd249bb in call_init (env=0x36b743228b20, argv=0x7ffc8f2dbda8, argc=13, l=<optimized out>) at dl-init.c:30
#10 0x00007ffaacd249bb in _dl_init (main_map=main_map@entry=0x36b7433b1f20, argc=13, argv=0x7ffc8f2dbda8, env=0x36b743228b20) at dl-init.c:120
#11 0x00007ffaacd28f38 in dl_open_worker (a=a@entry=0x7ffc8f2d7d90) at dl-open.c:575
#12 0x00007ffaacd24754 in _dl_catch_error (objname=objname@entry=0x7ffc8f2d7d80, errstring=errstring@entry=0x7ffc8f2d7d88, mallocedp=mallocedp@entry=0x7ffc8f2d7d7f, operate=operate@entry=0x7ffaacd28b50 <dl_open_worker>, args=args@entry=0x7ffc8f2d7d90) at dl-error.c:187
#13 0x00007ffaacd286e9 in _dl_open (file=0x36b74317b3e0 "libGLX_nvidia.so.0", mode=-2147483647, caller_dlopen=0x7ffa8522046c, nsid=-2, argc=<optimized out>, argv=<optimized out>, env=0x36b743228b20)
    at dl-open.c:660
#14 0x00007ffa95f86ee9 in dlopen_doit (a=a@entry=0x7ffc8f2d7fc0) at dlopen.c:66
#15 0x00007ffaacd24754 in _dl_catch_error (objname=0x36b74329f390, errstring=0x36b74329f398, mallocedp=0x36b74329f388, operate=0x7ffa95f86e90 <dlopen_doit>, args=0x7ffc8f2d7fc0) at dl-error.c:187
#16 0x00007ffa95f87531 in _dlerror_run (operate=operate@entry=0x7ffa95f86e90 <dlopen_doit>, args=args@entry=0x7ffc8f2d7fc0) at dlerror.c:163
#17 0x00007ffa95f86f82 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#18 0x00007ffa8522046c in  () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#19 0x00007ffa8522127c in  () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#20 0x00007ffa85219fe8 in glXQueryExtensionsString () at /usr/lib/x86_64-linux-gnu/libGLX.so.0
#21 0x00007ffaa33019b5 in (anonymous namespace)::GLXApiBase::glXQueryExtensionsStringFn(Display*, int) (this=0x36b74326ce20, dpy=0x36b743277820, screen=0) at ../../ui/gl/gl_bindings_autogen_glx.cc:410
#22 0x00007ffaa3313145 in (anonymous namespace)::RealGLXApi::glXQueryExtensionsStringFn(Display*, int) (this=0x36b74326ce20, dpy=0x36b743277820, screen=0) at ../../ui/gl/gl_glx_api_implementation.cc:94
#23 0x00007ffaa3315ec6 in (anonymous namespace)::GLSurfaceGLX::GetGLXExtensions() () at ../../ui/gl/gl_surface_glx.cc:488
#24 0x00007ffaa3315e81 in (anonymous namespace)::GLSurfaceGLX::HasGLXExtension(char const*) (name=0x7ffaa333af1b "GLX_EXT_visual_rating") at ../../ui/gl/gl_surface_glx.cc:493
#25 0x00007ffaa331e57b in (anonymous namespace)::GLVisualPickerGLX::GLVisualPickerGLX() (this=0x36b7431fedf0) at ../../ui/gl/gl_visual_picker_glx.cc:149
#26 0x00007ffaa331efd0 in (anonymous namespace)::DefaultSingletonTraits<gl::GLVisualPickerGLX>::New() () at ../../base/memory/singleton.h:54
#27 0x00007ffaa331ed46 in (anonymous namespace)::Singleton<gl::GLVisualPickerGLX, base::DefaultSingletonTraits<gl::GLVisualPickerGLX>, gl::GLVisualPickerGLX>::get() () at ../../base/memory/singleton.h:258
#28 0x00007ffaa331cd19 in (anonymous namespace)::GLVisualPickerGLX::GetInstance() () at ../../ui/gl/gl_visual_picker_glx.cc:30
#29 0x00007ffaa33156ea in (anonymous namespace)::GLSurfaceGLX::InitializeOneOff() () at ../../ui/gl/gl_surface_glx.cc:432
#30 0x00007ffa91c5f823 in (anonymous namespace)::(anonymous namespace)::InitializeGLOneOffPlatform() () at ../../ui/gl/init/gl_initializer_x11.cc:152
#31 0x00007ffa91c580bd in (anonymous namespace)::(anonymous namespace)::InitializeGLOneOffImplementation(enum (anonymous namespace)::GLImplementation, bool, bool, bool, bool) (impl=(anonymous namespace)::kGLImplementationDesktopGL, fallback_to_software_gl=false, gpu_service_logging=false, disable_gl_drawing=false, init_extensions=false) at ../../ui/gl/init/gl_factory.cc:81
#32 0x00007ffa91c57ee2 in (anonymous namespace)::(anonymous namespace)::(anonymous namespace)::InitializeGLOneOffHelper(bool) (init_extensions=false) at ../../ui/gl/init/gl_factory.cc:58
#33 0x00007ffa91c5801b in (anonymous namespace)::(anonymous namespace)::InitializeGLNoExtensionsOneOff() () at ../../ui/gl/init/gl_factory.cc:72
#34 0x00007ffaa3846db8 in (anonymous namespace)::GpuInit::InitializeAndStartSandbox((anonymous namespace)::CommandLine const&, bool) (this=0x7ffc8f2dac50, command_line=..., in_process_gpu=false)
    at ../../gpu/ipc/service/gpu_init.cc:243
#35 0x00007ffaa4a83b4d in (anonymous namespace)::GpuMain((anonymous namespace)::MainFunctionParams const&) (parameters=...) at ../../content/gpu/gpu_main.cc:271
#36 0x00007ffaa7827e54 in (anonymous namespace)::RunNamedProcessTypeMain((anonymous namespace)::(anonymous namespace)::string const&, (anonymous namespace)::MainFunctionParams const&, (anonymous namespace)::ContentMainDelegate*) (process_type=..., main_function_params=..., delegate=0x7ffc8f2dbbd8) at ../../content/app/content_main_runner.cc:426
#37 0x00007ffaa782ab5e in (anonymous namespace)::ContentMainRunnerImpl::Run() (this=0x36b7432955c0) at ../../content/app/content_main_runner.cc:709
#38 0x00007ffaa782562d in (anonymous namespace)::ContentServiceManagerMainDelegate::RunEmbedderProcess() (this=0x7ffc8f2dbb60) at ../../content/app/content_service_manager_main_delegate.cc:51
#39 0x00007ffaacda1d05 in (anonymous namespace)::Main((anonymous namespace)::MainParams const&) (params=...) at ../../services/service_manager/embedder/main.cc:469
#40 0x00007ffaa7826ccf in (anonymous namespace)::ContentMain((anonymous namespace)::ContentMainParams const&) (params=...) at ../../content/app/content_main.cc:19
#41 0x00005592d96b63ae in ChromeMain(int, char const**) (argc=13, argv=0x7ffc8f2dbda8) at ../../chrome/app/chrome_main.cc:122
#42 0x00005592d96b62c2 in main(int, char const**) (argc=13, argv=0x7ffc8f2dbda8) at ../../chrome/app/chrome_exe_main_aura.cc:17


They indeed come from the first glX call (glXQueryExtensionsString in this case) which triggers loading internal libraries. I re-confirmed that not calling XInitThreads prevent these internal threads from being created.

It looks like 381.22 (NVIDIA's "short-lived branch") does *not* create threads before the sandbox.

And 384.69 (NVIDIA's "long-lived branch) either.

So it looks like the more stable drivers would be safe if we fixed calling XInitThreads early. My suggestion is to update our bots that have a driver that causes problems to something more stable, and restore the proper behavior in Chrome. It means bad drivers will have the sandbox disabled (unless we start relying on TSYNC), but I think it's the best solution, at least for now.

Filed  Issue 760276  about upgrading the Linux experimental NVIDIA bot on the chromium.gpu.fyi waterfall to 384.69. If that bot's green after the upgrade then we'll request that the rest of the fleet be updated.

Thanks!

I'm adding an histogram so that we can check in the wild if changing the initialization order will dramatically change our ability to sandbox, but it's not going to make it to a dev channel before the branch, so I will wait until we have collected baseline data before changing the initialization order.

I'll bring the TSYNC issue up during our next Chrome security architecture sync and see how people feel about it.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3e3004db8cdd06fd1d03970872a318684561b53d

commit 3e3004db8cdd06fd1d03970872a318684561b53d
Author: Antoine Labour <piman@chromium.org>
Date: Tue Aug 29 22:04:07 2017

Add GPU.Sandbox.InitializedSuccessfully histogram

This allows verifying assumptions in the wild about how drivers behave,
and catch regressions.

Bug:  756885 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I84e42b07194b6bd4eb3db325d708eca067f67729
Reviewed-on: https://chromium-review.googlesource.com/642172
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Commit-Queue: Antoine Labour <piman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498248}
[modify] https://crrev.com/3e3004db8cdd06fd1d03970872a318684561b53d/gpu/ipc/service/gpu_init.cc
[modify] https://crrev.com/3e3004db8cdd06fd1d03970872a318684561b53d/tools/metrics/histograms/histograms.xml

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ac2af5c3db9859d18266102fdec36e1ac40efe2e

commit ac2af5c3db9859d18266102fdec36e1ac40efe2e
Author: Antoine Labour <piman@chromium.org>
Date: Mon Sep 11 23:07:26 2017

Remove XInitThreads workaround

Properly calling XInitThreads before creating the main thread Display
was causing a particular driver to create threads in the first GLX call,
preventing sandbox initialization. So we had a workaround to call
XInitThreads after the Display is created, which is against the spec but
worked if you squint enough.

As this was a non-stable driver, we instead do proper initialization and
this particular driver will have the sandboxed disabled. We can monitor
GPU.Sandbox.InitializedSuccessfully to check behavior in the wild.

Bug:  756885 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I0a39f6e3d2e085c2c59ff654a8b1c60b00e45752
Reviewed-on: https://chromium-review.googlesource.com/660811
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: Antoine Labour <piman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501079}
[modify] https://crrev.com/ac2af5c3db9859d18266102fdec36e1ac40efe2e/ui/gl/gl_surface_glx.cc

No regression on the success numbers, so marking fixed.