content_browsertests failing on chromium.memory/Linux TSan Tests |
||||||
Issue descriptioncontent_browsertests failing on chromium.memory/Linux TSan Tests Builders failed on: - Linux TSan Tests: https://build.chromium.org/p/chromium.memory/builders/Linux%20TSan%20Tests This failure has been happening with some regularity since this morning: https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/5428 https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/5424 https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/5423 https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests/builds/5422 I think this might be related to https://codereview.chromium.org/2861163002 -- it appears to be a race to set the environment vars, and this patch (which sets env vars) landed at the same time that TSan started noticing the race. I'm going to revert speculatively, and it can be relanded if the failures persist.
,
May 9 2017
Please consider adding the actual race report to the bug.
In this case e.g.:
[ RUN ] WorkerTest.WebSocketSharedWorker
[15144:15144:0508/105153.728225:9062930216:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected
[15144:15144:0508/105153.728509:9062930448:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected
Xlib: extension "RANDR" missing on display ":99".
==================
WARNING: ThreadSanitizer: data race (pid=15144)
Write of size 8 at 0x7b3c00080f70 by main thread:
#0 realloc <null> (content_browsertests+0x4be1e9)
#1 __add_to_environ /build/eglibc-MjiXCM/eglibc-2.19/stdlib/setenv.c:142 (libc.so.6+0x3ba1f)
#2 AppendToPythonPath(base::FilePath const&) net/test/python_utils.cc:43:10 (content_browsertests+0x28dac28)
#3 net::LocalTestServer::SetPythonPath() const net/test/spawned_test_server/local_test_server.cc:177:3 (content_browsertests+0x28d0780)
#4 StartInBackground net/test/spawned_test_server/local_test_server.cc:105:8 (content_browsertests+0x28d068d)
#5 net::LocalTestServer::Start() net/test/spawned_test_server/local_test_server.cc:94 (content_browsertests+0x28d068d)
#6 content::WorkerTest_WebSocketSharedWorker_Test::RunTestOnMainThread() content/browser/shared_worker/worker_browsertest.cc:215:3 (content_browsertests+0x9074a0)
#7 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() content/public/test/browser_test_base.cc:309:5 (content_browsertests+0x2704af0)
#8 Invoke<content::BrowserTestBase *> base/bind_internal.h:214:12 (content_browsertests+0x2705a55)
#9 MakeItSo<void (content::BrowserTestBase::*const &)(), content::BrowserTestBase *> base/bind_internal.h:285 (content_browsertests+0x2705a55)
#10 RunImpl<void (content::BrowserTestBase::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > &, 0> base/bind_internal.h:361 (content_browsertests+0x2705a55)
#11 base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:339 (content_browsertests+0x2705a55)
#12 Run base/callback.h:80:12 (content_browsertests+0x27978c1)
#13 content::ShellBrowserMainParts::PreMainMessageLoopRun() content/shell/browser/shell_browser_main_parts.cc:196 (content_browsertests+0x27978c1)
#14 content::BrowserMainLoop::PreMainMessageLoopRun() content/browser/browser_main_loop.cc:1165:13 (content_browsertests+0x1f4bbd0)
#15 Invoke<content::BrowserMainLoop *> base/bind_internal.h:214:12 (content_browsertests+0x1f4ed95)
#16 MakeItSo<int (content::BrowserMainLoop::*const &)(), content::BrowserMainLoop *> base/bind_internal.h:285 (content_browsertests+0x1f4ed95)
#17 RunImpl<int (content::BrowserMainLoop::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > &, 0> base/bind_internal.h:361 (content_browsertests+0x1f4ed95)
#18 base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:339 (content_browsertests+0x1f4ed95)
#19 Run base/callback.h:80:12 (content_browsertests+0x242b55c)
#20 content::StartupTaskRunner::RunAllTasksNow() content/browser/startup_task_runner.cc:45 (content_browsertests+0x242b55c)
#21 content::BrowserMainLoop::CreateStartupTasks() content/browser/browser_main_loop.cc:959:25 (content_browsertests+0x1f49866)
#22 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) content/browser/browser_main_runner.cc:126:17 (content_browsertests+0x1f4f6f6)
#23 ShellBrowserMain(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserMainRunner, std::__1::default_delete<content::BrowserMainRunner> > const&) content/shell/browser/shell_browser_main.cc:23:32 (content_browsertests+0x2772747)
#24 content::ShellMainDelegate::RunProcess(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, content::MainFunctionParams const&) content/shell/app/shell_main_delegate.cc:296:16 (content_browsertests+0x2764883)
#25 content::RunNamedProcessTypeMain(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, content::MainFunctionParams const&, content::ContentMainDelegate*) content/app/content_main_runner.cc:399:35 (content_browsertests+0x1e3743b)
#26 content::ContentMainRunnerImpl::Run() content/app/content_main_runner.cc:705:12 (content_browsertests+0x1e38064)
#27 content::ContentServiceManagerMainDelegate::RunEmbedderProcess() content/app/content_service_manager_main_delegate.cc:41:32 (content_browsertests+0xfdd9bf)
#28 service_manager::Main(service_manager::MainParams const&) services/service_manager/embedder/main.cc:468:29 (content_browsertests+0x40e2615)
#29 content::ContentMain(content::ContentMainParams const&) content/app/content_main.cc:19:10 (content_browsertests+0xfde37b)
#30 content::BrowserTestBase::SetUp() content/public/test/browser_test_base.cc:265:3 (content_browsertests+0x270475f)
#31 content::ContentBrowserTest::SetUp() content/public/test/content_browser_test.cc:89:20 (content_browsertests+0x26f76c6)
#32 HandleExceptionsInMethodIfSupported<testing::Test, void> third_party/googletest/src/googletest/src/gtest.cc:2458:12 (content_browsertests+0x26102b6)
#33 testing::Test::Run() third_party/googletest/src/googletest/src/gtest.cc:2470 (content_browsertests+0x26102b6)
#34 testing::TestInfo::Run() third_party/googletest/src/googletest/src/gtest.cc:2656:11 (content_browsertests+0x261115d)
#35 testing::TestCase::Run() third_party/googletest/src/googletest/src/gtest.cc:2774:28 (content_browsertests+0x26118a6)
#36 testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/src/googletest/src/gtest.cc:4647:43 (content_browsertests+0x261c516)
#37 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/src/googletest/src/gtest.cc:2458:12 (content_browsertests+0x261be44)
#38 testing::UnitTest::Run() third_party/googletest/src/googletest/src/gtest.cc:4255 (content_browsertests+0x261be44)
#39 RUN_ALL_TESTS third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46 (content_browsertests+0x2738d97)
#40 base::TestSuite::Run() base/test/test_suite.cc:271 (content_browsertests+0x2738d97)
#41 content::ContentTestLauncherDelegate::RunTestSuite(int, char**) content/test/content_test_launcher.cc:105:48 (content_browsertests+0x26fc55b)
#42 content::LaunchTests(content::TestLauncherDelegate*, int, int, char**) content/public/test/test_launcher.cc:520:31 (content_browsertests+0x272346d)
#43 main content/test/content_test_launcher.cc:131:10 (content_browsertests+0x26fc4e2)
Previous read of size 8 at 0x7b3c00080f70 by thread T17 (mutexes: write M254588807855316976):
#0 memcpy <null> (content_browsertests+0x4c831d)
#1 <null> <null> (libfreeblpriv3.so+0x15144)
#2 crypto::EnsureNSSInit() crypto/nss_util.cc:864:19 (content_browsertests+0x35bf671)
#3 net::CertDatabase::CertDatabase() net/cert/cert_database_nss.cc:14:3 (content_browsertests+0x388d1f0)
#4 New base/memory/singleton.h:55:16 (content_browsertests+0x3869937)
#5 get base/memory/singleton.h:260 (content_browsertests+0x3869937)
#6 net::CertDatabase::GetInstance() net/cert/cert_database.cc:16 (content_browsertests+0x3869937)
#7 net::CachingCertVerifier::CachingCertVerifier(std::__1::unique_ptr<net::CertVerifier, std::__1::default_delete<net::CertVerifier> >) net/cert/caching_cert_verifier.cc:27:3 (content_browsertests+0x36a9ee1)
#8 MakeUnique<net::CachingCertVerifier, std::__1::unique_ptr<net::MultiThreadedCertVerifier, std::__1::default_delete<net::MultiThreadedCertVerifier> > > base/memory/ptr_util.h:56:33 (content_browsertests+0x3658e92)
#9 net::CertVerifier::CreateDefault() net/cert/cert_verifier.cc:84 (content_browsertests+0x3658e92)
#10 content::ShellURLRequestContextGetter::GetURLRequestContext() content/shell/browser/shell_url_request_context_getter.cc:166:33 (content_browsertests+0x279c1b5)
#11 content::ChromeAppCacheService::InitializeOnIOThread(base::FilePath const&, content::ResourceContext*, net::URLRequestContextGetter*, scoped_refptr<storage::SpecialStoragePolicy>) content/browser/appcache/chrome_appcache_service.cc:44:49 (content_browsertests+0x1f003fc)
#12 Invoke<const scoped_refptr<content::ChromeAppCacheService> &, const base::FilePath &, content::ResourceContext *const &, net::URLRequestContextGetter *, storage::SpecialStoragePolicy *> base/bind_internal.h:214:12 (content_browsertests+0x243483c)
#13 MakeItSo<void (content::ChromeAppCacheService::*const &)(const base::FilePath &, content::ResourceContext *, net::URLRequestContextGetter *, scoped_refptr<storage::SpecialStoragePolicy>), const scoped_refptr<content::ChromeAppCacheService> &, const base::FilePath &, content::ResourceContext *const &, net::URLRequestContextGetter *, storage::SpecialStoragePolicy *> base/bind_internal.h:285 (content_browsertests+0x243483c)
#14 RunImpl<void (content::ChromeAppCacheService::*const &)(const base::FilePath &, content::ResourceContext *, net::URLRequestContextGetter *, scoped_refptr<storage::SpecialStoragePolicy>), const std::__1::tuple<scoped_refptr<content::ChromeAppCacheService>, base::FilePath, content::ResourceContext *, base::internal::RetainedRefWrapper<net::URLRequestContextGetter>, base::internal::RetainedRefWrapper<storage::SpecialStoragePolicy> > &, 0, 1, 2, 3, 4> base/bind_internal.h:361 (content_browsertests+0x243483c)
#15 base::internal::Invoker<base::internal::BindState<void (content::ChromeAppCacheService::*)(base::FilePath const&, content::ResourceContext*, net::URLRequestContextGetter*, scoped_refptr<storage::SpecialStoragePolicy>), scoped_refptr<content::ChromeAppCacheService>, base::FilePath, content::ResourceContext*, base::internal::RetainedRefWrapper<net::URLRequestContextGetter>, base::internal::RetainedRefWrapper<storage::SpecialStoragePolicy> >, void ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:339 (content_browsertests+0x243483c)
#16 Run base/callback.h:91:12 (content_browsertests+0x2cbeb3c)
#17 base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) base/debug/task_annotator.cc:59 (content_browsertests+0x2cbeb3c)
#18 base::MessageLoop::RunTask(base::PendingTask*) base/message_loop/message_loop.cc:404:19 (content_browsertests+0x2c09c42)
#19 base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) base/message_loop/message_loop.cc:415:5 (content_browsertests+0x2c0a1ea)
#20 base::MessageLoop::DoWork() base/message_loop/message_loop.cc:503:13 (content_browsertests+0x2c0a634)
#21 base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) base/message_loop/message_pump_libevent.cc:219:31 (content_browsertests+0x2c0f170)
#22 base::MessageLoop::RunHandler() base/message_loop/message_loop.cc:368:10 (content_browsertests+0x2c09773)
#23 base::RunLoop::Run() base/run_loop.cc:105:10 (content_browsertests+0x2c38d26)
#24 base::Thread::Run(base::RunLoop*) base/threading/thread.cc:255:13 (content_browsertests+0x2c70539)
#25 content::BrowserThreadImpl::IOThreadRun(base::RunLoop*) content/browser/browser_thread_impl.cc:278:11 (content_browsertests+0x1f5cc4f)
#26 content::BrowserThreadImpl::Run(base::RunLoop*) content/browser/browser_thread_impl.cc:313:14 (content_browsertests+0x1f5ce72)
#27 base::Thread::ThreadMain() base/threading/thread.cc:338:3 (content_browsertests+0x2c70784)
#28 base::(anonymous namespace)::ThreadFunc(void*) base/threading/platform_thread_posix.cc:71:13 (content_browsertests+0x2c68a1d)
Mutex M254588807855316976 is already destroyed.
Thread T17 'Chrome_IOThread' (tid=15168, running) created by main thread at:
#0 pthread_create <null> (content_browsertests+0x4bf633)
#1 base::(anonymous namespace)::CreateThread(unsigned long, bool, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:110:13 (content_browsertests+0x2c685a7)
#2 base::PlatformThread::CreateWithPriority(unsigned long, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:193:10 (content_browsertests+0x2c684a5)
#3 base::Thread::StartWithOptions(base::Thread::Options const&) base/threading/thread.cc:112:15 (content_browsertests+0x2c7007c)
#4 content::BrowserThreadImpl::StartWithOptions(base::Thread::Options const&) content/browser/browser_thread_impl.cc:404:25 (content_browsertests+0x1f5d45a)
#5 content::BrowserMainLoop::CreateThreads() content/browser/browser_main_loop.cc:1138:49 (content_browsertests+0x1f4a6e7)
#6 Invoke<content::BrowserMainLoop *> base/bind_internal.h:214:12 (content_browsertests+0x1f4ed95)
#7 MakeItSo<int (content::BrowserMainLoop::*const &)(), content::BrowserMainLoop *> base/bind_internal.h:285 (content_browsertests+0x1f4ed95)
#8 RunImpl<int (content::BrowserMainLoop::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > &, 0> base/bind_internal.h:361 (content_browsertests+0x1f4ed95)
#9 base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:339 (content_browsertests+0x1f4ed95)
#10 Run base/callback.h:80:12 (content_browsertests+0x242b55c)
#11 content::StartupTaskRunner::RunAllTasksNow() content/browser/startup_task_runner.cc:45 (content_browsertests+0x242b55c)
#12 content::BrowserMainLoop::CreateStartupTasks() content/browser/browser_main_loop.cc:959:25 (content_browsertests+0x1f49866)
#13 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) content/browser/browser_main_runner.cc:126:17 (content_browsertests+0x1f4f6f6)
#14 ShellBrowserMain(content::MainFunctionParams const&, std::__1::unique_ptr<content::BrowserMainRunner, std::__1::default_delete<content::BrowserMainRunner> > const&) content/shell/browser/shell_browser_main.cc:23:32 (content_browsertests+0x2772747)
#15 content::ShellMainDelegate::RunProcess(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, content::MainFunctionParams const&) content/shell/app/shell_main_delegate.cc:296:16 (content_browsertests+0x2764883)
#16 content::RunNamedProcessTypeMain(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, content::MainFunctionParams const&, content::ContentMainDelegate*) content/app/content_main_runner.cc:399:35 (content_browsertests+0x1e3743b)
#17 content::ContentMainRunnerImpl::Run() content/app/content_main_runner.cc:705:12 (content_browsertests+0x1e38064)
#18 content::ContentServiceManagerMainDelegate::RunEmbedderProcess() content/app/content_service_manager_main_delegate.cc:41:32 (content_browsertests+0xfdd9bf)
#19 service_manager::Main(service_manager::MainParams const&) services/service_manager/embedder/main.cc:468:29 (content_browsertests+0x40e2615)
#20 content::ContentMain(content::ContentMainParams const&) content/app/content_main.cc:19:10 (content_browsertests+0xfde37b)
#21 content::BrowserTestBase::SetUp() content/public/test/browser_test_base.cc:265:3 (content_browsertests+0x270475f)
#22 content::ContentBrowserTest::SetUp() content/public/test/content_browser_test.cc:89:20 (content_browsertests+0x26f76c6)
#23 HandleExceptionsInMethodIfSupported<testing::Test, void> third_party/googletest/src/googletest/src/gtest.cc:2458:12 (content_browsertests+0x26102b6)
#24 testing::Test::Run() third_party/googletest/src/googletest/src/gtest.cc:2470 (content_browsertests+0x26102b6)
#25 testing::TestInfo::Run() third_party/googletest/src/googletest/src/gtest.cc:2656:11 (content_browsertests+0x261115d)
#26 testing::TestCase::Run() third_party/googletest/src/googletest/src/gtest.cc:2774:28 (content_browsertests+0x26118a6)
#27 testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/src/googletest/src/gtest.cc:4647:43 (content_browsertests+0x261c516)
#28 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/src/googletest/src/gtest.cc:2458:12 (content_browsertests+0x261be44)
#29 testing::UnitTest::Run() third_party/googletest/src/googletest/src/gtest.cc:4255 (content_browsertests+0x261be44)
#30 RUN_ALL_TESTS third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46 (content_browsertests+0x2738d97)
#31 base::TestSuite::Run() base/test/test_suite.cc:271 (content_browsertests+0x2738d97)
#32 content::ContentTestLauncherDelegate::RunTestSuite(int, char**) content/test/content_test_launcher.cc:105:48 (content_browsertests+0x26fc55b)
#33 content::LaunchTests(content::TestLauncherDelegate*, int, int, char**) content/public/test/test_launcher.cc:520:31 (content_browsertests+0x272346d)
#34 main content/test/content_test_launcher.cc:131:10 (content_browsertests+0x26fc4e2)
SUMMARY: ThreadSanitizer: data race (/b/s/w/ir/out/Release/content_browsertests+0x4be1e9) in __interceptor_realloc
==================
Trying to start websocket server on ws://127.0.0.1:0...
WebSocket server started on ws://127.0.0.1:46184...
sending server_data: {"host": "127.0.0.1", "port": 46184} (36 bytes)
[ OK ] WorkerTest.WebSocketSharedWorker (2925 ms)
,
May 9 2017
,
May 9 2017
Ryan, do you know what's happening here? Is EnsureNSSInit() supposed to call getenv(), which is thread-unsafe?
,
May 9 2017
I took a look to this. crrev.com/2861163002 has been reverted as it caused a TSan failure. I think my CL isn't introducing a race itself, but it's causing TSan to discover another race that happens later on. The race sounds very similar to Issue 66835 . I had a chat offline with glider@ and he suggested to add a suppression here. More context: In my CL I am I'm adding a setenv() call in the main(), before any thread is created (hence why I am convinced the race is not there). After my CL has landed TSan complains about a race around setenv. However the two code paths calling setenv seem completely unrelated with my CL. If I am reading that log correctly, TSan is complaining about a setenv happening, on the main thread, after my code, in: service_manager::Main(service_manager::MainParams const&) ... content::BrowserTestBase::ProxyRunTestOnMainThreadLoop content::WorkerTest_WebSocketSharedWorker_Test::RunTestOnMainThread() ... AppendToPythonPath(base::FilePath const&) And a getenv happening on the Chrome IO thread from: libfreeblpriv3.so crypto::EnsureNSSInit() net::CertDatabase::GetInstance() ... content::BrowserThreadImpl::IOThreadRun(base::RunLoop*) Which sounds real, but at the same time sounds independent of my CL. Re #4, sadly I think the getenv indirectly happens from the NSS binary (libfreeblpriv3.so, which seems part of NSS). Is that supposed to be thread safe?
,
May 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2fc330d0b93d4bfd7bd04b9fdd3102e529901f91 commit 2fc330d0b93d4bfd7bd04b9fdd3102e529901f91 Author: primiano <primiano@chromium.org> Date: Tue May 09 13:07:12 2017 Reland of Linux: Disable DBus auto-launch (patchset #1 id:1 of https://codereview.chromium.org/2869843003/ ) Reason for reland: Adding TSan suppression. The race is independent of this CL, see crbug.com/719633 Original issue's description: > Revert of Linux: Disable DBus auto-launch (patchset #1 id:1 of https://codereview.chromium.org/2861163002/ ) > > Reason for revert: > Speculative revert -- the TSan bots have been reporting a data race when setting Envvars (in this case, appending to the python path to start a websocket server). The race appeared immediately after this patch landed, so it may be legitimate. Reverting this to see if it clears the failures up; if so, we'll probably just have to serialize the calls to setenv. > > Filed crbug.com/719633 for this as well. > > Original issue's description: > > Linux: Disable DBus auto-launch > > > > This is a workaround (ETA ~ 2-3 years) for libdbus not being multi-threading > > friendly and causing random hangs when running chrome outside of Linux > > desktop environments. > > > > Background: > > ----------- > > Typically, Linux desktop environments set the DBUS_SESSION_BUS_ADDRESS > > environment variable. This variable allows the dbus client library to > > directly connect to the existing bus, which is started by the desktop > > environment or systemd. > > When this variable is missing, the dbus client library will fallback > > to auto-launch mode [1], which causes 4 nested fork() + exec() calls. > > Doing this has two problems: (i) slows down startup; (ii) can hang > > the browser if the fork() happens while another thread is in a malloc() > > (Chrome's tcmalloc has no at-fork handlers). > > This situation (no env variable) is very common in test scenarios > > (browsertests, chromedriver, etc). > > > > Change introduced by this CL: > > ----------------------------- > > This CL sets the bus address env variable to "disabled:" if not set. > > This effectively shuts down the dbus auto-launch. If necessary, this > > behavior can be restored by setting, before launching chrome, > > DBUS_SESSION_BUS_ADDRESS="autolaunch:" . > > This workaround will be necessary until libdbus and gspawn are fixed > > to be multi-threading friendly [2,3] and that fix rolls into the > > various distributions. > > The change is introduced in the main embedder rather than in the > > google-chrome wrapper, as several binaries can be affected by this, > > for instance: > > - browser tests (http://crbug.com/693668) > > - chrome --headless > > - webdriver/selenium which seem to directly invoke "chrome" > > see https://github.com/SeleniumHQ/docker-selenium/issues/87 > > > > [1] https://dbus.freedesktop.org/doc/dbus-launch.1.html > > [2] https://bugs.freedesktop.org/show_bug.cgi?id=100843 > > [3] https://bugs.chromium.org/p/chromedriver/issues/detail?id=1699 > > > > BUG= 715658 , 695643 , 713947 > > TEST=strace -ff -o trace chrome; grep dbus-launch trace* > > > > Review-Url: https://codereview.chromium.org/2861163002 > > Cr-Commit-Position: refs/heads/master@{#469987} > > Committed: https://chromium.googlesource.com/chromium/src/+/8511820ec8280caacbd4f81f3ecd13b6c61681b0 > Review-Url: https://codereview.chromium.org/2869843003 > Cr-Commit-Position: refs/heads/master@{#470059} > Committed: https://chromium.googlesource.com/chromium/src/+/1e78cb7863da28bb3411286cdbcc4fb4510ce173 BUG= 715658 , 695643 , 713947 , 719633 TBR=satorux@google.com,thestig@chromium.org,jam@chromium.org Review-Url: https://codereview.chromium.org/2865283002 Cr-Commit-Position: refs/heads/master@{#470301} [modify] https://crrev.com/2fc330d0b93d4bfd7bd04b9fdd3102e529901f91/build/sanitizers/tsan_suppressions.cc [modify] https://crrev.com/2fc330d0b93d4bfd7bd04b9fdd3102e529901f91/services/service_manager/embedder/main.cc
,
May 9 2017
Removing from sheriff queue. It looks like relanding with that suppression has made TSan happy. Odd that it didn't pick up the real race before, but you're right, primiano, that calling setenv *before* spawning any threads should be completely safe. Is the real issue not that the tests modify $PYTHONPATH in the current env before calling the Python binary to start the websocket server?
,
May 9 2017
It used to be threadsafe ;( https://cs.chromium.org/chromium/src/crypto/nss_util.h?rcl=9eaf17cc7c79bc2dabaabc8f306acef1680e946e&l=35 (Or at least, we had documented it as such) Looks like Mozilla always dispatches NSS initialization to their main thread. NSS has some getenv calls (although prefers getenv_secure), and while it internally guards those with locks, understandably, that wouldn't catch other cases.
,
May 10 2017
Well, getenv *is* technically threadsafe, but that's technically only because setenv/putenv are MT-Unsafe, and so it can simply be assumed that setenv will never be called in a program where threads are in use. http://man7.org/linux/man-pages/man3/getenv.3.html#ATTRIBUTES http://man7.org/linux/man-pages/man3/setenv.3.html#ATTRIBUTES As long as the environment is static, getenv (and presumably EnsureNSSInit) are fine. Calling setenv from a thread invalidates that.
,
May 15 2017
Adding more information as sheriff. Seems like more work is needed here. According to https://uberchromegw.corp.google.com/i/chromium.memory/builders/Linux%20TSan%20Tests?numbuilds=200 content_browsertests has failed in more than 50% of the rounds since May 12. Any ideas on how to improve the current state?
,
May 15 2017
> Any ideas on how to improve the current state? Fix the bug. setenv for a subprocess should be set between fork and exec. Or, looking at the code it seems that it sets the same constant env again and again, in such case it could be set once on process startup before and threads are created.
,
May 15 2017
Can you point out the log? I looked at the link in #10 but it seems that now content_browsertests is flaky on TopDocumentIsolationTest.CrossSiteAtLevelTwo which doesn't seem to be realted at all with this bug or setenv (or am I missing something)? If that's the case please open a new bug. There seems to be a non-setenv race related with base::FeatureList which is accessed also by net::CertVerifier::CreateDefault(). I don't think this is any realated with anything else in this bug.
,
May 24 2017
Issue 725973 has been merged into this issue.
,
May 26 2017
,
Jun 1 2017
ClusterFuzz testcase 5795705930383360 is verified as fixed, so closing issue. If this is incorrect, please add ClusterFuzz-Wrong label and re-open the issue. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by olka@chromium.org
, May 9 2017