New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 638862 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Data races in SingleProcessMemoryTracingTest.ManyInterleavedDumps

Project Member Reported by glider@chromium.org, Aug 18 2016

Issue description

See https://build.chromium.org/p/chromium.memory.full/builders/Linux%20TSan%20Tests/builds/650/steps/content_browsertests%20on%20Ubuntu-12.04/logs/stdio:

[ RUN      ] SingleProcessMemoryTracingTest.ManyInterleavedDumps
[17807:17807:0817/220212:14799158853:WARNING:audio_manager.cc(317)] Multiple instances of AudioManager detected
[17807:17807:0817/220212:14799159078:WARNING:audio_manager.cc(278)] Multiple instances of AudioManager detected
Xlib:  extension "RANDR" missing on display ":9".
[17807:17868:0817/220213:14799657076:WARNING:histograms.cc(40)] Started multiple compositor clients (Browser, Renderer) in one process. Some metrics will be disabled.
==================
WARNING: ThreadSanitizer: data race (pid=17807)
  Read of size 4 at 0x7d5c0000c7e8 by thread T14 (mutexes: write M355359168799994240):
    #0 gpu::CommandBufferProxyImpl::Flush(int) gpu/ipc/client/command_buffer_proxy_impl.cc:254:19 (content_browsertests+0x000000bb53db)
    #1 gpu::CommandBufferHelper::Flush() gpu/command_buffer/client/cmd_buffer_helper.cc:180:22 (content_browsertests+0x000000b8fc71)
    #2 gpu::gles2::GLES2Implementation::FlushHelper() gpu/command_buffer/client/gles2_implementation.cc:1380:33 (content_browsertests+0x00000391b63a)
    #3 gpu::gles2::GLES2Implementation::ShallowFlushCHROMIUM() gpu/command_buffer/client/gles2_implementation.cc:1374:3 (content_browsertests+0x00000391b713)
    #4 cc::OneCopyRasterBufferProvider::CopyOnWorkerThread(cc::StagingBuffer*, cc::ResourceProvider::ScopedWriteLockGL*, gpu::SyncToken const&, cc::RasterSource const*, unsigned long, unsigned long) cc/raster/one_copy_raster_buffer_provider.cc:357:13 (content_browsertests+0x0000030e45b3)
    #5 cc::OneCopyRasterBufferProvider::PlaybackAndCopyOnWorkerThread(cc::Resource const*, cc::ResourceProvider::ScopedWriteLockGL*, gpu::SyncToken const&, cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, float, cc::RasterSource::PlaybackSettings const&, unsigned long, unsigned long) cc/raster/one_copy_raster_buffer_provider.cc:199:3 (content_browsertests+0x0000030e34f1)
    #6 cc::OneCopyRasterBufferProvider::RasterBufferImpl::Playback(cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, unsigned long, float, cc::RasterSource::PlaybackSettings const&) cc/raster/one_copy_raster_buffer_provider.cc:60:12 (content_browsertests+0x0000030e3297)
    #7 cc::(anonymous namespace)::RasterTaskImpl::RunOnWorkerThread() cc/tiles/tile_manager.cc:92:21 (content_browsertests+0x00000304487d)
    #8 cc::SingleThreadTaskGraphRunner::RunTaskWithLockAcquired() cc/raster/single_thread_task_graph_runner.cc:155:11 (content_browsertests+0x00000301af46)
    #9 Run cc/raster/single_thread_task_graph_runner.cc:117:10 (content_browsertests+0x00000301b1cf)
    #10 non-virtual thunk to cc::SingleThreadTaskGraphRunner::Run() cc/raster/single_thread_task_graph_runner.cc:113 (content_browsertests+0x00000301b1cf)
    #11 base::DelegateSimpleThread::Run() base/threading/simple_thread.cc:92:13 (content_browsertests+0x000002833738)
    #12 base::SimpleThread::ThreadMain() base/threading/simple_thread.cc:68:3 (content_browsertests+0x0000028334b2)
    #13 base::(anonymous namespace)::ThreadFunc(void*) base/threading/platform_thread_posix.cc:71:13 (content_browsertests+0x00000282e95d)

  Previous write of size 8 at 0x7d5c0000c7e8 by main thread:
    #0 memcpy <null> (content_browsertests+0x0000004b9b5e)
    #1 Read gpu/command_buffer/common/command_buffer_shared.h:55:14 (content_browsertests+0x000000bb5369)
    #2 TryUpdateState gpu/ipc/client/command_buffer_proxy_impl.cc:727 (content_browsertests+0x000000bb5369)
    #3 gpu::CommandBufferProxyImpl::GetLastToken() gpu/ipc/client/command_buffer_proxy_impl.cc:248 (content_browsertests+0x000000bb5369)
    #4 last_token_read gpu/command_buffer/client/cmd_buffer_helper.h:182:61 (content_browsertests+0x000000b944f5)
    #5 gpu::RingBuffer::GetLargestFreeSizeNoWaiting() gpu/command_buffer/client/ring_buffer.cc:149 (content_browsertests+0x000000b944f5)
    #6 gpu::RingBuffer::GetTotalFreeSizeNoWaiting() gpu/command_buffer/client/ring_buffer.cc:174:36 (content_browsertests+0x000000b94c5e)
    #7 gpu::TransferBuffer::GetFreeSize() const gpu/command_buffer/client/transfer_buffer.cc:96:39 (content_browsertests+0x000000b96d6c)
    #8 gpu::gles2::GLES2Implementation::OnMemoryDump(base::trace_event::MemoryDumpArgs const&, base::trace_event::ProcessMemoryDump*) gpu/command_buffer/client/gles2_implementation.cc:451:37 (content_browsertests+0x0000039174ea)
    #9 non-virtual thunk to gpu::gles2::GLES2Implementation::OnMemoryDump(base::trace_event::MemoryDumpArgs const&, base::trace_event::ProcessMemoryDump*) gpu/command_buffer/client/gles2_implementation.cc:433:27 (content_browsertests+0x000003917594)
    #10 base::trace_event::MemoryDumpManager::InvokeOnMemoryDump(base::trace_event::MemoryDumpManager::ProcessMemoryDumpAsyncState*) base/trace_event/memory_dump_manager.cc:581:52 (content_browsertests+0x00000283f9bd)
    #11 Invoke<base::trace_event::MemoryDumpManager *, base::trace_event::MemoryDumpManager::ProcessMemoryDumpAsyncState *> base/bind_internal.h:214:12 (content_browsertests+0x000002841a06)
...
  Mutex M355359168799994240 is already destroyed.
...
SUMMARY: ThreadSanitizer: data race gpu/ipc/client/command_buffer_proxy_impl.cc:254:19 in gpu::CommandBufferProxyImpl::Flush(int)
==================
==================
WARNING: ThreadSanitizer: data race (pid=17807)
  Write of size 8 at 0x7d5c0000c7e8 by main thread:
    #0 memcpy <null> (content_browsertests+0x0000004b9b5e)
    #1 Read gpu/command_buffer/common/command_buffer_shared.h:55:14 (content_browsertests+0x000000bb5369)
    #2 TryUpdateState gpu/ipc/client/command_buffer_proxy_impl.cc:727 (content_browsertests+0x000000bb5369)
    #3 gpu::CommandBufferProxyImpl::GetLastToken() gpu/ipc/client/command_buffer_proxy_impl.cc:248 (content_browsertests+0x000000bb5369)
    #4 last_token_read gpu/command_buffer/client/cmd_buffer_helper.h:182:61 (content_browsertests+0x000000b944f5)
    #5 gpu::RingBuffer::GetLargestFreeSizeNoWaiting() gpu/command_buffer/client/ring_buffer.cc:149 (content_browsertests+0x000000b944f5)
...
  Previous read of size 4 at 0x7d5c0000c7e8 by thread T14 (mutexes: write M474423083948598872, write M355359168799994240):
    #0 gpu::CommandBufferProxyImpl::GetLastError() gpu/ipc/client/command_buffer_proxy_impl.cc:688:22 (content_browsertests+0x000000bb8aaa)
    #1 gpu::CommandBufferHelper::IsContextLost() gpu/command_buffer/client/cmd_buffer_helper.cc:59:54 (content_browsertests+0x000000b8f658)
    #2 gpu::gles2::QueryTracker::Query::CheckResultsAvailable(gpu::CommandBufferHelper*) gpu/command_buffer/client/query_tracker.cc:188:34 (content_browsertests+0x000003966d59)
    #3 gpu::gles2::GLES2Implementation::GetQueryObjectValueHelper(char const*, unsigned int, unsigned int, unsigned long*) gpu/command_buffer/client/gles2_implementation.cc:1251:24 (content_browsertests+0x00000391a89c)
    #4 gpu::gles2::GLES2Implementation::GetQueryObjectuivEXT(unsigned int, unsigned int, unsigned int*) gpu/command_buffer/client/gles2_implementation.cc:5556:7 (content_browsertests+0x000003934074)
    #5 CheckForQueryResult cc/raster/staging_buffer_pool.cc:33:7 (content_browsertests+0x0000030e6ade)
    #6 cc::StagingBufferPool::AcquireStagingBuffer(cc::Resource const*, unsigned long) cc/raster/staging_buffer_pool.cc:246 (content_browsertests+0x0000030e6ade)
    #7 cc::OneCopyRasterBufferProvider::PlaybackAndCopyOnWorkerThread(cc::Resource const*, cc::ResourceProvider::ScopedWriteLockGL*, gpu::SyncToken const&, cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, float, cc::RasterSource::PlaybackSettings const&, unsigned long, unsigned long) cc/raster/one_copy_raster_buffer_provider.cc:192:21 (content_browsertests+0x0000030e34af)
    #8 cc::OneCopyRasterBufferProvider::RasterBufferImpl::Playback(cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, unsigned long, float, cc::RasterSource::PlaybackSettings const&) cc/raster/one_copy_raster_buffer_provider.cc:60:12 (content_browsertests+0x0000030e3297)
    #9 cc::(anonymous namespace)::RasterTaskImpl::RunOnWorkerThread() cc/tiles/tile_manager.cc:92:21 (content_browsertests+0x00000304487d)
    #10 cc::SingleThreadTaskGraphRunner::RunTaskWithLockAcquired() cc/raster/single_thread_task_graph_runner.cc:155:11 (content_browsertests+0x00000301af46)
...

  Mutex M474423083948598872 is already destroyed.

  Mutex M355359168799994240 is already destroyed.
...
SUMMARY: ThreadSanitizer: data race (/b/swarm_slave/w/irZ2Vqtq/out/Release/content_browsertests+0x4b9b5e) in __interceptor_memcpy
==================
==================
WARNING: ThreadSanitizer: data race (pid=17807)
  Read of size 8 at 0x7d5c0000c7e0 by thread T14 (mutexes: write M355359168799994240):
    #0 memcpy <null> (content_browsertests+0x0000004b9b5e)
    #1 gpu::CommandBufferProxyImpl::GetLastState() gpu/ipc/client/command_buffer_proxy_impl.cc:244:10 (content_browsertests+0x000000bb5279)
    #2 get_offset gpu/command_buffer/client/cmd_buffer_helper.h:185:29 (content_browsertests+0x000000b8f50f)
    #3 gpu::CommandBufferHelper::CalcImmediateEntries(int) gpu/command_buffer/client/cmd_buffer_helper.cc:74 (content_browsertests+0x000000b8f50f)
    #4 gpu::CommandBufferHelper::Flush() gpu/command_buffer/client/cmd_buffer_helper.cc:182:5 (content_browsertests+0x000000b8fc8f)
    #5 gpu::gles2::GLES2Implementation::FlushHelper() gpu/command_buffer/client/gles2_implementation.cc:1380:33 (content_browsertests+0x00000391b63a)
    #6 gpu::gles2::GLES2Implementation::ShallowFlushCHROMIUM() gpu/command_buffer/client/gles2_implementation.cc:1374:3 (content_browsertests+0x00000391b713)
    #7 cc::OneCopyRasterBufferProvider::CopyOnWorkerThread(cc::StagingBuffer*, cc::ResourceProvider::ScopedWriteLockGL*, gpu::SyncToken const&, cc::RasterSource const*, unsigned long, unsigned long) cc/raster/one_copy_raster_buffer_provider.cc:357:13 (content_browsertests+0x0000030e45b3)
    #8 cc::OneCopyRasterBufferProvider::PlaybackAndCopyOnWorkerThread(cc::Resource const*, cc::ResourceProvider::ScopedWriteLockGL*, gpu::SyncToken const&, cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, float, cc::RasterSource::PlaybackSettings const&, unsigned long, unsigned long) cc/raster/one_copy_raster_buffer_provider.cc:199:3 (content_browsertests+0x0000030e34f1)
    #9 cc::OneCopyRasterBufferProvider::RasterBufferImpl::Playback(cc::RasterSource const*, gfx::Rect const&, gfx::Rect const&, unsigned long, float, cc::RasterSource::PlaybackSettings const&) cc/raster/one_copy_raster_buffer_provider.cc:60:12 (content_browsertests+0x0000030e3297)
    #10 cc::(anonymous namespace)::RasterTaskImpl::RunOnWorkerThread() cc/tiles/tile_manager.cc:92:21 (content_browsertests+0x00000304487d)
    #11 cc::SingleThreadTaskGraphRunner::RunTaskWithLockAcquired() cc/raster/single_thread_task_graph_runner.cc:155:11 (content_browsertests+0x00000301af46)
    #12 Run cc/raster/single_thread_task_graph_runner.cc:117:10 (content_browsertests+0x00000301b1cf)
    #13 non-virtual thunk to cc::SingleThreadTaskGraphRunner::Run() cc/raster/single_thread_task_graph_runner.cc:113 (content_browsertests+0x00000301b1cf)
    #14 base::DelegateSimpleThread::Run() base/threading/simple_thread.cc:92:13 (content_browsertests+0x000002833738)
    #15 base::SimpleThread::ThreadMain() base/threading/simple_thread.cc:68:3 (content_browsertests+0x0000028334b2)
    #16 base::(anonymous namespace)::ThreadFunc(void*) base/threading/platform_thread_posix.cc:71:13 (content_browsertests+0x00000282e95d)

  Previous write of size 8 at 0x7d5c0000c7e0 by main thread:
    #0 memcpy <null> (content_browsertests+0x0000004b9b5e)
    #1 Read gpu/command_buffer/common/command_buffer_shared.h:55:14 (content_browsertests+0x000000bb5369)
    #2 TryUpdateState gpu/ipc/client/command_buffer_proxy_impl.cc:727 (content_browsertests+0x000000bb5369)
    #3 gpu::CommandBufferProxyImpl::GetLastToken() gpu/ipc/client/command_buffer_proxy_impl.cc:248 (content_browsertests+0x000000bb5369)
    #4 last_token_read gpu/command_buffer/client/cmd_buffer_helper.h:182:61 (content_browsertests+0x000000b944f5)
    #5 gpu::RingBuffer::GetLargestFreeSizeNoWaiting() gpu/command_buffer/client/ring_buffer.cc:149 (content_browsertests+0x000000b944f5)
    #6 gpu::RingBuffer::GetTotalFreeSizeNoWaiting() gpu/command_buffer/client/ring_buffer.cc:174:36 (content_browsertests+0x000000b94c5e)
    #7 gpu::TransferBuffer::GetFreeSize() const gpu/command_buffer/client/transfer_buffer.cc:96:39 (content_browsertests+0x000000b96d6c)
    #8 gpu::gles2::GLES2Implementation::OnMemoryDump(base::trace_event::MemoryDumpArgs const&, base::trace_event::ProcessMemoryDump*) gpu/command_buffer/client/gles2_implementation.cc:451:37 (content_browsertests+0x0000039174ea)
    #9 non-virtual thunk to gpu::gles2::GLES2Implementation::OnMemoryDump(base::trace_event::MemoryDumpArgs const&, base::trace_event::ProcessMemoryDump*) gpu/command_buffer/client/gles2_implementation.cc:433:27 (content_browsertests+0x000003917594)
...
SUMMARY: ThreadSanitizer: data race (/b/swarm_slave/w/irZ2Vqtq/out/Release/content_browsertests+0x4b9b5e) in __interceptor_memcpy
==================
 

Comment 1 by piman@chromium.org, Aug 18 2016

Cc: piman@chromium.org
Owner: ericrk@chromium.org
->ericrk, it looks like the memory dump infra is accessing arbitrary commands buffers on arbitrary threads, without taking locks, and that's not ok.

Comment 2 by piman@chromium.org, Aug 18 2016

Cc: ericrk@chromium.org
Owner: reve...@chromium.org
Actually ->reveman who implemented this one (GLES2Implementation::OnMemoryDump) in particular
Components: Build

Comment 4 by piman@chromium.org, Oct 27 2016

Cc: gman@chromium.org a...@chromium.org
 Issue 659872  has been merged into this issue.

Comment 5 by piman@chromium.org, Oct 27 2016

Components: -Build Internals>GPU>Internals
@reveman: any update?
Owner: ericrk@chromium.org
I think I know how to address this. I'll take a look.
Owner: sunn...@chromium.org
sunnyps@, your latest CL should fix this I believe. Please feel free to assign back to me if this is not the case.
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 9 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1285660590d371fedced6253c43be569c2d054ee

commit 1285660590d371fedced6253c43be569c2d054ee
Author: sunnyps <sunnyps@chromium.org>
Date: Fri Dec 09 21:06:43 2016

gpu: Thread-safe command buffer state lookup.

This makes client side (in-process / proxy) command buffer state thread-
safe and adds release count to the client-server shared state. This
allows the compositor thread to monitor worker context sync token in the
service.

Client side command buffer state is synchronized using a lock. Extra
caching of the state is added to command buffer helper so that it
doesn't acquire the lock for every command.

Also fixes command buffer memory tracing so that it happens on the same
thread which the context provider is bound to.

R=piman@chromium.org,jbauman@chromium.org
BUG= 514813 , 638862 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2550583002
Cr-Commit-Position: refs/heads/master@{#437651}

[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/cc/test/test_context_support.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/cc/test/test_context_support.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/content/common/gpu/client/context_provider_command_buffer.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/content/renderer/pepper/ppb_graphics_3d_impl.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/client_test_helper.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/client_test_helper.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/cmd_buffer_helper.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/cmd_buffer_helper.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/cmd_buffer_helper_test.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/context_support.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/gles2_implementation.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/gles2_implementation.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/gpu_control.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/client/ring_buffer.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/common/cmd_buffer_common.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/common/command_buffer.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/common/command_buffer_mock.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/service/command_buffer_service.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/service/command_buffer_service.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/tests/gl_manager.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/tests/gl_manager.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/command_buffer/tests/gl_readback_unittest.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/gles2_conform_support/egl/context.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/gles2_conform_support/egl/context.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/ipc/client/command_buffer_proxy_impl.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/ipc/client/command_buffer_proxy_impl.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/ipc/in_process_command_buffer.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/ipc/in_process_command_buffer.h
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/gpu/ipc/service/gpu_command_buffer_stub.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/ppapi/proxy/ppapi_command_buffer_proxy.cc
[modify] https://crrev.com/1285660590d371fedced6253c43be569c2d054ee/ppapi/proxy/ppapi_command_buffer_proxy.h

Cc: sunn...@chromium.org jbau...@chromium.org vmi...@chromium.org ccameron@chromium.org
 Issue 667741  has been merged into this issue.
Status: Fixed (was: Assigned)
Should be fixed. Are there any test suppressions to revert? I couldn't find any.

Sign in to add a comment