So indeed this is a testing-only race. I think that there is no real race (there should be no threads active at that point) but what happens is that TSan observes:
- a thread (that ends before the test reaches TearDown()) accessing the instance pointer in GetOrCreateLazyPointer with a load-acquire
- clearing up the instance pointer (in TearDown()) with a non-barriered write

so it says "at some point you will have a race here"
I was almost tempted to file a suppression. But then I realized that years ago, young myself implemented this DeleteForTesting as a gross hack in base/singleton.h .
At very least I can clear it up now that we have base::NoDestructor.

https://chromium-review.googlesource.com/#/c/chromium/src/+/935841

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/919402657fd592a1c1388905279a300d6448e08d

commit 919402657fd592a1c1388905279a300d6448e08d
Author: Primiano Tucci <primiano@chromium.org>
Date: Mon Feb 26 23:06:32 2018

tracing: Singleton -> NoDestructor and fix TSan test violation

This CL cleans up the hack put in place in base::Singleton to
reset the TraceLog instance in tests, in favour of a least worst
hack to reset the instance within trace_log.cc .
Other than cleaning up singleton.h, this also removes a TSan
violation caused by calling Singleton::OnExit() (see bug).
The violation was reported, I believe, for the following reason:
- A thread (that ends before the test reaches TearDown()) accesses
  the Singleton instance pointer in GetOrCreateLazyPointer with
  a load-acquire.
- TearDown() clears up the Singleton instance pointer
  with a non-barriered write.
This is enough to make TSan conclude "this will be racy" even if
the race doesn't happen (because of causality in the TearDown(),
all threads created by the tests should be gone once there).

This CL fixes the problem by switching TraceLog's singleton to
a base::NoDestructor. This should also make TraceLog::GetInstance
slightly faster in production builds as it removes one layer of
pointer dereferencing (Singleton's instance is heap-based,
base::NoDestructor is bss-based).
The ResetForTesting method is implemented by simply invoking the
destructor, clearing up the static storage and
re-placement-new-ing the object. This keeps the pointer returned
by GetInstance() unchanged, but still achieves the reset needed
by tests.

Bug:  815234 
TBR: rouslan@chromium.org
Change-Id: I4c0dfc33ea45e64f36234da2006ece96905914b9
Reviewed-on: https://chromium-review.googlesource.com/935841
Reviewed-by: Rouslan Solomakhin <rouslan@chromium.org>
Reviewed-by: Lei Zhang <thestig@chromium.org>
Commit-Queue: Primiano Tucci <primiano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539307}
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/base/memory/singleton.h
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/base/trace_event/memory_dump_manager_unittest.cc
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/base/trace_event/trace_event_unittest.cc
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/base/trace_event/trace_log.cc
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/base/trace_event/trace_log.h
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/components/payments/content/service_worker_payment_app_factory.h
[modify] https://crrev.com/919402657fd592a1c1388905279a300d6448e08d/services/resource_coordinator/public/cpp/memory_instrumentation/tracing_integration_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4b4bfc5848baa03343cba8c470df845968e42559

commit 4b4bfc5848baa03343cba8c470df845968e42559
Author: Mostyn Bramley-Moore <mostynb@vewd.com>
Date: Tue Feb 27 13:08:58 2018

[jumbo] followup after 'tracing: Singleton -> NoDestructor and fix TSan test violation'

There are multiple g_instance_for_testing globals after
https://chromium-review.googlesource.com/c/chromium/src/+/935841 - let's
rename one of them to avoid jumbo build breakage.

TBR=primiano@chromium.org

Bug:  815234 ,775547
Change-Id: I184578b694e59dad8b06643db199f32820fd5887
Reviewed-on: https://chromium-review.googlesource.com/938821
Commit-Queue: Mostyn Bramley-Moore <mostynb@vewd.com>
Reviewed-by: Mostyn Bramley-Moore <mostynb@vewd.com>
Cr-Commit-Position: refs/heads/master@{#539424}
[modify] https://crrev.com/4b4bfc5848baa03343cba8c470df845968e42559/base/trace_event/trace_log.cc

https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_tsan_rel_ng seems green now