New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 602964 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 3
Type: Bug



Sign in to add a comment

Leak in NetworkMetricsProvider::ProbeWifiPHYLayerProtocol()

Project Member Reported by thestig@chromium.org, Apr 13 2016

Issue description

Not sure when this started, as the code has not changed in several weeks. It's been happening for at least a couple days now.

glider: Did you see this one on your shift?

e.g. https://build.chromium.org/p/chromium.memory.fyi/builders/Linux%20Tests%20%28valgrind%29%285%29/builds/40942

Leak_DefinitelyLost
292 (40 direct, 252 indirect) bytes in 1 blocks are definitely lost in loss record 40,758 of 41,552
  operator new(unsigned long) (m_replacemalloc/vg_replace_malloc.c:1140)
  _ZN4base4BindIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEJNS_8internal17UnretainedWrapperIS2_EEEEENS_8CallbackINS5_22MakeUnboundRunTypeImplIT_JDpT0_EE4TypeELNS5_8CopyModeE1EEESA_DpOSB_ (base/bind.h:106)
  base::internal::PostTaskAndReplyImpl::PostTaskAndReply(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::Callback<void (), (base::internal::CopyMode)1> const&) (base/threading/post_task_and_reply_impl.cc:83)
  base::TaskRunner::PostTaskAndReply(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::Callback<void (), (base::internal::CopyMode)1> const&) (/mnt/data/b/build/slave/chromium-rel-linux-valgrind-tests-5/build/src/out/Release/unit_tests)
  bool base::PostTaskAndReplyWithResult<net::WifiPHYLayerProtocol, net::WifiPHYLayerProtocol>(base::TaskRunner*, tracked_objects::Location const&, base::Callback<net::WifiPHYLayerProtocol (), (base::internal::CopyMode)1> const&, base::Callback<void (net::WifiPHYLayerProtocol), (base::internal::CopyMode)1> const&) (/mnt/data/b/build/slave/chromium-rel-linux-valgrind-tests-5/build/src/out/Release/unit_tests)
  metrics::NetworkMetricsProvider::ProbeWifiPHYLayerProtocol() (components/metrics/net/network_metrics_provider.cc:154)
  metrics::NetworkMetricsProvider::NetworkMetricsProvider(base::TaskRunner*) (components/metrics/net/network_metrics_provider.cc:38)
  ChromeMetricsServiceClient::Initialize() (chrome/browser/metrics/chrome_metrics_service_client.cc:372)
  ChromeMetricsServiceClient::Create(metrics::MetricsStateManager*, PrefService*) (chrome/browser/metrics/chrome_metrics_service_client.cc:205)
  ChromeMetricsServicesManagerClient::CreateMetricsServiceClient() (/mnt/data/b/build/slave/chromium-rel-linux-valgrind-tests-5/build/src/out/Release/unit_tests)
  metrics_services_manager::MetricsServicesManager::GetMetricsServiceClient() (components/metrics_services_manager/metrics_services_manager.cc:61)
  metrics_services_manager::MetricsServicesManager::GetMetricsService() (/mnt/data/b/build/slave/chromium-rel-linux-valgrind-tests-5/build/src/out/Release/unit_tests)
  BrowserProcessImpl::metrics_service() (/mnt/data/b/build/slave/chromium-rel-linux-valgrind-tests-5/build/src/out/Release/unit_tests)
  IOThread::IOThread(PrefService*, policy::PolicyService*, net_log::ChromeNetLog*, extensions::EventRouterForwarder*) (chrome/browser/io_thread.cc:509)
  BrowserProcessImpl::PreCreateThreads() (chrome/browser/browser_process_impl.cc:998)
  BrowserProcessImplTest_LifeCycle_Test::TestBody() (chrome/browser/browser_process_impl_unittest.cc:83)
Suppression (error hash=#1FCE8081F47A3D4D#):
  For more info on using suppressions see http://dev.chromium.org/developers/tree-sheriffs/sheriff-details-chromium/memory-sheriff#TOC-Suppressing-memory-reports
{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   fun:_Znw*
   fun:_ZN4base4BindIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEJNS_8internal17UnretainedWrapperIS2_EEEEENS_8CallbackINS5_22MakeUnboundRunTypeImplIT_JDpT0_EE4TypeELNS5_8CopyModeE1EEESA_DpOSB_
   fun:_ZN4base8internal20PostTaskAndReplyImpl16PostTaskAndReplyERKN15tracked_objects8LocationERKNS_8CallbackIFvvELNS0_8CopyModeE1EEESB_
   fun:_ZN4base10TaskRunner16PostTaskAndReplyERKN15tracked_objects8LocationERKNS_8CallbackIFvvELNS_8internal8CopyModeE1EEESB_
   fun:_ZN4base26PostTaskAndReplyWithResultIN3net20WifiPHYLayerProtocolES2_EEbPNS_10TaskRunnerERKN15tracked_objects8LocationERKNS_8CallbackIFT_vELNS_8internal8CopyModeE1EEERKNS9_IFvT0_ELSD_1EEE
   fun:_ZN7metrics22NetworkMetricsProvider25ProbeWifiPHYLayerProtocolEv
   fun:_ZN7metrics22NetworkMetricsProviderC1EPN4base10TaskRunnerE
   fun:_ZN26ChromeMetricsServiceClient10InitializeEv
   fun:_ZN26ChromeMetricsServiceClient6CreateEPN7metrics19MetricsStateManagerEP11PrefService
   fun:_ZN34ChromeMetricsServicesManagerClient26CreateMetricsServiceClientEv
   fun:_ZN24metrics_services_manager22MetricsServicesManager23GetMetricsServiceClientEv
   fun:_ZN24metrics_services_manager22MetricsServicesManager17GetMetricsServiceEv
   fun:_ZN18BrowserProcessImpl15metrics_serviceEv
   fun:_ZN8IOThreadC1EP11PrefServicePN6policy13PolicyServiceEPN7net_log12ChromeNetLogEPN10extensions20EventRouterForwarderE
   fun:_ZN18BrowserProcessImpl16PreCreateThreadsEv
   fun:_ZN37BrowserProcessImplTest_LifeCycle_Test8TestBodyEv
}
 
Project Member

Comment 1 by bugdroid1@chromium.org, Apr 13 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a7ca0e9e8299f235561c3717bee075a19d8ceb23

commit a7ca0e9e8299f235561c3717bee075a19d8ceb23
Author: thestig <thestig@chromium.org>
Date: Wed Apr 13 10:22:36 2016

Valgrind: Add a new suppression and update another.

Updated suppression most likely due to r386547.

BUG= 576259 , 602964 
TBR=glider@chromium.org

Review URL: https://codereview.chromium.org/1884933002

Cr-Commit-Position: refs/heads/master@{#386959}

[modify] https://crrev.com/a7ca0e9e8299f235561c3717bee075a19d8ceb23/tools/valgrind/memcheck/suppressions.txt

Comment 2 by glider@chromium.org, Apr 13 2016

Nope, I didn't, and it doesn't show up in my last `waterfall.sh match` run
Cc: gayane@chromium.org
https://build.chromium.org/p/chromium.memory.fyi/builders/Linux%20Tests%20%28valgrind%29%285%29/builds/40889/ is the earliest instance I found. Perhaps r385845 caused this?

I suspect the Linux Valgrind bots no longer output the individual error hashes, so waterfall.sh isn't picking them up?

Comment 4 by gayane@chromium.org, Apr 14 2016

the code path in the memory dump is indeed a new change from my recent CL https://codereview.chromium.org/1818613002/

But I don't understand why it thinks there is a memory leak. Also this scenario is not possible in production, so if the setup is done properly this path wouldn't even be triggered. 

Does this happen in some unittest or memory_analyzer runs independently after all unittests?

Comment 5 by gab@chromium.org, Aug 11 2016

Status: Asss (was: Untriaged)

Comment 6 by gab@chromium.org, Aug 11 2016

Cc: asvitk...@chromium.org
Components: Internals
Owner: gab@chromium.org
Status: Assigned (was: Asss)
Oops, one too many 's' before hitting <ENTER> for autofill :-O!

https://codereview.chromium.org/2237643002/ exposes this more reliably (and bypasses the current suppression by being a slightly new post stack). Basically the issue is that the PostTaskAndReplyRelay is always leaked if the reply isn't run (which can happen if the |task| doesn't complete before the MessageLoop is gone on shutdown).

This is WAI for PostTaskAndReplyRelay:  https://crbug.com/371974#c14 

I'll look into adding a suppression in base/threading/post_task_and_reply_impl.cc and removing all the individual suppressions in suppressions.txt for this same bug.
Sweet - thanks for getting to the bottom of this!
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3f00cf8a85b52702c4d0714a154a17d00006382e

commit 3f00cf8a85b52702c4d0714a154a17d00006382e
Author: gab <gab@chromium.org>
Date: Tue Aug 23 19:05:56 2016

Annotate PostTaskAndReplyRelay as being self-managed and potentially leaked in PostTaskAndReplyImpl.

The leaks occur on shutdown when the |task| either hasn't been processed
yet or doesn't have time to post back to the origin sequence before
it stops receiving or processing tasks.

Those leaks being racy with shutdown, this has resulted in numerous
flaky LSan failures resulting in suppressions over the years.

BUG= 602964 

Review-Url: https://codereview.chromium.org/2236993004
Cr-Commit-Position: refs/heads/master@{#413803}

[modify] https://crrev.com/3f00cf8a85b52702c4d0714a154a17d00006382e/base/threading/post_task_and_reply_impl.cc
[modify] https://crrev.com/3f00cf8a85b52702c4d0714a154a17d00006382e/tools/valgrind/memcheck/suppressions.txt

Comment 9 by gab@chromium.org, Aug 31 2016

Status: Fixed (was: Assigned)

Sign in to add a comment