New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 749297 link

Starred by 2 users
Status: WontFix
Merged: issue 749504
Owner:
█ stanisc@chromium.org
not on Chrome anymore
Closed: Jul 2017
Cc:
danakj@chromium.org
thestig@chromium.org
oysteine@chromium.org
thakis@chromium.org
mark@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Process::SetProcessBackgrounded [DCHECK]

Project Member Reported by erikc...@chromium.org, Jul 26 2017 
I appear to be hitting a DCHECK in telemetry_perf_unittests.

"""
  	0033da68 6fba476b 0033da90 6fba4203 71531d6c chrome!base::debug::BreakDebugger+0x9
  	0033da70 6fba4203 71531d6c 0000014c 0b7963f6 chrome!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x27
  	0033da90 6fba475e 6fba4763 0033dc04 0033dc08 chrome!base::internal::FunctorTraits<void (__cdecl*)(char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >),void>::Invoke<char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >+0x20
  	0033daac 6fbc2c78 02b28b90 0033dc04 0033dc08 chrome!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x1a
  	0033e034 6fc1d8f0 0b75d754 0b75d748 00000003 chrome!logging::LogMessage::~LogMessage+0x328
  	0033e0f8 6fc1d92b 00000848 0033e1d0 70be2ece chrome!base::ProcessMetrics::ProcessMetrics+0x90
  	0033e104 70be2ece 0033e1d8 00000848 0033e3a8 chrome!base::ProcessMetrics::CreateProcessMetrics+0x1b
  	0033e1d0 70baae0c 0033e3a8 00000001 71cd59a0 chrome!performance_monitor::ProcessMetricsHistory::Initialize+0x5e
  	0033e2c4 70baac94 0033e3a8 00000001 00000002 chrome!performance_monitor::PerformanceMonitor::MarkProcessAsAlive+0xf9
  	0033e3b8 6fc023e5 08003118 00000002 cee467b1 chrome!performance_monitor::PerformanceMonitor::GatherMetricsMapOnUIThread+0xd9
  	0033e3e0 6fc5bc3e 08003168 0033ec8c 719a4008 chrome!base::Timer::RunScheduledTask+0xb5
  	0033e4e0 6fc05ace 7152fd7c 0033ec8c cee467b1 chrome!base::debug::TaskAnnotator::RunTask+0x18e
  	0033ec70 6fc04838 0033ec8c 02b27128 00000002 chrome!base::MessageLoop::RunTask+0x55e
  	0033ec84 6fc04bb8 07ff3f78 08035120 c7bd59b3 chrome!base::MessageLoop::DeferOrRunPendingTask+0x48
  	0033ed30 6fc5d6a5 02b2e228 00000000 02b2e220 chrome!base::MessageLoop::DoDelayedWork+0xd8
  	0033ed64 6fc5de1a 02b2712c 02b27128 02b27128 chrome!base::MessagePumpForUI::DoRunLoop+0x75
  	0033ed8c 6fc0555b 02b27128 00000000 0033f070 chrome!base::MessagePumpWin::Run+0x4a
  	0033ee58 6fc2e06c 00000000 02b26848 00000000 chrome!base::MessageLoop::Run+0x6b
  	0033efac 6fb62420 00000000 02b34788 71459680 chrome!base::RunLoop::Run+0x9c
  	0033f098 6f615caa 02b34798 71459db0 02b2c710 chrome!ChromeBrowserMainParts::MainMessageLoopRun+0xeb
  	0033f0ac 6f616ea6 ffffffff 02b2c710 00000000 chrome!content::BrowserMainLoop::RunMainMessageLoopParts+0x50
  	0033f174 6f612774 00000000 02978850 02b0af48 chrome!content::BrowserMainRunnerImpl::Run+0x98
  	0033f18c 6fad12fc 0033f324 00000000 02b00000 chrome!content::BrowserMain+0x82
  	0033f250 6fad121b 0033f33c 0033f324 0033f5b0 chrome!content::RunNamedProcessTypeMain+0xb0
  	0033f358 6fae9051 001b8a08 00000000 001bb940 chrome!content::ContentMainRunnerImpl::Run+0x118
  	0033f570 6fad0aa2 0033f59c 00000000 714f9654 chrome!service_manager::Main+0x22e
  	0033f5a0 6f30a6bb 0033f5d4 00000000 713cf37c chrome!content::ContentMain+0x27
"""

See https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/497757

cc-ing some base owners.

Comment 1 by danakj@chromium.org, Jul 26 2017 

From a patch you're writing?

Comment 2 by erikc...@chromium.org, Jul 26 2017 

The CL appears fairly unrelated:

https://chromium-review.googlesource.com/c/578482

Comment 3 by danakj@chromium.org, Jul 26 2017 

is this happening in TOT on the waterfall? can you help find a culprit?

Comment 4 by erikc...@chromium.org, Jul 26 2017 

This error occurs non-deterministically on waterfall. I filed a bug much like I would file a bug if I ran into a DCHECK from a local build.

Comment 5 by danakj@chromium.org, Jul 27 2017 

There's a number of failures visible on http://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWgsSBUZsYWtlIk9iZW5jaG1hcmtzLmJlbmNobWFya19zbW9rZV91bml0dGVzdC5CZW5jaG1hcmtTbW9rZVRlc3QudGFiX3N3aXRjaGluZy50eXBpY2FsXzI1DA

The first is https://luci-milo.appspot.com/buildbot/tryserver.chromium.win/win_chromium_rel_ng/438028:

  Stack Trace:
  ********************************************************************************
  	Last event: 7d0.dbc: Break instruction exception - code 80000003 (first/second chance not available)
  	  debugger time: Mon May  8 01:32:37.336 2017 (UTC - 7:00)
  	ChildEBP RetAddr  Args to Child              
  	001ddf78 6f40795c 001ddfa0 6f4073fe 70cdc110 chrome_6eb50000!base::debug::BreakDebugger+0x9
  	001ddf80 6f4073fe 70cdc110 000000f2 0105f213 chrome_6eb50000!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x27
  	001ddfa0 6f40794f 6f407954 001de114 001de118 chrome_6eb50000!base::internal::FunctorTraits<void (__cdecl*)(char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >),void>::Invoke<char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >+0x20
  	001ddfbc 6f427868 01180128 001de114 001de118 chrome_6eb50000!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x1a
  	001de544 6f489034 011551b0 01173048 01057a40 chrome_6eb50000!logging::LogMessage::~LogMessage+0x328
  	001de628 6f3c4c94 70b88010 01173048 01041008 chrome_6eb50000!base::win::RegKey::DeleteKey+0x64
  	001dea18 6f3c31f1 01173048 00000000 00000000 chrome_6eb50000!ChromeBrowserMainParts::SetupFieldTrials+0x5aa
  	001dec9c 6f3c2792 01173048 01155128 00000001 chrome_6eb50000!ChromeBrowserMainParts::PreCreateThreadsImpl+0x967
  	001dee28 6f3aba1e 0117a9c8 712a9a6c 00000000 chrome_6eb50000!ChromeBrowserMainParts::PreCreateThreads+0x72

Second: Logs won't load.

Third: https://luci-milo.appspot.com/buildbot/tryserver.chromium.win/win_chromium_x64_rel_ng/421604

  Stack Trace:
  ********************************************************************************
  	Last event: 790.a58: Break instruction exception - code 80000003 (first/second chance not available)
  	  debugger time: Mon May  8 23:31:20.008 2017 (UTC - 7:00)
  	RetAddr           : Args to Child                                                           : Call Site
  	000007fe`ee1c6f08 : 00000000`00000000 000007fe`efa1447b 00000000`0023d690 00000000`00000000 : chrome_7feed590000!base::debug::BreakDebugger+0xd
  	000007fe`ee1c6ee6 : 000007fe`f03e3748 000007fe`000000f2 00000000`0023d660 00000000`0023d650 : chrome_7feed590000!?Run@?$Invoker@U?$BindState@P6AXPEBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPEBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPEAVBindStateBase@23@$$QEAPEBD$$QEAH$$QEAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x5c
  	000007fe`ee1f78da : 00000000`00000000 00000000`0023ddd8 000007fe`f0b6d2c0 00000000`0023d688 : chrome_7feed590000!?Run@?$Invoker@U?$BindState@P6AXPEBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPEBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPEAVBindStateBase@23@$$QEAPEBD$$QEAH$$QEAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x3a
  	000007fe`ee273a88 : 00000000`00000000 00000000`00000000 000007fe`f0251f70 00000000`02702930 : chrome_7feed590000!logging::LogMessage::~LogMessage+0x3da
  	000007fe`ee163c4e : 00000000`00000002 00000000`0275d190 00000000`0275d190 00000000`026d3820 : chrome_7feed590000!base::win::RegKey::DeleteKey+0x78
  	000007fe`ee1619be : 00000000`00000000 00000000`02728710 00000000`00000001 00000000`02702930 : chrome_7feed590000!ChromeBrowserMainParts::SetupFieldTrials+0x772
  	000007fe`ee160aea : 00000000`0023ec60 00000000`02702930 00000000`00000000 000007fe`f0b5db70 : chrome_7feed590000!ChromeBrowserMainParts::PreCreateThreadsImpl+0xd7a
  	000007fe`ee13db48 : 00000000`00000000 00000000`0000000b 00000000`02702930 00000000`02702930 : chrome_7feed590000!ChromeBrowserMainParts::PreCreateThreads+0x9a
  	000007fe`eda383bd : 00000000`00000030 000007fe`f0b5db70 00000000`0270a5d0 00000000`0023eda9 : chrome_7feed590000!ChromeBrowserMainPartsWin::PreCreateThreads+0x240

Fourth: https://luci-milo.appspot.com/buildbot/tryserver.chromium.win/win_chromium_rel_ng/439375

  Stack Trace:
  ********************************************************************************
  	Last event: d40.690: Break instruction exception - code 80000003 (first/second chance not available)
  	  debugger time: Tue May  9 04:08:21.814 2017 (UTC - 7:00)
  	ChildEBP RetAddr  Args to Child              
  	0045e230 6c85b436 0045e258 6c85aed8 6e12e2e0 chrome_6bfa0000!base::debug::BreakDebugger+0x9
  	0045e238 6c85aed8 6e12e2e0 000000f2 02582adb chrome_6bfa0000!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x27
  	0045e258 6c85b429 6c85b42e 0045e3cc 0045e3d0 chrome_6bfa0000!base::internal::FunctorTraits<void (__cdecl*)(char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >),void>::Invoke<char const *,int,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > >+0x20
  	0045e274 6c87b8c8 026e1ec0 0045e3cc 0045e3d0 chrome_6bfa0000!?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@base@@SAXPAVBindStateBase@23@$$QAPBD$$QAH$$QAV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@3@Z+0x1a
  	0045e7fc 6c8dc7d4 026ca900 026ccae0 025791e0 chrome_6bfa0000!logging::LogMessage::~LogMessage+0x328
  	0045e8e0 6c81845c 6dfda040 026ccae0 026d7ef8 chrome_6bfa0000!base::win::RegKey::DeleteKey+0x64
  	0045ecd0 6c8169c3 026ccae0 00000000 00000000 chrome_6bfa0000!ChromeBrowserMainParts::SetupFieldTrials+0x5aa
  	0045ef54 6c815f64 026ccae0 026ca878 00000001 chrome_6bfa0000!ChromeBrowserMainParts::PreCreateThreadsImpl+0x967
  	0045f0e0 6c7fed54 026f6e98 6e6fea6c 00000000 chrome_6bfa0000!ChromeBrowserMainParts::PreCreateThreads+0x72
  	0045f138 6c321e76 00000000 026cfa10 026cf9e0 chrome_6bfa0000!ChromeBrowserMainPartsWin::PreCreateThreads+0x129

Comment 6 by danakj@chromium.org, Jul 27 2017 

For some reason try-flakes doesn't show build win_chromium_rel_ng/builds/497757 at all.

Comment 7 by danakj@chromium.org, Jul 27 2017

Mergedinto: 749504
Status: Duplicate (was: Untriaged)
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=telemetry_perf_unittests&tests=benchmarks%2Fbenchmark_smoke_unittest%2FBenchmarkSmokeTest%2Ftab_switching%2Ftypical_25

Shows the test being incredibly flaky too across many bots. Example: https://build.chromium.org/p/chromium.fyi/builders/Mojo%20ChromiumOS/builds/20387

 	Thread 7 (crashed)
  	 0  libc-2.19.so + 0x36c37
  	 1  test_chrome!~LogMessage [callback.h : 80 + 0x8]
  	 2  test_chrome!IsValid [weak_ptr.h : 117 + 0x8]
  	 3  test_chrome!Run [weak_ptr.h : 143 + 0x5]
  	 4  test_chrome!RunTaskAndPostReply [callback.h : 91 + 0x3]
  	 5  test_chrome!RunTask [callback.h : 91 + 0x3]
  	 6  test_chrome!PerformRunTask [task_tracker.cc : 335 + 0xf]
  	 7  test_chrome!PerformRunTask [task_tracker_posix.cc : 22 + 0xb]
  	 8  test_chrome!RunNextTask [task_tracker.cc : 251 + 0x8]
  	 9  test_chrome!ThreadMain [scheduler_worker.cc : 84 + 0x8]
  	10  test_chrome!ThreadFunc [platform_thread_posix.cc : 71 + 0x8]

I think there's something bad going on with these telemetry tests. There's a few other bugs open, I will dupe.

Comment 8 by erikc...@chromium.org, Jul 27 2017

Status: Untriaged (was: Duplicate)
Please don't dup this into that bug.

telemetry_perf_unittests is flaky. This is one of several flakes observed. Part of the problem is that telemetry runs end-to-end chromium builds with DCHECKs enabled [which no other test suite does]. 

This is a real DCHECK that is being hit, and needs to be examined on its own. It's not a recent regression, hence cc-ing several base/ OWNERs.

Comment 9 by danakj@chromium.org, Jul 27 2017

Cc: thestig@chromium.org oysteine@chromium.org
Owner: stanisc@chromium.org
Status: Assigned (was: Untriaged)
Ok. Assigning to the author of the DCHECK. I don't know if this implies the ProcessMetrics is being misued or not. MSDN isn't particularly clear about this to me.

+thestig and oysteine who worked on ProcessMetricsHistory which is calling to here.

Comment 10 by oysteine@chromium.org, Jul 27 2017 

I don't think this code has changed in ~3.5 years, has this DCHECK been firing randomly all that time?

Comment 11 by danakj@chromium.org, Jul 27 2017 

Possibly. Do you know if that means we're using ::DuplicateHandle() wrong, or ProcessMetrics wrong?

Comment 12 by stanisc@chromium.org, Jul 27 2017 

The DuplicateHandle code is relatively new. 

The only two reasons for the failure I can think of is that either the process handle becomes invalid by the time this function is called (i.e. there is a race condition with that process being destroyed) or perhaps the system runs out of handles. The latter is unlikely unless there is a handle leak somewhere.

I guess a good first step would be to log the windows error before the DCHECK.

Comment 13 by erikc...@chromium.org, Jul 29 2017

Status: WontFix (was: Assigned)
This is user error on my part.  

base::Process(handle) takes ownership of the handle on Windows, but not POSIX. This was causing a double-close of the handle for thec urrent process.
►

Sign in to add a comment