New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 690127 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Feb 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression

Blocking:
issue 673543



Sign in to add a comment

Flaky DCHECK in BeginFrameObserverBase::OnBeginFrame during GpuProcess_driver_bug_workarounds_upon_gl_renderer

Project Member Reported by ynovikov@chromium.org, Feb 8 2017

Issue description

Seen here:
https://build.chromium.org/p/chromium.gpu.fyi/builders/Win7%20Debug%20%28AMD%29/builds/1053/steps/gpu_process_launch_tests%20on%20ATI%20GPU%20on%20Windows%20on%20Windows-2008ServerR2-SP1/logs/stdio

[1656:3264:0208/065407.261:ERROR:process_win.cc(140)] Unable to terminate process: Access is denied. (0x5)
[1656:3244:0208/065407.265:FATAL:begin_frame_source.cc(42)] Check failed: args.sequence_number > last_begin_frame_args_.sequence_number || args.source_id != last_begin_frame_args_.source_id. 
Backtrace:
	base::debug::StackTrace::StackTrace [0x10094627+55]
	base::debug::StackTrace::StackTrace [0x10094201+17]
	logging::LogMessage::~LogMessage [0x100E064B+59]
	cc::BeginFrameObserverBase::OnBeginFrame [0x15EFBBB1+385]
	cc::ExternalBeginFrameSource::OnBeginFrame [0x15EFBCFA+138]
	cc::DirectCompositorFrameSink::OnBeginFrame [0x16CD2EEE+30]
	cc::CompositorFrameSinkSupport::OnBeginFrame [0x16CCBB25+69]
	cc::DelayBasedBeginFrameSource::AddObserver [0x15EFB206+662]
	cc::CompositorFrameSinkSupport::UpdateNeedsBeginFramesInternal [0x16CCC24C+140]
	cc::CompositorFrameSinkSupport::SetNeedsBeginFrame [0x16CCC0BB+27]
	cc::DirectCompositorFrameSink::OnNeedsBeginFrames [0x16CD2F1E+30]
	cc::ExternalBeginFrameSource::AddObserver [0x15EFB42C+364]
	cc::Scheduler::SetupNextBeginFrameIfNeeded [0x15F0F74B+107]
	cc::Scheduler::ProcessScheduledActions [0x15F0EBBB+731]
	cc::Scheduler::SetDeferCommits [0x15F0F3D4+196]
	cc::SingleThreadProxy::SetDeferCommits [0x160BE6D1+529]
	cc::LayerTreeHost::SetDeferCommits [0x15FD32A9+41]
	ui::Compositor::UnlockCompositor [0x17901040+160]
	ui::CompositorLock::CancelLock [0x178FEC6D+29]
	??$Invoke@ABV?$WeakPtr@VCompositorLock@ui@@@base@@$$V@?$FunctorTraits@P8CompositorLock@ui@@AEXXZX@internal@base@@SAXP8CompositorLock@ui@@AEXXZABV?$WeakPtr@VCompositorLock@ui@@@2@@Z [0x178F2B48+24]
	??$MakeItSo@ABQ8CompositorLock@ui@@AEXXZABV?$WeakPtr@VCompositorLock@ui@@@base@@$$V@?$InvokeHelper@$00X@internal@base@@SAXABQ8CompositorLock@ui@@AEXXZABV?$WeakPtr@VCompositorLock@ui@@@2@@Z [0x178F2EB9+57]
	base::internal::Invoker<base::internal::BindState<void (__thiscall ui::CompositorLock::*)(void),base::WeakPtr<ui::CompositorLock> >,void __cdecl(void)>::RunImpl<void (__thiscall ui::CompositorLock::*const &)(void),std::tuple<base::WeakPtr<ui::CompositorLo [0x178F2F74+52]
	base::internal::Invoker<base::internal::BindState<void (__thiscall ui::CompositorLock::*)(void),base::WeakPtr<ui::CompositorLock> >,void __cdecl(void)>::Run [0x17900354+36]
	base::internal::RunMixin<base::Callback<void __cdecl(void),0,0> >::Run [0x1009A134+68]
	base::debug::TaskAnnotator::RunTask [0x1009A2CF+367]
	base::MessageLoop::RunTask [0x101104F6+614]
	base::MessageLoop::DeferOrRunPendingTask [0x1010E59C+44]
	base::MessageLoop::DoDelayedWork [0x1010E94C+316]
	base::MessagePumpForUI::DoRunLoop [0x10118480+128]
	base::MessagePumpWin::Run [0x101193DB+123]
	base::MessageLoop::RunHandler [0x1011021E+398]
	base::RunLoop::Run [0x101C3CB6+166]
	ChromeBrowserMainParts::MainMessageLoopRun [0x04C89A08+248]
	content::BrowserMainLoop::RunMainMessageLoopParts [0x1190D351+225]
	content::BrowserMainRunnerImpl::Run [0x11913C33+243]
	content::BrowserMain [0x118FD26B+155]
	content::RunNamedProcessTypeMain [0x13270537+135]
	content::ContentMainRunnerImpl::Run [0x13270415+405]
	content::ContentMain [0x1326E414+100]
	ChromeMain [0x036C8F30+256]
	MainDllLoader::Launch [0x0042ED64+1012]
	wWinMain [0x00429AE6+566]
	invoke_main [0x004E7FCE+30] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:118)
	__scrt_common_main_seh [0x004E7E30+336] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:253)
	__scrt_common_main [0x004E7CCD+13] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:296)
	wWinMainCRTStartup [0x004E7FE8+8] (f:\dd\vctools\crt\vcstartup\src\startup\exe_wwinmain.cpp:17)
	BaseThreadInitThunk [0x7611337A+18]
	RtlInitializeExceptionChain [0x777792B2+99]
	RtlInitializeExceptionChain [0x77779285+54]

Fuller log attached.

brianderson@, can you please triage this as the current pixel wrangler?

 
log.txt
80.7 KB View Download
Cc: briander...@chromium.org
Owner: eseckler@chromium.org
Eric, can you take a look?
This appears to be the one of the top flakes when combined across the various tests that are flaking: http://chromium-try-flakes.appspot.com

I wonder why it only started to show up recently. The CHECK has been in for a while.
Labels: -Type-Bug -Pri-3 M-57 Pri-1 Type-Bug-Regression
I guess this deserves a higher priority, then.
Hm, I can't spot with certainty what might be going wrong here :)

CompositorFrameSinkSupport::OnBeginFrame shouldn't let the same BFArgs through to the ExternalBFS twice, because it updates LastUsedBeginFrameArgs and that's checked by DelayBasedBeginFrameSource::AddObserver.

Could the cc::Scheduler have been attached to a different CompositorFrameSink before, which is fed by the same DelayBasedBFS? In that case, it would be possible to receive the same BFArgs twice in the Scheduler, because the new CompositorFrameSink might not have seen the latest args yet. Maybe we have to check whether observers should receive the BFArgs provided to ExternalBeginFrameSource::OnBeginFrame.

Brian, WDYT, is that possible? I'm happy to add said check. I'll also add some more debugging info about the current/last args to the DCHECKs, so that we get a better indication of what might be going wrong when a DCHECK is hit - since it seems that this sort of thing is hard to reproduce :-/
Project Member

Comment 5 by chromium...@appspot.gserviceaccount.com, Feb 9 2017

Labels: Sheriff-Chromium
Detected 3 new flakes for test/step "CaptivePortalBlockingPageTest.WiredNetwork_LoginURL_With_SSID". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNySAsSBUZsYWtlIj1DYXB0aXZlUG9ydGFsQmxvY2tpbmdQYWdlVGVzdC5XaXJlZE5ldHdvcmtfTG9naW5VUkxfV2l0aF9TU0lEDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Project Member

Comment 6 by chromium...@appspot.gserviceaccount.com, Feb 9 2017

Detected 4 new flakes for test/step "SafeBrowsingBlockingPageIDNTestWithThreatType/SafeBrowsingBlockingPageIDNTest.SafeBrowsingBlockingPageDecodesIDN/1". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfQsSBUZsYWtlInJTYWZlQnJvd3NpbmdCbG9ja2luZ1BhZ2VJRE5UZXN0V2l0aFRocmVhdFR5cGUvU2FmZUJyb3dzaW5nQmxvY2tpbmdQYWdlSUROVGVzdC5TYWZlQnJvd3NpbmdCbG9ja2luZ1BhZ2VEZWNvZGVzSUROLzEM. This message was posted automatically by the chromium-try-flakes app.
Project Member

Comment 7 by chromium...@appspot.gserviceaccount.com, Feb 9 2017

Detected 4 new flakes for test/step "CaptivePortalBlockingPageTest.WiredNetwork_LoginURL". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyPgsSBUZsYWtlIjNDYXB0aXZlUG9ydGFsQmxvY2tpbmdQYWdlVGVzdC5XaXJlZE5ldHdvcmtfTG9naW5VUkwM. This message was posted automatically by the chromium-try-flakes app.
Cc: kylec...@chromium.org staraz@chromium.org
The flakes went away, I suspect because of the reverts of the following two patches which was causing flakes in other places:

https://codereview.chromium.org/2612083002 - DirectCompositorFrameSink Uses CompositorFrameSinkSupport
https://codereview.chromium.org/2687433002 - Move surface reference code to CompositorFrameSinkSupport

+staraz and kylechar so they are aware this might also need to be addressed before relanding. Not sure what the root cause is yet.
Sounds like it was the first of the two (https://codereview.chromium.org/2612083002), because the first flakes occurred before the second one landed.
Most likely it was 2612083002. I relanded 2687433002 a couple hours ago and I don't see any new flakes.
Owner: staraz@chromium.org
@staraz: Do you know if what Eric suspects in #4 is possible after your patch?

Comment 12 by bsep@chromium.org, Feb 9 2017

Labels: -Sheriff-Chromium
It doesn't look like this needs action from sheriffs, removing from the queue.
Blocking: 673543
Cc: eseckler@chromium.org
Owner: ----
Status: Available (was: Assigned)
2612083002 was reverted yesterday. 
Could you confirm if the flakes persists?

https://bugs.chromium.org/p/chromium/issues/detail?id=676070
This bug is about some flaky tests failing the same DCHECKs. Reverting 2612083002 didn't seem to fix the flakes.

I'm marking this available since my CL was reverted.
@staraz: I don't see any new flakes after your patch was reverted, also not in  https://crbug.com/676070 . Sounds like the root cause is the same though.

Can you advise regarding #4?
Ah, I just found https://codereview.chromium.org/2683583005/ which attempted to fix the flakes. From its description:

For  bug 676070 ,  689916  (flaky tests):
The two bugs are caused by CompositorFrameSinkSupport doesn't stop
observing BeginFrameSource when it should. By notifying client on output 
surface lost correctly, DirectCompositorFrameSink detaches from client 
and destroys CompositorFrameSinkSupport (and hence stop observing 
BeginFrameSource) at the right time.

Comment 18 Deleted

Status: Fixed (was: Available)
I'm working on rebasing and splitting the CLs into smaller parts before relanding them.
Tentatively marking the bug as fixed as no new flakes showed up after the revert. Feel free to reopen it if there's new flakes
Owner: staraz@chromium.org
The flakes are gone after my CL was reverted.
Assigning it back to me since I need to figure out how to land that CL again without reintroducing the flakes
Cc: -staraz@chromium.org fsam...@chromium.org
Project Member

Comment 22 by bugdroid1@chromium.org, Feb 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d69ef1ad3f5700af10d560a4912751394b9918cc

commit d69ef1ad3f5700af10d560a4912751394b9918cc
Author: eseckler <eseckler@chromium.org>
Date: Sat Feb 18 13:41:57 2017

Check for BeginFrame time/seqnum continuity in ExternalBFS::OnBF.

BUG= 690127 
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_trusty_blink_rel

Review-Url: https://codereview.chromium.org/2706493002
Cr-Commit-Position: refs/heads/master@{#451459}

[modify] https://crrev.com/d69ef1ad3f5700af10d560a4912751394b9918cc/cc/scheduler/begin_frame_source.cc

Project Member

Comment 23 by bugdroid1@chromium.org, Feb 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0a6d72780a0317bf6d1bfcabc023123e5cdfb454

commit 0a6d72780a0317bf6d1bfcabc023123e5cdfb454
Author: eseckler <eseckler@chromium.org>
Date: Thu Feb 23 12:17:22 2017

[cc] Add test for ExternalBFS::OnBeginFrame continuity check.

As promised in https://codereview.chromium.org/2706493002/.

BUG= 690127 
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_trusty_blink_rel

Review-Url: https://codereview.chromium.org/2715493003
Cr-Commit-Position: refs/heads/master@{#452471}

[modify] https://crrev.com/0a6d72780a0317bf6d1bfcabc023123e5cdfb454/cc/scheduler/begin_frame_source_unittest.cc

Sign in to add a comment