New issue
Advanced search Search tips

Issue 806586 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

ContentVerifierTest.FailOnRead is Flaky

Project Member Reported by Findit, Jan 28 2018

Issue description

log

[ RUN      ] ContentVerifierTest.FailOnRead
Xlib:  extension "RANDR" missing on display ":99".
Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf
[14569:14607:0128/061441.733270:ERROR:bus.cc(394)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[14569:14569:0128/061441.765158:WARNING:password_store_factory.cc(241)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options.
[14569:14569:0128/061443.391195:WARNING:chrome_content_verifier_delegate.cc(206)] Disabling extension npnbmohejbjohgpjnmjagbafnjhkmgko ('Test Extension') due to content verification failure. In tests you might want to use a ScopedIgnoreContentVerifierForTest instance to prevent this.
[14569:14597:0128/061443.393823:FATAL:content_verify_job.cc(192)] Check failed: !failed_.
#0 0x7ff7fa70ab9d base::debug::StackTrace::StackTrace()
#1 0x7ff7fa708fdc base::debug::StackTrace::StackTrace()
#2 0x7ff7fa78ff0a logging::LogMessage::~LogMessage()
#3 0x0000041e0f2d extensions::ContentVerifyJob::OnHashesReady()
#4 0x0000041e3607 _ZN4base8internal13FunctorTraitsIMN10extensions16ContentVerifyJobEFvbEvE6InvokeIRK13scoped_refptrIS3_EJbEEEvS5_OT_DpOT0_
#5 0x0000041e355f _ZN4base8internal12InvokeHelperILb0EvE8MakeItSoIRKMN10extensions16ContentVerifyJobEFvbEJRK13scoped_refptrIS5_EbEEEvOT_DpOT0_
#6 0x0000041e34f0 _ZN4base8internal7InvokerINS0_9BindStateIMN10extensions16ContentVerifyJobEFvbEJ13scoped_refptrIS4_EEEEFvbEE7RunImplIRKS6_RKNSt3__15tupleIJS8_EEEJLm0EEEEvOT_OT0_NSF_16integer_sequenceImJXspT1_EEEEOb
#7 0x0000041e3483 _ZN4base8internal7InvokerINS0_9BindStateIMN10extensions16ContentVerifyJobEFvbEJ13scoped_refptrIS4_EEEEFvbEE3RunEPNS0_13BindStateBaseEb
#8 0x000001f5607d _ZNO4base12OnceCallbackIFvbEE3RunEb
#9 0x000003d5e34c base::internal::ReplyAdapter<>()
#10 0x000003d5827d _ZN4base8internal13FunctorTraitsIPFvNS_12OnceCallbackIFNSt3__110unique_ptrINS_5ValueENS3_14default_deleteIS5_EEEEvEEEPS8_EvE6InvokeIJSA_SB_EEEvSD_DpOT_
#11 0x000003d58220 _ZN4base8internal12InvokeHelperILb0EvE8MakeItSoIPFvNS_12OnceCallbackIFNSt3__110unique_ptrINS_5ValueENS5_14default_deleteIS7_EEEEvEEEPSA_EJSC_SD_EEEvOT_DpOT0_
#12 0x000003d58722 _ZN4base8internal7InvokerINS0_9BindStateIPFvNS_12OnceCallbackIFvNSt3__110unique_ptrINS_5ValueENS4_14default_deleteIS6_EEEEEEEPS9_EJSB_NS0_12OwnedWrapperIS9_EEEEEFvvEE7RunImplISE_NS4_5tupleIJSB_SG_EEEJLm0ELm1EEEEvOT_OT0_NS4_16integer_sequenceImJXspT1_EEEE
#13 0x000003d58639 _ZN4base8internal7InvokerINS0_9BindStateIPFvNS_12OnceCallbackIFvNSt3__110unique_ptrINS_5ValueENS4_14default_deleteIS6_EEEEEEEPS9_EJSB_NS0_12OwnedWrapperIS9_EEEEEFvvEE7RunOnceEPNS0_13BindStateBaseE
#14 0x7ff7fa6b9f41 _ZNO4base12OnceCallbackIFvvEE3RunEv
#15 0x7ff7fa90f24b base::(anonymous namespace)::PostTaskAndReplyRelay::RunReplyAndSelfDestruct()
#16 0x7ff7fa90f4ad _ZN4base8internal13FunctorTraitsIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEvE6InvokeIPS3_JEEEvS5_OT_DpOT0_
#17 0x7ff7fa90f3f4 _ZN4base8internal12InvokeHelperILb0EvE8MakeItSoIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEJPS5_EEEvOT_DpOT0_
#18 0x7ff7fa90f3a5 _ZN4base8internal7InvokerINS0_9BindStateIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEJNS0_17UnretainedWrapperIS4_EEEEEFvvEE7RunImplIS6_NSt3__15tupleIJS8_EEEJLm0EEEEvOT_OT0_NSD_16integer_sequenceImJXspT1_EEEE
#19 0x7ff7fa90f2f9 _ZN4base8internal7InvokerINS0_9BindStateIMNS_12_GLOBAL__N_121PostTaskAndReplyRelayEFvvEJNS0_17UnretainedWrapperIS4_EEEEEFvvEE7RunOnceEPNS0_13BindStateBaseE
#20 0x7ff7fa6b9f41 _ZNO4base12OnceCallbackIFvvEE3RunEv
#21 0x7ff7fa70e87a base::debug::TaskAnnotator::RunTask()
#22 0x7ff7fa7ac829 base::internal::IncomingTaskQueue::RunTask()
#23 0x7ff7fa7b595e base::MessageLoop::RunTask()
#24 0x7ff7fa7b5bf8 base::MessageLoop::DeferOrRunPendingTask()
#25 0x7ff7fa7b5f40 base::MessageLoop::DoWork()
#26 0x7ff7fa7bbcd9 base::MessagePumpLibevent::Run()
#27 0x7ff7fa7b50f2 base::MessageLoop::Run()
#28 0x7ff7fa8663f0 base::RunLoop::Run()
#29 0x7ff7fa927ef8 base::Thread::Run()
#30 0x7ff7f2d88ac6 content::BrowserThreadImpl::IOThreadRun()
#31 0x7ff7f2d88dbf content::BrowserThreadImpl::Run()
#32 0x7ff7fa928ad3 base::Thread::ThreadMain()
#33 0x7ff7fa90e821 base::(anonymous namespace)::ThreadFunc()
#34 0x7ff7faab2184 start_thread
#35 0x7ff7dff41ffd clone
[ RUN      ] ContentVerifierTest.FailOnRead
The failing DCHECK has been added in the identified CL https://chromium-review.googlesource.com/c/chromium/src/+/879961. Reverting the CL.
Cc: rdevlin....@chromium.org
Labels: -Pri-3 Pri-1 Type-Bug
Owner: lazyboy@chromium.org
Status: Assigned (was: Available)
The revert is https://chromium-review.googlesource.com/c/chromium/src/+/893258.

Please have a look and reland.
Project Member

Comment 4 by bugdroid1@chromium.org, Jan 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2181b01c2d22784987e7e166b97718c0d74a981a

commit 2181b01c2d22784987e7e166b97718c0d74a981a
Author: vitaliii <vitaliii@chromium.org>
Date: Tue Jan 30 13:45:26 2018

Revert "Fix extraneous call to ContentVerifyJob::DispatchFailureCallback."

This reverts commit 6e548ee1ca4829c87752ef04b2d97ad15a3ff3f9.

Reason for revert: DCHECK at content_verify_job.cc:192 occasionally fails (see  crbug.com/806586 ).

Original change's description:
> Fix extraneous call to ContentVerifyJob::DispatchFailureCallback.
> 
> If ContentVerifyJob receives hashes (ContentVerifyJob::OnHashesReady)
> after the content is read (ContentVerifyJob::DoneReading), then it
> was possible for DispatchFailureCallback to be called more than once.
> This causes a DCHECK failure in DispatchFailureCallback
> (DCHECK(!failed_)).
> 
> Bail out early in OnHashesReady if call to ContentVerifyJob::BytesRead
> leads to a ContentVerifyJob failure.
> 
> Note that this isn't harmful in Release builds as
> DispatchFailureCallback tests ContentVerifyJob::failure_callback_
> before calling the callback, which is reset once it is called.
> Nevertheless, this is incorrect behavior and this CL fixes that.
> 
> 
> Bug:  804630 
> Test: See bug description for repro steps.
> Change-Id: Icf3a5c8b4c0d01cb20e02de14b11aca4aeff03e5
> Reviewed-on: https://chromium-review.googlesource.com/879961
> Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
> Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#531755}

TBR=lazyboy@chromium.org,rdevlin.cronin@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  804630 , 806586 
Change-Id: Ia5c67f60e95ddb607cb364c745f31dde465d3c4f
Reviewed-on: https://chromium-review.googlesource.com/893258
Reviewed-by: vitaliii <vitaliii@chromium.org>
Commit-Queue: vitaliii <vitaliii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#532854}
[modify] https://crrev.com/2181b01c2d22784987e7e166b97718c0d74a981a/extensions/browser/content_verifier/content_hash.h
[modify] https://crrev.com/2181b01c2d22784987e7e166b97718c0d74a981a/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/2181b01c2d22784987e7e166b97718c0d74a981a/extensions/browser/content_verify_job_unittest.cc
[modify] https://crrev.com/2181b01c2d22784987e7e166b97718c0d74a981a/extensions/common/constants.cc
[modify] https://crrev.com/2181b01c2d22784987e7e166b97718c0d74a981a/extensions/common/constants.h

Labels: -Sheriff-Chromium
Project Member

Comment 6 by Findit, Jan 30 2018

Findit identified the culprit r531755 with confidence 80.5% in the config "chromium.linux / Linux Tests (dbg)(1)"
based on the flakiness trend:

https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVypgELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCJwY2hyb21pdW0ubGludXgvTGludXggVGVzdHMgKGRiZykoMSkvNjk5MDcvbmV0d29ya19zZXJ2aWNlX2Jyb3dzZXJfdGVzdHMvUTI5dWRHVnVkRlpsY21sbWFXVnlWR1Z6ZEM1R1lXbHNUMjVTWldGawwLEhNNYXN0ZXJGbGFrZUFuYWx5c2lzGAIM

Automatically posted by the findit-for-me app (https://goo.gl/Ot9f7N).
Flake Analyzer is in alpha version.
Feedback is welcome using component Tools>Test>FindIt>Flakiness !
Status: Started (was: Assigned)
I'm in the process of resubmitting the CL here after taking care of the issues.

CL: https://chromium-review.googlesource.com/c/chromium/src/+/894824

The test in question was in the process of being removed b/c of orthogonal issues https://chromium-review.googlesource.com/c/chromium/src/+/888327
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/15f07945f50fd50f710cdb8a0191ac2a83050087

commit 15f07945f50fd50f710cdb8a0191ac2a83050087
Author: Istiaque Ahmed <lazyboy@chromium.org>
Date: Wed Jan 31 05:15:03 2018

Revert "Revert "Fix extraneous call to ContentVerifyJob::DispatchFailureCallback.""

This reverts commit 2181b01c2d22784987e7e166b97718c0d74a981a.

Reason for revert: The offending test
(ContentVerifierTest.FailOnRead) has been taken care of in
CL: https://chromium-review.googlesource.com/c/chromium/src/+/888327

Original change's description:
> Revert "Fix extraneous call to ContentVerifyJob::DispatchFailureCallback."
>
> This reverts commit 6e548ee1ca4829c87752ef04b2d97ad15a3ff3f9.
>
> Reason for revert: DCHECK at content_verify_job.cc:192 occasionally fails (see  crbug.com/806586 ).
>
> Original change's description:
> > Fix extraneous call to ContentVerifyJob::DispatchFailureCallback.
> >
> > If ContentVerifyJob receives hashes (ContentVerifyJob::OnHashesReady)
> > after the content is read (ContentVerifyJob::DoneReading), then it
> > was possible for DispatchFailureCallback to be called more than once.
> > This causes a DCHECK failure in DispatchFailureCallback
> > (DCHECK(!failed_)).
> >
> > Bail out early in OnHashesReady if call to ContentVerifyJob::BytesRead
> > leads to a ContentVerifyJob failure.
> >
> > Note that this isn't harmful in Release builds as
> > DispatchFailureCallback tests ContentVerifyJob::failure_callback_
> > before calling the callback, which is reset once it is called.
> > Nevertheless, this is incorrect behavior and this CL fixes that.
> >
> >
> > Bug:  804630 
> > Test: See bug description for repro steps.
> > Change-Id: Icf3a5c8b4c0d01cb20e02de14b11aca4aeff03e5
> > Reviewed-on: https://chromium-review.googlesource.com/879961
> > Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
> > Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#531755}
>
> TBR=lazyboy@chromium.org,rdevlin.cronin@chromium.org
>
> # Not skipping CQ checks because original CL landed > 1 day ago.
>
> Bug:  804630 , 806586 
> Change-Id: Ia5c67f60e95ddb607cb364c745f31dde465d3c4f
> Reviewed-on: https://chromium-review.googlesource.com/893258
> Reviewed-by: vitaliii <vitaliii@chromium.org>
> Commit-Queue: vitaliii <vitaliii@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#532854}

TBR=lazyboy@chromium.org,rdevlin.cronin@chromium.org,vitaliii@chromium.org

Change-Id: I82988d836f49033e1d618125e06f0d45d1124071
Bug:  804630 ,  806586 
Reviewed-on: https://chromium-review.googlesource.com/894824
Commit-Queue: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533209}
[modify] https://crrev.com/15f07945f50fd50f710cdb8a0191ac2a83050087/extensions/browser/content_verifier/content_hash.h
[modify] https://crrev.com/15f07945f50fd50f710cdb8a0191ac2a83050087/extensions/browser/content_verify_job.cc
[modify] https://crrev.com/15f07945f50fd50f710cdb8a0191ac2a83050087/extensions/browser/content_verify_job_unittest.cc
[modify] https://crrev.com/15f07945f50fd50f710cdb8a0191ac2a83050087/extensions/common/constants.cc
[modify] https://crrev.com/15f07945f50fd50f710cdb8a0191ac2a83050087/extensions/common/constants.h

Status: Fixed (was: Started)

Sign in to add a comment