New issue
Advanced search Search tips

Issue 919929 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Jan 15
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug-Regression
Proj-Servicification



Sign in to add a comment

AndroidStreamReaderURLLoaderTest.ReadFakeStream and ValidRangeRequest failing on android-marshmallow-arm64-rel

Project Member Reported by kbr@chromium.org, Jan 8

Issue description

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel?limit=200

Example failures:
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/162888
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/162886
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/162883
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/162882

Log excerpt:


[FATAL:url_loader.mojom.cc(816)] The outgoing message will trigger VALIDATION_ERROR_UNEXPECTED_INVALID_HANDLE at the receiving side (invalid body in URLLoaderClient.OnStartLoadingResponseBody request).

[ERROR:test_suite.cc(325)] Currently running: AndroidStreamReaderURLLoaderTest.ReadFakeStream
Reading Android symbols from: /b/swarming/w/ir
Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release
Searching for native crashes in: /b/swarming/w/it0_JRk8/tmpHib78t
Unknown Android release, consider passing --packed-lib.
Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release

Stack Trace:
  RELADDR   FUNCTION                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  FILE:LINE
  000000000422bc73  logging::LogMessage::~LogMessage()                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        ??:0:0
  000000000252235f  network::mojom::URLLoaderClientProxy::OnStartLoadingResponseBody(mojo::ScopedHandleBase<mojo::DataPipeConsumerHandle>)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ??:0:0
  00000000021aeecf  android_webview::AndroidStreamReaderURLLoader::SendBody()                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ??:0:0
  00000000021aecd3  android_webview::AndroidStreamReaderURLLoader::HeadersComplete(int, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                ??:0:0
  00000000021aee17  android_webview::AndroidStreamReaderURLLoader::OnReaderSeekCompleted(int)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ??:0:0
  00000000021a491f  base::internal::Invoker<base::internal::BindState<void (android_webview::AndroidStreamReaderURLRequestJob::*)(int), base::WeakPtr<android_webview::AndroidStreamReaderURLRequestJob> >, void (int)>::RunOnce(base::internal::BindStateBase*, int)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         ??:0:0
  0000000002172dc7  base::OnceCallback<void (content::CertificateRequestResultType)>::Run(content::CertificateRequestResultType) &&                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           ??:0:0
  00000000021a4acf  void base::internal::ReplyAdapter<int, int>(base::OnceCallback<void (int)>, std::__ndk1::unique_ptr<int, std::__ndk1::default_delete<int> >*)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ??:0:0
  0000000002177ee7  void base::internal::FunctorTraits<void (*)(base::OnceCallback<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > ()>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), void>::Invoke<void (*)(base::OnceCallback<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > ()>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), base::OnceCallback<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > ()>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*>(void (*&&)(base::OnceCallback<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > ()>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), base::OnceCallback<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > ()>&&, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*&&)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ??:0:0
  00000000021780eb  void base::internal::Invoker<base::internal::BindState<void (*)(base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, base::internal::OwnedWrapper<std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > > > >, void ()>::RunImpl<void (*)(base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), std::__ndk1::tuple<base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, base::internal::OwnedWrapper<std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > > > >, 0ul, 1ul>(void (*&&)(base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > >*), std::__ndk1::tuple<base::OnceCallback<void (std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >)>, base::internal::OwnedWrapper<std::__ndk1::unique_ptr<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> >, std::__ndk1::default_delete<std::__ndk1::unique_ptr<android_webview::AwWebResourceResponse, std::__ndk1::default_delete<android_webview::AwWebResourceResponse> > > > > >&&, std::__ndk1::integer_sequence<unsigned long, 0ul, 1ul>)  ??:0:0
  00000000042a5d97  base::(anonymous namespace)::PostTaskAndReplyRelay::RunReply(base::(anonymous namespace)::PostTaskAndReplyRelay)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          ??:0:0
  00000000042a5e3f  base::internal::Invoker<base::internal::BindState<void (*)(base::(anonymous namespace)::PostTaskAndReplyRelay), base::(anonymous namespace)::PostTaskAndReplyRelay>, void ()>::RunOnce(base::internal::BindStateBase*)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ??:0:0
  000000000421beeb  base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      ??:0:0
  00000000042822e3  base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::TimeTicks*)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       ??:0:0
...

Most of the last 200 builds on this bot have failed with this error. It's causing valid CLs to be rejected from the CQ because the failures are flaky. Example:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/162643
https://chromium-review.googlesource.com/c/chromium/src/+/1398562
 
Cc: timvolod...@chromium.org
Components: Internals>Services>Network
Components: -Internals>Network
Cc: jam@chromium.org rockot@google.com
 Issue 920045  has been merged into this issue.
Owner: jam@chromium.org
From  https://crbug.com/920045 :

"Looks like some mojo error? https://chromium-review.googlesource.com/c/chromium/src/+/1391913 shows up in the first failing build, and reverting locally appears to cause the tests to pass. Assigning to author of CL."
hmm looks like it started failing at build #25820 (Marshmallow tester) [1], however the above CL seems to be a BUILD.gn change only. Perhaps we need to init the ScopedDataPipeConsumerHandle [2] in some other way now?

[1] https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Marshmallow%2064%20bit%20Tester/25820

[2] https://cs.chromium.org/chromium/src/android_webview/browser/net_network_service/android_stream_reader_url_loader.cc?l=195

The BUILD.gn change changes generated code in a significant way.

The difference is almost certainly going to that some structurespassed over an intraprocess Mojom interface is going to be serialized, deserialized, and *validated* now, whereas before it would just moved from place to place with no need for validation.

This means the sender is probably doing something wrong, but the mistake has gone undetected due to lazy serialization.
I ran a bot with and without the revert of "Disable lazy serialization in network interfaces.". AndroidStreamReaderURLLoaderTest.* tests passed with the revert and din't pass without the revert. Therefore, per policy at https://www.chromium.org/developers/tree-sheriffs, I created a revert of the change https://chromium-review.googlesource.com/c/chromium/src/+/1403055 and I'll let owners decide what to do next.
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 9

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/007850b95fe68fc29f2d615500d9a4e31d530da3

commit 007850b95fe68fc29f2d615500d9a4e31d530da3
Author: François Doray <fdoray@chromium.org>
Date: Wed Jan 09 15:20:59 2019

Revert "Disable lazy serialization in network interfaces."

This reverts commit e9bf5c8011e6c35cb361e623726cbe969a7a2d92.

Reason for revert: Tentative fix for  https://crbug.com/919929 .

Original change's description:
> Disable lazy serialization in network interfaces.
>
> Per discussion in https://chromium-review.googlesource.com/c/chromium/src/+/1374957, we don't see a
> performance gain from enabling them. Since this has a binary size cost, disable it.
>
> Bug: 803125
>
> Change-Id: I03090214fdb7ecba9e987065e62ca2a1d6dfd430
> Reviewed-on: https://chromium-review.googlesource.com/c/1391913
> Reviewed-by: Ken Rockot <rockot@google.com>
> Commit-Queue: John Abd-El-Malek <jam@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#620507}

TBR=jam@chromium.org,rockot@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 803125,  919929 
Change-Id: Ib6224258071d6716555fab185f8034f1081a5af3
Reviewed-on: https://chromium-review.googlesource.com/c/1403055
Commit-Queue: François Doray <fdoray@chromium.org>
Reviewed-by: François Doray <fdoray@chromium.org>
Cr-Commit-Position: refs/heads/master@{#621145}
[modify] https://crrev.com/007850b95fe68fc29f2d615500d9a4e31d530da3/services/network/public/mojom/BUILD.gn

right, if there is more validation now and the sender is wrong then perhaps it makes sense to disable the tests instead. I have a patch here: https://chromium-review.googlesource.com/c/chromium/src/+/1402811

jam@: depending on what you prefer I can land that.

Yes please disable the test and reland the revert, thanks.
Project Member

Comment 11 by bugdroid1@chromium.org, Jan 9

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/239c70ab530b157754e05195cbadca1c32ad6262

commit 239c70ab530b157754e05195cbadca1c32ad6262
Author: Tim Volodine <timvolodine@chromium.org>
Date: Wed Jan 09 17:21:04 2019

[AW NS] Disable failing AndroidStreamReaderURLLoaderTests

Failing don Marshmallow bots due to more extensive checking
turned on by:
https://chromium-review.googlesource.com/c/chromium/src/+/1391913

BUG= 919929 
TBR=jam@chromium.org,tobiasjs@chromium.org
NOTRY=true

Change-Id: I4b3d740b50a2205a899b4d78a4108cc70e712adc
Reviewed-on: https://chromium-review.googlesource.com/c/1402811
Reviewed-by: Tim Volodine <timvolodine@chromium.org>
Reviewed-by: John Abd-El-Malek <jam@chromium.org>
Commit-Queue: John Abd-El-Malek <jam@chromium.org>
Cr-Commit-Position: refs/heads/master@{#621196}
[modify] https://crrev.com/239c70ab530b157754e05195cbadca1c32ad6262/android_webview/browser/net_network_service/android_stream_reader_url_loader_unittest.cc

Owner: timvolod...@chromium.org
Status: Assigned (was: Untriaged)
Project Member

Comment 13 by bugdroid1@chromium.org, Jan 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b0041cfd59313062e36ca8cb7334325abd8507a2

commit b0041cfd59313062e36ca8cb7334325abd8507a2
Author: Tim Volodine <timvolodine@chromium.org>
Date: Mon Jan 14 16:15:26 2019

[AW NS] Re-enable AndroidStreamReaderURLLoaderTest tests

Re-enable: ValidRangeRequest and ReadFakeStream tests.

BUG= 919929 
TBR=jam@chromium.org,tobiasjs@chromium.org

Change-Id: Ia5ac10f44ac85ccb52e5c95d052894fd39f1cb8e
Reviewed-on: https://chromium-review.googlesource.com/c/1407197
Reviewed-by: Tim Volodine <timvolodine@chromium.org>
Commit-Queue: Tim Volodine <timvolodine@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622469}
[modify] https://crrev.com/b0041cfd59313062e36ca8cb7334325abd8507a2/android_webview/browser/net_network_service/android_stream_reader_url_loader_unittest.cc

Status: Fixed (was: Assigned)
seems to be fixed now after https://chromium-review.googlesource.com/c/chromium/src/+/1375990

Nice! Thanks for updating.

Comment 16 by sbash...@chromium.org, Jan 16 (6 days ago)

Status: Verified (was: Fixed)
Marking Verified as per comment #14. Please reopen if issue not fixed.Thanks

Sign in to add a comment