crash in RenderThreadImpl::render_message_filter() when fetching code cache |
||||
Issue descriptionThis is a regression in M71, stats below. 71.0.3551.0 14.71% 5 71.0.3550.3 11.76% 4 71.0.3550.0 17.65% 6 71.0.3549.1 5.88% 2 71.0.3549.0 17.65% 6 Sample report - go/crash/688429763860d391 ============== Thread 24 (id: 0xaafa77) CRASHED [EXC_BAD_ACCESS / KERN_INVALID_ADDRESS @ 0x00000558 ] MAGIC SIGNATURE THREAD Stack Quality84%Show frame trust levels 0x000000010ee3336a (Google Chrome Framework -memory:2619 ) content::RenderThreadImpl::render_message_filter() 0x000000010ee5be6d (Google Chrome Framework -renderer_blink_platform_impl.cc:475 ) content::RendererBlinkPlatformImpl::FetchCachedCode(GURL const&, base::OnceCallback<void (base::Time, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&)>) 0x000000010ed3d1d2 (Google Chrome Framework -code_cache_loader_impl.cc:73 ) content::CodeCacheLoaderImpl::FetchFromCodeCacheImpl(GURL const&, base::OnceCallback<void (base::Time const&, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&)>, base::WaitableEvent*) 0x000000010ed3d221 (Google Chrome Framework -code_cache_loader_impl.cc:63 ) content::CodeCacheLoaderImpl::FetchFromCodeCache(GURL const&, base::OnceCallback<void (base::Time const&, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&)>) 0x000000010a4489a8 (Google Chrome Framework -resource_loader.cc:169 ) blink::ResourceLoader::CodeCacheRequest::FetchFromCodeCache(blink::WebURLLoader*, blink::ResourceLoader*) 0x000000010a44970f (Google Chrome Framework -resource_loader.cc:398 ) blink::ResourceLoader::StartWith(blink::ResourceRequest const&) 0x000000010a446500 (Google Chrome Framework -resource_load_scheduler.cc:682 ) blink::ResourceLoadScheduler::Request(blink::ResourceLoadSchedulerClient*, blink::ResourceLoadScheduler::ThrottleOption, blink::WebURLRequest::Priority, int, unsigned long long*) 0x000000010a449380 (Google Chrome Framework -resource_loader.cc:348 ) blink::ResourceLoader::Start() 0x000000010a43b57a (Google Chrome Framework -resource_fetcher.cc:1757 ) blink::ResourceFetcher::StartLoad(blink::Resource*) 0x000000010a43a5dd (Google Chrome Framework -resource_fetcher.cc:935 ) blink::ResourceFetcher::RequestResource(blink::FetchParameters&, blink::ResourceFactory const&, blink::ResourceClient*, blink::SubstituteData const&) 0x000000010e3bec51 (Google Chrome Framework -script_resource.cc:81 ) blink::ScriptResource::Fetch(blink::FetchParameters&, blink::ResourceFetcher*, blink::ResourceClient*) 0x000000010e3b08c8 (Google Chrome Framework -worklet_module_script_fetcher.cc:41 ) blink::WorkletModuleScriptFetcher::Fetch(blink::FetchParameters&, blink::ModuleGraphLevel, blink::ModuleScriptFetcher::Client*) 0x000000010e3aba9a (Google Chrome Framework -module_script_loader.cc:204 ) blink::ModuleScriptLoader::FetchInternal(blink::ModuleScriptFetchRequest const&, blink::FetchClientSettingsObjectSnapshot*, blink::ModuleGraphLevel, blink::ModuleScriptCustomFetchType) 0x000000010e4c7826 (Google Chrome Framework -module_map.cc:133 ) blink::ModuleMap::FetchSingleModuleScript(blink::ModuleScriptFetchRequest const&, blink::FetchClientSettingsObjectSnapshot*, blink::ModuleGraphLevel, blink::ModuleScriptCustomFetchType, blink::SingleModuleClient*) 0x000000010e3accd3 (Google Chrome Framework -module_tree_linker.cc:231 ) blink::ModuleTreeLinker::FetchRoot(blink::KURL const&, blink::ScriptFetchOptions const&) 0x000000010e4c5fef (Google Chrome Framework -modulator_impl_base.cc:71 ) blink::ModulatorImplBase::FetchTree(blink::KURL const&, blink::FetchClientSettingsObjectSnapshot*, blink::WebURLRequest::RequestContext, blink::ScriptFetchOptions const&, blink::ModuleScriptCustomFetchType, blink::ModuleTreeClient*) 0x000000010e5c3193 (Google Chrome Framework -worker_or_worklet_global_scope.cc:200 ) blink::WorkerOrWorkletGlobalScope::FetchModuleScript(blink::KURL const&, blink::FetchClientSettingsObjectSnapshot*, blink::WebURLRequest::RequestContext, network::mojom::FetchCredentialsMode, blink::ModuleScriptCustomFetchType, blink::ModuleTreeClient*) 0x000000010e5caaa8 (Google Chrome Framework -worklet_global_scope.cc:108 ) blink::WorkletGlobalScope::FetchAndInvokeScript(blink::KURL const&, network::mojom::FetchCredentialsMode, blink::FetchClientSettingsObjectSnapshot*, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*) 0x000000010e5b7f51 (Google Chrome Framework -threaded_worklet_object_proxy.cc:38 ) blink::ThreadedWorkletObjectProxy::FetchAndInvokeScript(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*) 0x000000010e5b7938 (Google Chrome Framework -bind_internal.h:516 ) void base::internal::FunctorTraits<void (blink::ThreadedWorkletObjectProxy::*)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), void>::Invoke<void (blink::ThreadedWorkletObjectProxy::*)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), blink::ThreadedWorkletObjectProxy*, blink::KURL const&, network::mojom::FetchCredentialsMode const&, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner> const&, blink::CrossThreadPersistent<blink::WorkletPendingTasks> const&, blink::WorkerThread*>(void (blink::ThreadedWorkletObjectProxy::*)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), blink::ThreadedWorkletObjectProxy*&&, blink::KURL const&&&, network::mojom::FetchCredentialsMode const&&&, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >&&, scoped_refptr<base::SingleThreadTaskRunner> const&&&, blink::CrossThreadPersistent<blink::WorkletPendingTasks> const&&&, blink::WorkerThread*&&) 0x000000010e5b77de (Google Chrome Framework -bind_internal.h:616 ) void base::internal::Invoker<base::internal::BindState<void (blink::ThreadedWorkletObjectProxy::*)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), WTF::CrossThreadUnretainedWrapper<blink::ThreadedWorkletObjectProxy>, blink::KURL, network::mojom::FetchCredentialsMode, WTF::PassedWrapper<std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> > >, scoped_refptr<base::SingleThreadTaskRunner>, blink::CrossThreadPersistent<blink::WorkletPendingTasks>, WTF::CrossThreadUnretainedWrapper<blink::WorkerThread> >, void ()>::RunImpl<void (blink::ThreadedWorkletObjectProxy::* const&)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), std::__1::tuple<WTF::CrossThreadUnretainedWrapper<blink::ThreadedWorkletObjectProxy>, blink::KURL, network::mojom::FetchCredentialsMode, WTF::PassedWrapper<std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> > >, scoped_refptr<base::SingleThreadTaskRunner>, blink::CrossThreadPersistent<blink::WorkletPendingTasks>, WTF::CrossThreadUnretainedWrapper<blink::WorkerThread> > const&, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul>(void (blink::ThreadedWorkletObjectProxy::* const&&&)(blink::KURL const&, network::mojom::FetchCredentialsMode, std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> >, scoped_refptr<base::SingleThreadTaskRunner>, blink::WorkletPendingTasks*, blink::WorkerThread*), std::__1::tuple<WTF::CrossThreadUnretainedWrapper<blink::ThreadedWorkletObjectProxy>, blink::KURL, network::mojom::FetchCredentialsMode, WTF::PassedWrapper<std::__1::unique_ptr<blink::CrossThreadFetchClientSettingsObjectData, std::__1::default_delete<blink::CrossThreadFetchClientSettingsObjectData> > >, scoped_refptr<base::SingleThreadTaskRunner>, blink::CrossThreadPersistent<blink::WorkletPendingTasks>, WTF::CrossThreadUnretainedWrapper<blink::WorkerThread> > const&&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul>) 0x000000010da9757b (Google Chrome Framework -callback.h:140 ) blink::(anonymous namespace)::RunCrossThreadClosure(WTF::CrossThreadFunction<void ()>) 0x000000010da97c02 (Google Chrome Framework -bind_internal.h:416 ) base::internal::Invoker<base::internal::BindState<void (*)(WTF::CrossThreadFunction<void ()>), WTF::CrossThreadFunction<void ()> >, void ()>::RunOnce(base::internal::BindStateBase*) 0x000000010aa1a171 (Google Chrome Framework -callback.h:99 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) 0x000000010aa80e23 (Google Chrome Framework -thread_controller_impl.cc:196 ) base::sequence_manager::internal::ThreadControllerImpl::DoWork(base::sequence_manager::internal::ThreadControllerImpl::WorkType) 0x000000010aa1a171 (Google Chrome Framework -callback.h:99 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) 0x000000010aa3596d (Google Chrome Framework -message_loop.cc:434 ) base::MessageLoop::RunTask(base::PendingTask*) 0x000000010aa35cd2 (Google Chrome Framework -message_loop.cc:445 ) base::MessageLoop::DoWork() 0x000000010aa37558 (Google Chrome Framework -message_pump_default.cc:37 ) base::MessagePumpDefault::Run(base::MessagePump::Delegate*) 0x000000010aa5a3d4 (Google Chrome Framework -run_loop.cc:102 ) <name omitted> 0x000000010aa9d660 (Google Chrome Framework -thread.cc:357 ) base::Thread::ThreadMain() 0x000000010aacf976 (Google Chrome Framework -platform_thread_posix.cc:76 ) base::(anonymous namespace)::ThreadFunc(void*) 0x00007fff5be9e660 (libsystem_pthread.dylib + 0x00003660 ) _pthread_body 0x00007fff5be9e50c (libsystem_pthread.dylib + 0x0000350c ) _pthread_start 0x00007fff5be9dbf8 (libsystem_pthread.dylib + 0x00002bf8 ) thread_start 0x000000010aacf91f (Google Chrome Framework + 0x0254491f ) The stack frame seem to be slightly different, but the root cause seems to be same. Manual regression Range ======================= https://crash.corp.google.com/browse?q=expanded_custom_data.ChromeCrashProto.ptype%3D%27renderer%27+AND+expanded_custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3ARenderThreadImpl%3A%3Arender_message_filter%27 Assigning to Mythri for further updates.
,
Sep 14
,
Sep 17
I think this is happening because code caching is not supported for web workers. The current implementation posts messages for renderer message filter which may not work for web workers. I am still verifying this is indeed the case.
,
Sep 17
This issue is marked as a release blocker with no OS labels associated. Please add an appropriate OS label. All release blocking issues should have OS labels associated to it, so that the issue can tracked and promptly verified, once it gets fixed. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Sep 17
Pls apply appropriate OSs label. Thank you.
,
Sep 19
,
Sep 19
,
Sep 21
Friendly ping to get an update as it is marked as RBB. Thanks.!
,
Sep 21
The fix for this is landing now: https://chromium-review.googlesource.com/c/chromium/src/+/1226597. I will update with the version once it lands.
,
Sep 21
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c5a24d544deb060628aa5b50ce8ead21d8af11e3 commit c5a24d544deb060628aa5b50ce8ead21d8af11e3 Author: Mythri Alle <mythria@chromium.org> Date: Fri Sep 21 08:41:19 2018 Fetch from code cache only on main thread Currently the code cache uses RenderMessageFilter to post tasks to the browser thread. This is available only on the main thread. Hence, we should not fetch code caches on non-main threads. Fetching caches would be enabled on all threads once code cache has its own thread-safe mojo interface. Bug: chromium:884135 Change-Id: I1a91307aaa8d3be6aff37b1a7fb7b4bc8270dd87 Reviewed-on: https://chromium-review.googlesource.com/1226597 Reviewed-by: Kinuko Yasuda <kinuko@chromium.org> Commit-Queue: Mythri Alle <mythria@chromium.org> Cr-Commit-Position: refs/heads/master@{#593122} [modify] https://crrev.com/c5a24d544deb060628aa5b50ce8ead21d8af11e3/third_party/blink/renderer/platform/loader/fetch/resource_loader.cc
,
Sep 21
Fix landed in 71.0.3559.0
,
Sep 25
Just to update: RenderThreadImpl::render_message_filter() No instances seen post-71.0.3554.4 as fixed build-71.0.3559.0 is live for 1 day on Linux OS. No instances seen post-71.0.3558.0 as fixed build-71.0.3559.0 is live for 1 day on Mac OS. Link to the list of builds: -------------------------- https://goto.google.com/rqxit Thanks..!
,
Sep 25
Thanks for checking. Since there are no failures for more than a day, marking this as fixed. |
||||
►
Sign in to add a comment |
||||
Comment 1 by mythria@chromium.org
, Sep 14