Issue metadata
Sign in to add a comment
|
webview_instrumentation_test_apk flaky on android_n5x_swarming_rel trybot |
||||||||||||||||||||||
Issue descriptionhttps://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/?limit=200 Seeing the following flakes: https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/245723 org.chromium.android_webview.test.AndroidScrollIntegrationTest#testOverScrollY__multiprocess_mode C 462.403s Main [ FAILED ] 1 test, listed below: C 462.403s Main [ FAILED ] org.chromium.android_webview.test.AndroidScrollIntegrationTest#testOverScrollY__multiprocess_mode (CRASHED) No obvious stack trace in the log. https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/245680 org.chromium.android_webview.test.AndroidScrollIntegrationTest#testOverScrollY__multiprocess_mode C 461.497s Main [ FAILED ] 1 test, listed below: C 461.497s Main [ FAILED ] org.chromium.android_webview.test.AndroidScrollIntegrationTest#testScrollOffsetAfterCapturePicture__multiprocess_mode (CRASHED) I couldn't find a stack trace in the logs. Marking P1 because this flakiness is causing long retries on the commit queue. It just slowed down Catapult roll https://chromium-review.googlesource.com/c/621866 by an hour. There's been a bug open for a while, Issue 670354, about a failure of one test in this suite but it hasn't been fixed and this is high priority to resolve.
,
Aug 18 2017
What's the next action then? Getting stack traces symbolized on the Swarmed Android bots? Local repro? Marking tests flaky? Disabling them? Running stress tests locally for a repro? Who will own this?
,
Aug 18 2017
Disable & mark this bug as blocked on 631213?
,
Aug 18 2017
looping locally now, no failures so far :/ disable or marking flaky probably doesn't help, unless there's data that it really is only this test? Crash is in probably gpu code, so has nothing to do with the test itself. it's just that test is either super unlucky, or tickles something in gpu
,
Aug 18 2017
jbudorick: do the bots save the symbols somewhere right now? that might be easier
,
Aug 18 2017
Bots have but do not currently upload unstripped libs. We had them doing so recently but had to disable that due to golo bandwidth issues. hzl is working on having them upload a smaller, partially stripped libs that will give us lower fidelity but hopefully still useful traces.
,
Aug 23 2017
Changing synopsis because multiple failures are now seen. It seems there are only four failures in the last 200 builds, so downgrading to P2. https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248642 org.chromium.android_webview.test.AwLegacyQuirksTest#testWideViewportInitialScaleDoesNotExpandFixedLayoutWidth failed https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248624 org.chromium.android_webview.test.AndroidScrollIntegrationTest#testPinchZoomUpdatesScrollRangeSynchronously failed https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248622 org.chromium.android_webview.test.WebKitHitTestTest#testUnfocusedNodeAndTouchRace__multiprocess_mode failed https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248574 org.chromium.android_webview.test.AndroidScrollIntegrationTest#testOverScrollX__multiprocess_mode failed Bo, could you please triage this? If these tests are unavoidably flaky, can retries be added?
,
Aug 25 2017
> https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248642 Already has retry. And bug already filed: crbug.com/746264 > https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248624 > https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248622 > https://luci-milo.appspot.com/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/248574 These three all looks like the same gpu null pointer crash we were looking at above, and got stuck. Different tests. So my suspicion that disabling tests won't help here, since it looks like it could affect any test :(
,
Oct 11 2017
Two more failures of org.chromium.android_webview.test.AndroidScrollIntegrationTest#testPinchZoomUpdatesScrollRangeSynchronously__multiprocess_mode : https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/280068 https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/280141 Example shard: https://chromium-swarm.appspot.com/task?id=3925d1d496673f10&refresh=10&show_raw=1 I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: numtests=1 I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: stream= I 307.823s run_tests_on_device(0078f1f6834a3dd6) org.chromium.android_webview.test.AndroidScrollIntegrationTest: I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: id=AndroidJUnitRunner I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: test=testPinchZoomUpdatesScrollRangeSynchronously__multiprocess_mode I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: class=org.chromium.android_webview.test.AndroidScrollIntegrationTest I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS: current=1 I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_STATUS_CODE: 1 I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_RESULT: shortMsg=Native crash I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_RESULT: longMsg=Native crash: Segmentation fault I 307.823s run_tests_on_device(0078f1f6834a3dd6) INSTRUMENTATION_CODE: 0 There's worse flakiness on this bot, but can we please try to reproduce this locally and track it down? Why aren't we getting a symbolized stack trace from the bot? Generating one should be a priority.
,
Oct 11 2017
we have symbolized tombstones: https://storage.cloud.google.com/chromium-result-details/html/webview_instrumentation_test_apk_android_n5x_swarming_rel_280068_2017_10_11_T20_10_32-UTC (from 280068) symbolized logcat is blocked on some other logcat handling changes we're making
,
Oct 11 2017
Was a tombstone supposed to be generated during this test run where (presumably) the GPU process crashed? It doesn't look like it and I don't see tombstone processing having run.
,
Oct 12 2017
It's present for both of the builds you linked in #9 in result_details, which is what I linked in #10. (Surfacing them effectively in the bot logs is challenging within recipes.) The tombstone looks like this in both cases: https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F39260d119f9ddc11%2F%2B%2Ftombstones_20171011T210441-UTC_00ca0b9380a08d24
,
Oct 12 2017
Aha! Thank you! Bo, here's the stack trace: tombstone_00 created on 2017-10-11 21:03:00, about this long ago: 0:00:28 Device: 00ca0b9380a08d24 Searching for native crashes in: /b/swarming/w/itNyg0qU/tmpD1AWn8 Reading Android symbols from: /b/swarming/w/ir Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release/lib:/b/swarming/w/ir/out/Release Using toolchain from: /b/swarming/w/ir/third_party/android_tools/ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android- pid: 17802, tid: 17867, name: Chrome_InProcGp >>> org.chromium.android_webview.shell <<< signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 Stack Trace: RELADDR FUNCTION FILE:LINE 000000000072f504 void std::__ndk1::__tree_left_rotate<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*)+4 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:208 000000000072f3e4 std::__ndk1::__tree<scoped_refptr<net::ProxyService::PacRequest>, std::__ndk1::less<scoped_refptr<net::ProxyService::PacRequest> >, std::__ndk1::allocator<scoped_refptr<net::ProxyService::PacRequest> > >::__insert_node_at(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*&, std::__ndk1::__tree_node_base<void*>*)+56 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1717 0000000000f2c7c8 std::__ndk1::__tree<scoped_refptr<content::TraceMessageFilter>, std::__ndk1::less<scoped_refptr<content::TraceMessageFilter> >, std::__ndk1::allocator<scoped_refptr<content::TraceMessageFilter> > >::__node_insert_unique(std::__ndk1::__tree_node<scoped_refptr<content::TraceMessageFilter>, void*>*)+76 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1928 0000000002ebf7ac std::__ndk1::pair<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long>, bool> std::__ndk1::__tree<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__map_value_compare<sh::TCache::TypeKey, std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::less<sh::TCache::TypeKey>, true>, std::__ndk1::allocator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*> > >::__insert_unique<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*> >(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1805 v------> std::__ndk1::pair<std::__ndk1::__map_iterator<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long> >, bool> std::__ndk1::map<sh::TCache::TypeKey, sh::TType const*, std::__ndk1::less<sh::TCache::TypeKey>, std::__ndk1::allocator<std::__ndk1::pair<sh::TCache::TypeKey const, sh::TType const*> > >::insert<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>, void>(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&) /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/map:1038 0000000002ebf360 sh::TCache::getType(sh::TBasicType, sh::TPrecision, sh::TQualifier, unsigned char, unsigned char)+192 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Cache.cpp:90 0000000002ed0f5c sh::InsertBuiltInFunctions(unsigned int, ShShaderSpec, ShBuiltInResources const&, sh::TSymbolTable&)+7176 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Initialize.cpp:268 0000000002ec4894 sh::TCompiler::InitBuiltInSymbolTable(ShBuiltInResources const&)+348 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:677 0000000002ec46e0 sh::TCompiler::Init(ShBuiltInResources const&)+80 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:286 0000000002efd80c sh::ConstructCompiler(unsigned int, ShShaderSpec, ShShaderOutput, ShBuiltInResources const*)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/ShaderLang.cpp:282 0000000002e9cdac gpu::gles2::ShaderTranslator::Init(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long, bool)+816 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator.cc:168 0000000002e9dc30 gpu::gles2::ShaderTranslatorCache::GetTranslator(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long)+268 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator_cache.cc:49 0000000002e0b464 gpu::gles2::GLES2DecoderImpl::InitializeShaderTranslator()+2904 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:4101 0000000002e26f5c gpu::gles2::GLES2DecoderImpl::GetTranslator(unsigned int)+28 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10668 0000000002e271bc gpu::gles2::GLES2DecoderImpl::DoCompileShader(unsigned int)+528 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10683 0000000002de222c gpu::gles2::GLES2DecoderImpl::HandleCompileShader(unsigned int, void const volatile*)+12 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder_autogen.h:539 0000000002e12d10 gpu::error::Error gpu::gles2::GLES2DecoderImpl::DoCommandsImpl<false>(unsigned int, void const volatile*, int, int*)+324 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:5384 0000000002db6500 gpu::CommandBufferService::Flush(int, gpu::AsyncAPIInterface*)+608 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/command_buffer_service.cc:90 0000000002f300e4 gpu::GpuCommandBufferStub::OnAsyncFlush(int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > const&)+336 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/service/gpu_command_buffer_stub.cc:996 0000000002f32efc void IPC::DispatchToMethod<gpu::GpuCommandBufferStub, void (gpu::GpuCommandBufferStub::*)(int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > const&), void, std::__ndk1::tuple<int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > > >(gpu::GpuCommandBufferStub*, void (gpu::GpuCommandBufferStub::*)(int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > const&), void*, std::__ndk1::tuple<int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > >&&)+24 /b/c/b/android_n5x_swarming_rel/src/ipc/ipc_message_templates.h:51 0000000002f2ff10 bool IPC::MessageT<GpuCommandBufferMsg_AsyncFlush_Meta, std::__ndk1::tuple<int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > >, void>::Dispatch<gpu::GpuCommandBufferStub, gpu::GpuCommandBufferStub, void, void (gpu::GpuCommandBufferStub::*)(int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > const&)>(IPC::Message const*, gpu::GpuCommandBufferStub*, gpu::GpuCommandBufferStub*, void*, void (gpu::GpuCommandBufferStub::*)(int, unsigned int, std::__ndk1::vector<ui::LatencyInfo, std::__ndk1::allocator<ui::LatencyInfo> > const&))+128 /b/c/b/android_n5x_swarming_rel/src/ipc/ipc_message_templates.h:146 0000000002f2ee8c gpu::GpuCommandBufferStub::OnMessageReceived(IPC::Message const&)+808 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/service/gpu_command_buffer_stub.cc:308 0000000002f29aa0 gpu::GpuChannel::HandleMessageHelper(IPC::Message const&)+68 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/service/gpu_channel.cc:1037 0000000002f26fc8 gpu::GpuChannel::HandleMessage(IPC::Message const&)+300 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/service/gpu_channel.cc:985 0000000000775718 base::OnceCallback<void ()>::Run() &&+48 /b/c/b/android_n5x_swarming_rel/src/base/callback.h:64 0000000002e985ec gpu::Scheduler::RunNextTask()+1116 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/scheduler.cc:503 0000000000775718 base::OnceCallback<void ()>::Run() &&+48 /b/c/b/android_n5x_swarming_rel/src/base/callback.h:64 00000000007af6a8 base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*)+140 /b/c/b/android_n5x_swarming_rel/src/base/debug/task_annotator.cc:57 00000000007d2e50 base::internal::IncomingTaskQueue::RunTask(base::PendingTask*)+148 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/incoming_task_queue.cc:130 00000000007d6744 base::MessageLoop::RunTask(base::PendingTask*)+552 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/message_loop.cc:392 00000000007d6a68 base::MessageLoop::DeferOrRunPendingTask(base::PendingTask)+212 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/message_loop.cc:404 00000000007d6cd4 base::MessageLoop::DoWork()+552 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/message_loop.cc:450 00000000007d8bb4 base::MessagePumpDefault::Run(base::MessagePump::Delegate*)+208 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/message_pump_default.cc:37 00000000007d6094 base::MessageLoop::Run()+164 /b/c/b/android_n5x_swarming_rel/src/base/message_loop/message_loop.cc:344 00000000007f957c base::RunLoop::Run()+164 /b/c/b/android_n5x_swarming_rel/src/base/run_loop.cc:118 000000000083159c base::Thread::Run(base::RunLoop*)+228 /b/c/b/android_n5x_swarming_rel/src/base/threading/thread.cc:255 0000000000831b00 base::Thread::ThreadMain()+744 /b/c/b/android_n5x_swarming_rel/src/base/threading/thread.cc:338 000000000082a178 base::(anonymous namespace)::ThreadFunc(void*)+108 /b/c/b/android_n5x_swarming_rel/src/base/threading/platform_thread_posix.cc:75 0000000000066d24 __pthread_start(void*)+52 /system/lib64/libc.so 000000000001eb84 __start_thread+16 /system/lib64/libc.so Is there some thread-unsafe access to the shader translator going on in WebView?
,
Oct 12 2017
shouldn't be, webview has one shader translator cache per thread (so two in total) unless angle keeps global state or something? but that's beyond my knowledge..
,
Oct 17 2017
Searching through the source code in https://chromium.googlesource.com/angle/angle/+/master/src/compiler/ , the TCache class (which shows up in the stack trace) is not thread safe. This means that any concurrent compilations are potentially risky. It turns out this has been a known issue for some time – see https://bugs.chromium.org/p/angleproject/issues/detail?id=1432 – I'm going to upgrade that to high priority.
,
Nov 17 2017
I see 21 webview_instrumentation_test_apk failures in last 200 android_n5x_swarming_rel builds. Most of them are *__multiprocess_mode tests. Maybe we can disable those until the underlying issue is fixed? Also, I'm not sure if all the failures are due to TCache, in a logcat from here: https://chromium-swarm.appspot.com/task?id=39e0a33e45715210&refresh=10&show_raw=1 https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F39e0a33e45715211%2F%2B%2Flogcat_logcat_org.chromium.android_webview.test.AwSettingsTest.testBlockNetworkImagesWithTwoViews__multiprocess_mode_20171117T023417-UTC_00ac8d3289f5338f I see: 11-17 02:34:18.732 27547 27591 F chromium: [FATAL:Functional.h(228)] Check failed: (sequence_checker_).CalledOnValidSequence().
,
Nov 17 2017
Probably 10% failure rate should be higher priority.
,
Nov 17 2017
The last time I looked this particular crash wasn't that frequent, but it's on my plate to make the TCache thread-local.
,
Nov 17 2017
Kai said he could pick this up.
,
Nov 28 2017
It turns out this is showing up moderately often on the android_n5x_swarming_rel tryserver. See these two webview_instrumentation_test_apk failures for example: org.chromium.android_webview.test.AndroidScrollIntegrationTest#testUiScrollReflectedInJs https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/312626 org.chromium.android_webview.test.AndroidScrollIntegrationTest#testOverScrollY https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/313968 Click the "result details" link under any one, then click the "tombstones" for the crashing test. The stack trace is: pid: 17481, tid: 17517, name: GLThread 663 >>> org.chromium.android_webview.shell <<< signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 Stack Trace: RELADDR FUNCTION FILE:LINE 000000000072e304 void std::__ndk1::__tree_left_rotate<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*)+4 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:208 000000000072e2d8 void std::__ndk1::__tree_balance_after_insert<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*)+216 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:272 000000000072e1e4 std::__ndk1::__tree<scoped_refptr<net::ProxyService::PacRequest>, std::__ndk1::less<scoped_refptr<net::ProxyService::PacRequest> >, std::__ndk1::allocator<scoped_refptr<net::ProxyService::PacRequest> > >::__insert_node_at(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*&, std::__ndk1::__tree_node_base<void*>*)+56 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1717 0000000000f25818 std::__ndk1::__tree<scoped_refptr<content::TraceMessageFilter>, std::__ndk1::less<scoped_refptr<content::TraceMessageFilter> >, std::__ndk1::allocator<scoped_refptr<content::TraceMessageFilter> > >::__node_insert_unique(std::__ndk1::__tree_node<scoped_refptr<content::TraceMessageFilter>, void*>*)+76 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1928 0000000002f9a6f4 std::__ndk1::pair<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long>, bool> std::__ndk1::__tree<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__map_value_compare<sh::TCache::TypeKey, std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::less<sh::TCache::TypeKey>, true>, std::__ndk1::allocator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*> > >::__insert_unique<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*> >(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1805 v------> std::__ndk1::pair<std::__ndk1::__map_iterator<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long> >, bool> std::__ndk1::map<sh::TCache::TypeKey, sh::TType const*, std::__ndk1::less<sh::TCache::TypeKey>, std::__ndk1::allocator<std::__ndk1::pair<sh::TCache::TypeKey const, sh::TType const*> > >::insert<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>, void>(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&) /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/map:1038 0000000002f9a66c sh::TCache::getType(sh::TBasicType, sh::TPrecision, sh::TQualifier, unsigned char, unsigned char)+192 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Cache.cpp:94 0000000002facf14 sh::InsertBuiltInFunctions(unsigned int, ShShaderSpec, ShBuiltInResources const&, sh::TSymbolTable&)+10404 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Initialize.cpp:451 0000000002f9f8f8 sh::TCompiler::InitBuiltInSymbolTable(ShBuiltInResources const&)+348 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:729 0000000002f9f744 sh::TCompiler::Init(ShBuiltInResources const&)+80 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:291 0000000002fdb6c4 sh::ConstructCompiler(unsigned int, ShShaderSpec, ShShaderOutput, ShBuiltInResources const*)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/ShaderLang.cpp:312 0000000002f77dc8 gpu::gles2::ShaderTranslator::Init(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long, bool)+816 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator.cc:168 0000000002f78de8 gpu::gles2::ShaderTranslatorCache::GetTranslator(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long)+264 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator_cache.cc:47 0000000002edf0cc gpu::gles2::GLES2DecoderImpl::InitializeShaderTranslator()+2920 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:4152 v------> gpu::gles2::GLES2DecoderImpl::GetOrCreateTranslator(unsigned int) /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10759 0000000002efa888 gpu::gles2::GLES2DecoderImpl::DoCompileShader(unsigned int)+516 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10774 0000000002eb5d74 gpu::gles2::GLES2DecoderImpl::HandleCompileShader(unsigned int, void const volatile*)+12 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder_autogen.h:539 0000000002ee64f0 gpu::error::Error gpu::gles2::GLES2DecoderImpl::DoCommandsImpl<false>(unsigned int, void const volatile*, int, int*)+324 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:5441 0000000002e8ac9c gpu::CommandBufferService::Flush(int, gpu::AsyncAPIInterface*)+608 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/command_buffer_service.cc:90 ...
,
Nov 28 2017
,
Dec 2 2017
Do crashes in this stack show up much in stable crash reports? Do we need Release-Block, and merge of a fix?
,
Dec 4 2017
Moving Infra>Client>Android -> Infra>Client>Chrome+OS=Android
,
Dec 4 2017
,
Dec 4 2017
Crash signatures also seem to possibly occur in buildMangledName: Stack Trace: RELADDR FUNCTION FILE:LINE 0000000002fdc880 sh::TType::buildMangledName() const+768 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Types.cpp:539 0000000002fdd27c sh::TType::getMangledName() const+28 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Types.cpp:765 0000000002fd850c sh::TFunction::buildMangledName() const+80 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:69 0000000002fdaa18 sh::TFunction::getMangledName() const+28 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.h:180 0000000002fd8698 sh::TSymbolTableLevel::insert(sh::TSymbol*)+36 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:99 0000000002fd9d94 sh::TSymbolTable::insertBuiltIn(int, TOperator, sh::TExtension, sh::TType const*, char const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*)+3328 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:504 0000000002fd9c14 sh::TSymbolTable::insertBuiltIn(int, TOperator, sh::TExtension, sh::TType const*, char const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*)+2944 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:470 0000000002fda564 sh::TSymbolTable::insertBuiltInOp(int, TOperator, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*)+156 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:520 0000000002fa635c sh::InsertBuiltInFunctions(unsigned int, ShShaderSpec, ShBuiltInResources const&, sh::TSymbolTable&)+5560 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Initialize.cpp:212 0000000002f9a02c sh::TCompiler::InitBuiltInSymbolTable(ShBuiltInResources const&)+348 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:729 0000000002f99e78 sh::TCompiler::Init(ShBuiltInResources const&)+80 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:291 0000000002fd5cf0 sh::ConstructCompiler(unsigned int, ShShaderSpec, ShShaderOutput, ShBuiltInResources const*)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/ShaderLang.cpp:311 0000000002f724fc gpu::gles2::ShaderTranslator::Init(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long, bool)+816 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator.cc:168 Could be the race condition is between getType and buildMangledName. Trying to investigate how to build a regression test for this. If this is a race condition, is there any way to view the stack signatures of the other threads active at the time one thread crashes? The tombstones only seem to have the one thread.
,
Dec 4 2017
Having a hard time with crash/ today but it does seem there are potentially related crashes in the wild on several platforms: https://crash.corp.google.com/browse?q=product.name%20LIKE%20%27Chrome%25%27%20OMIT%20RECORD%20IF%20SUM(CrashedStackTrace.StackFrame.FunctionName%20LIKE%20%27sh%3A%3ATCache%3A%3AgetType%25%27)%20%3D%200&sql_dialect=dremelsql&ignore_case=false&enable_rewrite=false&omit_field_name=&omit_field_value=&omit_field_opt=#-property-selector,+samplereports,magicsignature,clientid,operatingsystem,osversion 1 Chrome_Android 63.46% 1358 2 Chrome 31.21% 668 3 Chrome_ChromeOS 5.05% 108 4 Chrome_Linux 0.28% 6 However it's hard to tell how many of them will be fixed by my change. I actually don't know why we would be seeing crashes in this code on Windows at all.
,
Dec 5 2017
Note: saw this crash again here: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/319312 Saw that this fix just rolled in here: https://chromium-review.googlesource.com/807488 Looking forward to seeing this improve the reliability of this tryserver. For the record, result_details here: https://00e9e64bac38856a454b7f27d25c1d21ab185d5c688e251c8c-apidata.googleusercontent.com/download/storage/v1/b/chromium-result-details/o/html%2Fwebview_instrumentation_test_apk_android_n5x_swarming_rel_319312_2017_12_05_T03_19_31-UTC?qk=AD5uMEthD1iKAiWUx84_E6J-ej0eKRwSxEOU0PWw4z0A2bDGusI8QhYuq4rwCmhBI-_XBWCcpMHgiAt4OyESELhI2KTUy8BYKEUa7513XI1skboNKQqLV8xjVxAbPZ_5RETPkdYJvM3wGSxAChq-OROyxc7NFylTZ8xrEUoK4ig6v87Tc83Vnh3IDO7vFcFtg5WKyhMh1pnNOeeuMYruUaJf5rYrz5oybsYOOdmAToe9vbal1zqhwVea7arMZRpWmcQf3huMfVsTos28pzoLM7RgCk0gMheH5qSWOekrtZtZ7bob_ImlaQBOH_zXvbWLLf8pJArPdtWTQt9JL2MxGLh78abNoiKDMb52zGwmW_zhM1jb9ESGYLFDSxHNdhkUgL4KNxOQet3d3OlZg691B1_Nsjfa2iwrDFgvZlKgEnD8kkpyf0TdK39vtwwiznYtLuw_DilOFEsdQxwzYW1AXYlDNpoHTb2IevaVTG-6qUyb7phWBOSEoRj5YM35EQda3-ij1bVRvTTw0M4wVYpHztTkVf3a3QzgIFZt71rGJ11UXPIlGuD_gzjTZypiXxNV9kNZ2ltdLdq8Zf0KPFzipROVcdGabVuXnnOzXV5kAb-XLalP_I8RoImS-uwItCSk_66BEcrYr8eqhwKK2E5HHdaEZbyxDp6Gt2hRRoyfyQnsl4HP70rZqQPqhsg-pmlWsZ_iMzOsaZOqORD8t3r_8y1DM0EBp9uCYCWvX9faHOmQ4tPCjuIiYagfgJiAmH41Fs2F_FlRXhxXGse4ES6Y9LeFBOwZKmiF_IVh2oct67EByz33dhXnLFzNfcD5hZ8RrR4GwjwwXz6RJqTJsrWMh4sTLvGT-wquQEndJJRj40vdAKZJqMeDegg Stack trace: Stack Trace: RELADDR FUNCTION FILE:LINE 0000000000731008 void std::__ndk1::__tree_left_rotate<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*)+4 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:208 0000000000730ee8 std::__ndk1::__tree<scoped_refptr<net::ProxyService::PacRequest>, std::__ndk1::less<scoped_refptr<net::ProxyService::PacRequest> >, std::__ndk1::allocator<scoped_refptr<net::ProxyService::PacRequest> > >::__insert_node_at(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*&, std::__ndk1::__tree_node_base<void*>*)+56 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1717 0000000000f2b5ec std::__ndk1::__tree<scoped_refptr<content::TraceMessageFilter>, std::__ndk1::less<scoped_refptr<content::TraceMessageFilter> >, std::__ndk1::allocator<scoped_refptr<content::TraceMessageFilter> > >::__node_insert_unique(std::__ndk1::__tree_node<scoped_refptr<content::TraceMessageFilter>, void*>*)+76 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1928 0000000002fd6918 std::__ndk1::pair<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long>, bool> std::__ndk1::__tree<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__map_value_compare<sh::TCache::TypeKey, std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::less<sh::TCache::TypeKey>, true>, std::__ndk1::allocator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*> > >::__insert_unique<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*> >(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/__tree:1805 v------> std::__ndk1::pair<std::__ndk1::__map_iterator<std::__ndk1::__tree_iterator<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, std::__ndk1::__tree_node<std::__ndk1::__value_type<sh::TCache::TypeKey, sh::TType const*>, void*>*, long> >, bool> std::__ndk1::map<sh::TCache::TypeKey, sh::TType const*, std::__ndk1::less<sh::TCache::TypeKey>, std::__ndk1::allocator<std::__ndk1::pair<sh::TCache::TypeKey const, sh::TType const*> > >::insert<std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>, void>(std::__ndk1::pair<sh::TCache::TypeKey, sh::TType*>&&) /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libcxx/include/map:1038 0000000002fd6890 sh::TCache::getType(sh::TBasicType, sh::TPrecision, sh::TQualifier, unsigned char, unsigned char)+192 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Cache.cpp:94 000000000301a9a8 sh::SpecificType(sh::TType const*, int)+152 /data/app/org.chromium.android_webview.shell-1/lib/arm64/libstandalonelibwebviewchromium.so 000000000301b8f0 sh::TSymbolTable::insertBuiltIn(int, TOperator, sh::TExtension, sh::TType const*, char const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*)+2352 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:499 000000000301c50c sh::TSymbolTable::insertBuiltInOp(int, TOperator, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*, sh::TType const*)+156 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/SymbolTable.cpp:557 0000000002fe7570 sh::InsertBuiltInFunctions(unsigned int, ShShaderSpec, ShBuiltInResources const&, sh::TSymbolTable&)+2528 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Initialize.cpp:107 0000000002fdbc34 sh::TCompiler::InitBuiltInSymbolTable(ShBuiltInResources const&)+348 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:733 0000000002fdba80 sh::TCompiler::Init(ShBuiltInResources const&)+80 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/Compiler.cpp:291 0000000003017bac sh::ConstructCompiler(unsigned int, ShShaderSpec, ShShaderOutput, ShBuiltInResources const*)+52 /b/c/b/android_n5x_swarming_rel/src/third_party/angle/src/compiler/translator/ShaderLang.cpp:312 0000000002fb3f14 gpu::gles2::ShaderTranslator::Init(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long, bool)+816 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator.cc:168 0000000002fb4f34 gpu::gles2::ShaderTranslatorCache::GetTranslator(unsigned int, ShShaderSpec, ShBuiltInResources const*, ShShaderOutput, unsigned long)+264 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/shader_translator_cache.cc:47 0000000002f1b048 gpu::gles2::GLES2DecoderImpl::InitializeShaderTranslator()+2920 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:4154 v------> gpu::gles2::GLES2DecoderImpl::GetOrCreateTranslator(unsigned int) /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10761 0000000002f36804 gpu::gles2::GLES2DecoderImpl::DoCompileShader(unsigned int)+516 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:10776 0000000002ef1bcc gpu::gles2::GLES2DecoderImpl::HandleCompileShader(unsigned int, void const volatile*)+12 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder_autogen.h:539 0000000002f2246c gpu::error::Error gpu::gles2::GLES2DecoderImpl::DoCommandsImpl<false>(unsigned int, void const volatile*, int, int*)+324 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/gles2_cmd_decoder.cc:5443 0000000002ec6af4 gpu::CommandBufferService::Flush(int, gpu::AsyncAPIInterface*)+608 /b/c/b/android_n5x_swarming_rel/src/gpu/command_buffer/service/command_buffer_service.cc:90 000000000303cb7c gpu::InProcessCommandBuffer::FlushOnGpuThread(int, bool)+96 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/in_process_command_buffer.cc:578 000000000303c9c4 gpu::InProcessCommandBuffer::ProcessTasksOnGpuThread()+188 /b/c/b/android_n5x_swarming_rel/src/gpu/ipc/in_process_command_buffer.cc:537 000000000076cf00 android_webview::DeferredGpuCommandService::RunTasks()+160 /b/c/b/android_n5x_swarming_rel/src/android_webview/browser/deferred_gpu_command_service.cc:193 000000000076d500 android_webview::DeferredGpuCommandService::ScheduleTask(base::RepeatingCallback<void ()> const&)+68 /b/c/b/android_n5x_swarming_rel/src/android_webview/browser/deferred_gpu_command_service.cc:116
,
Dec 5 2017
I don't think it's possible that this bug still exists, since that code has been totally replaced, but I did take a look at the trybot and nothing like it seems to be happening (at least, not right now).
,
Dec 5 2017
Thank you Kai for picking up and fixing this longstanding and nasty bug!
,
Dec 12 2017
When was the code removed? This was reported in http://b/69920338 on 64.0.3282.12.
,
Dec 12 2017
It landed here: https://chromium.googlesource.com/angle/angle/+/614dd0f537d4cc1d0390e7cfa0691c3f0c698399 Included into Chrome in this roll: https://chromium.googlesource.com/chromium/src/+/f185104cf090e24efab72a24805936b79aadaca8 which per https://chromiumdash-staging.googleplex.com/commit/f185104cf090e24efab72a24805936b79aadaca8 was first released in Canary 65.0.3286.0. We have not merged back to Chrome 64. Might the issue you're seeing warrant a mergeback?
,
Dec 12 2017
> Might the issue you're seeing warrant a mergeback? I don't think so? I don't think the buganizer issue is high priority, and we'll drop M65 eventually. Thanks for confirming!
,
Dec 12 2017
Estelle, would we be prepared to cherry-pick an Angle roll to 64 at this stage? Normally I would feel hesitant, but it's possible that it would fix this set of crashes: https://crash.corp.google.com/browse?q=product.name%3D%27AndroidWebView%27%20AND%20%20EXISTS%20(SELECT%201%20FROM%20UNNEST(CrashedStackTrace.StackFrame)%20WHERE%20FunctionName%3D%27sh%3A%3ATCompiler%3A%3AInit(ShBuiltInResources%20const%26)%27)&sql_dialect=googlesql&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports (~125,000 on m62 stable)
,
Dec 12 2017
The query in question is product.name='AndroidWebView' AND EXISTS (SELECT 1 FROM UNNEST(CrashedStackTrace.StackFrame) WHERE FunctionName='sh::TCompiler::Init(ShBuiltInResources const&)') since the URL didn't end up clickable...
,
Dec 12 2017
It would definitely fix that set of crashes. I wasn't able to find good evidence at the time that this was a major crasher in the wild. I guess my crash/-fu is not so good.
,
Dec 12 2017
> I guess my crash/-fu is not so good. I assume you were only looking at chrome crashes, and this is only a problem on webview. they are different products on crash
,
Dec 12 2017
Aha, I bet you're right, thanks! I thought I was looking at "all products" but I may have accidentally ended up in product:Chrome since I wasn't thinking about that possibility.
,
Dec 12 2017
Couples of questions kainino@ is there any reason why the roll happened after the M64 branch point? What could possibility break if we were to take this roll into M64?
,
Dec 13 2017
I just happened to finish the work in ANGLE (just) after M64 had been branched. The patch is well tested and has been baking in Canary for a while without any issues. I don't think there is risk in merging it back, but it is a large patch.
,
Dec 13 2017
Also, this is not a recent regression; the bug has been there for a long time. Unless the frequency of crashes has gone up, it does not seem particularly critical.
,
Dec 13 2017
Actually it does seem to have been really high in M62, and looking potentially high in M63 too. I don't know why that would be the case, but since it's a race condition, it's reasonable that some other change would cause it to spike.
,
Dec 13 2017
Just want to mention that the merge request to M63 would be for a single CL rather than the roll: https://chromium-review.googlesource.com/786317
,
Dec 15 2017
Are we serious about merging this back? If so, someone should actually request the merge...
,
Dec 15 2017
Issue 771297 has been merged into this issue.
,
Dec 15 2017
Thanks for reviving this. I think we should merge to 64 in light of the fact that this crash got a lot worse recently. It's been baking in Canary since Dec 04 and we have not seen issues.
,
Dec 15 2017
This bug requires manual review: M64 has already been promoted to the beta branch, so this requires manual review Please contact the milestone owner if you have questions. Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), abdulsyed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 16 2017
Merge approved. Please make sure to verify the fix in M64 after merging it.
,
Dec 16 2017
Ok, thanks. It's hard to verify this fix in production (since it's a flaky race condition), so the only way to do it will be to watch for the reduction in WebView crash reports on M64. I will do that. Merged.
,
Dec 16 2017
(The merge comment went here because it isn't actually attached to this bug) https://bugs.chromium.org/p/angleproject/issues/detail?id=1432#c28
,
Dec 18 2017
,
Dec 19 2017
The NextAction date has arrived: 2017-12-19
,
Dec 19 2017
Still has not shipped to 64 Beta or 64 Dev according to dashboard.
,
Dec 22 2017
The NextAction date has arrived: 2017-12-22
,
Dec 22 2017
Crashes in this code have disappeared in the release where the code no longer exists. This verifies, at least, that the patch did make it into the release. We have not seen spikes of any other issues. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by boliu@chromium.org
, Aug 18 2017