ParallelDownloadTest.ParallelDownloadComplete in content_browsertests has been flaky on multiple bots |
|||||||||||||
Issue descriptionhttps://ci.chromium.org/buildbot/tryserver.chromium.android/linux_android_rel_ng/?limit=200 Only scanned a few builds but found multiple instances: https://ci.chromium.org/buildbot/tryserver.chromium.android/linux_android_rel_ng/433688 https://ci.chromium.org/buildbot/tryserver.chromium.android/linux_android_rel_ng/433637 Marking P1. Tests must be reliable. Log excerpt follows. [ RUN ] ParallelDownloadTest.ParallelDownloadComplete [WARNING:dns_config_service_posix.cc(341)] Failed to read DnsConfig. [ERROR:devtools_http_handler.cc(249)] Cannot start http server for devtools. Stop devtools. [ERROR:instance.cc(49)] Unable to locate service manifest for metrics [ERROR:service_manager.cc(889)] Failed to resolve service name: metrics [WARNING:child_process_launcher_helper_posix.cc(118)] Ignoring invalid file assets/snapshot_blob_64.bin [WARNING:simple_synchronous_entry.cc(1179)] Could not open platform files for entry. [FATAL:lock.cc(19)] Check failed: owning_thread_ref_.is_null(). [ERROR:test_suite.cc(298)] Currently running: ParallelDownloadTest.ParallelDownloadComplete Searching for native crashes in: /b/swarming/w/itf6ds9P/tmpBxU3c3 Unknown Android release, consider passing --packed-lib. Reading Android symbols from: /b/swarming/w/ir Searching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release/lib:/b/swarming/w/ir/out/Release Using toolchain from: /b/swarming/w/ir/third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi- Stack Trace: RELADDR FUNCTION FILE:LINE 00650c93 logging::LogMessage::~LogMessage()+114 /b/c/b/linux_android_rel_ng/src/base/logging.cc:581 0067c143 base::Lock::~Lock()+42 /b/c/b/linux_android_rel_ng/src/base/synchronization/lock.cc:19 v------> base::WaitableEvent::WaitableEventKernel::~WaitableEventKernel() /b/c/b/linux_android_rel_ng/src/base/synchronization/waitable_event_posix.cc:377 v------> void base::RefCountedThreadSafe<base::WaitableEvent::WaitableEventKernel, base::DefaultRefCountedThreadSafeTraits<base::WaitableEvent::WaitableEventKernel> >::DeleteInternal<base::WaitableEvent::WaitableEventKernel>(base::WaitableEvent::WaitableEventKernel const*) /b/c/b/linux_android_rel_ng/src/base/memory/ref_counted.h:397 v------> base::DefaultRefCountedThreadSafeTraits<base::WaitableEvent::WaitableEventKernel>::Destruct(base::WaitableEvent::WaitableEventKernel const*) /b/c/b/linux_android_rel_ng/src/base/memory/ref_counted.h:350 v------> base::RefCountedThreadSafe<base::WaitableEvent::WaitableEventKernel, base::DefaultRefCountedThreadSafeTraits<base::WaitableEvent::WaitableEventKernel> >::Release() const /b/c/b/linux_android_rel_ng/src/base/memory/ref_counted.h:386 v------> scoped_refptr<base::WaitableEvent::WaitableEventKernel>::Release(base::WaitableEvent::WaitableEventKernel*) /b/c/b/linux_android_rel_ng/src/base/memory/scoped_refptr.h:276 v------> scoped_refptr<base::WaitableEvent::WaitableEventKernel>::~scoped_refptr() /b/c/b/linux_android_rel_ng/src/base/memory/scoped_refptr.h:171 0067c581 base::WaitableEvent::~WaitableEvent()+70 /b/c/b/linux_android_rel_ng/src/base/synchronization/waitable_event_posix.cc:49 v------> content::NestedMessagePumpAndroid::RunState::~RunState() /b/c/b/linux_android_rel_ng/src/content/public/test/nested_message_pump_android.cc:26 01dd013b content::NestedMessagePumpAndroid::Run(base::MessagePump::Delegate*)+446 /b/c/b/linux_android_rel_ng/src/content/public/test/nested_message_pump_android.cc:118 00659d0d base::MessageLoop::Run(bool)+96 /b/c/b/linux_android_rel_ng/src/base/message_loop/message_loop.cc:345 006705e3 base::RunLoop::Run()+142 /b/c/b/linux_android_rel_ng/src/base/run_loop.cc:114 006708af base::RunLoop::RunUntilIdle()+58 /b/c/b/linux_android_rel_ng/src/base/run_loop.cc:127 00350d49 content::ParallelDownloadTest_ParallelDownloadComplete_Test::RunTestOnMainThread()+328 /b/c/b/linux_android_rel_ng/src/content/browser/download/download_browsertest.cc:2927 01dd6497 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop()+210 /b/c/b/linux_android_rel_ng/src/content/public/test/browser_test_base.cc:342 01dd6ac5 void base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::RunImpl<void (content::BrowserTestBase::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&, 0u>(void (content::BrowserTestBase::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&, std::__ndk1::integer_sequence<unsigned int, 0u>)+10 /b/c/b/linux_android_rel_ng/src/base/bind_internal.h:351 01e0e9b3 content::ShellBrowserMainParts::PreMainMessageLoopRun()+90 /b/c/b/linux_android_rel_ng/src/content/shell/browser/shell_browser_main_parts.cc:180 011b031d content::BrowserMainLoop::PreMainMessageLoopRun()+44 /b/c/b/linux_android_rel_ng/src/content/browser/browser_main_loop.cc:1184 011b1835 int base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::RunImpl<int (content::BrowserMainLoop::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&, 0u>(int (content::BrowserMainLoop::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&, std::__ndk1::integer_sequence<unsigned int, 0u>)+10 /b/c/b/linux_android_rel_ng/src/base/bind_internal.h:351 013e1a2b content::StartupTaskRunner::RunAllTasksNow()+18 /b/c/b/linux_android_rel_ng/src/content/browser/startup_task_runner.cc:45 011af513 content::BrowserMainLoop::CreateStartupTasks()+318 /b/c/b/linux_android_rel_ng/src/content/browser/browser_main_loop.cc:963 011b1ca7 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&)+334 /b/c/b/linux_android_rel_ng/src/content/browser/browser_main_runner.cc:119 011ae39f content::BrowserMain(content::MainFunctionParams const&)+86 /b/c/b/linux_android_rel_ng/src/content/browser/browser_main.cc:42 01dd62eb content::BrowserTestBase::SetUp()+754 /b/c/b/linux_android_rel_ng/src/content/public/test/browser_test_base.cc:295 01dce005 content::ContentBrowserTest::SetUp()+208 /b/c/b/linux_android_rel_ng/src/content/public/test/content_browser_test.cc:92 009591af testing::Test::Run()+38 /b/c/b/linux_android_rel_ng/src/third_party/googletest/src/googletest/src/gtest.cc:2468 009595eb testing::TestInfo::Run()+94 /b/c/b/linux_android_rel_ng/src/third_party/googletest/src/googletest/src/gtest.cc:2654 00959809 testing::TestCase::Run()+96 /b/c/b/linux_android_rel_ng/src/third_party/googletest/src/googletest/src/gtest.cc:2772 0095bfbb testing::internal::UnitTestImpl::RunAllTests()+322 /b/c/b/linux_android_rel_ng/src/third_party/googletest/src/googletest/src/gtest.cc:4677 0095be1d testing::UnitTest::Run()+72 /b/c/b/linux_android_rel_ng/src/third_party/googletest/src/googletest/src/gtest.cc:4285 01def2c1 base::TestSuite::Run()+72 /b/c/b/linux_android_rel_ng/src/base/test/test_suite.cc:272 01dcfdbf content::ContentTestLauncherDelegate::RunTestSuite(int, char**)+14 /b/c/b/linux_android_rel_ng/src/content/test/content_test_launcher.cc:106 01de3623 content::LaunchTests(content::TestLauncherDelegate*, unsigned int, int, char**)+234 /b/c/b/linux_android_rel_ng/src/content/public/test/test_launcher.cc:639 01dcfda7 main+46 /b/c/b/linux_android_rel_ng/src/content/test/content_test_launcher.cc:136 v------> testing::android::RunTests(_JNIEnv*, base::android::JavaParamRef<_jobject*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jobject*> const&, base::android::JavaParamRef<_jstring*> const&) /b/c/b/linux_android_rel_ng/src/testing/android/native_test/native_test_launcher.cc:130 00627bf9 Java_org_chromium_native_1test_NativeTest_nativeRunTests+440 /b/c/b/linux_android_rel_ng/src/out/Release/gen/testing/android/native_test/native_test_jni_headers/testing/jni/NativeTest_jni.h:56 0001dbcf <unknown> /system/lib/libdvm.so 0004e125 <unknown> /system/lib/libdvm.so 0004fb0d <unknown> /system/lib/libdvm.so 00026fe3 <unknown> /system/lib/libdvm.so 0002dfa3 <unknown> /system/lib/libdvm.so 0002b63b <unknown> /system/lib/libdvm.so 00060863 <unknown> /system/lib/libdvm.so 000687c5 <unknown> /system/lib/libdvm.so 00026fe3 <unknown> /system/lib/libdvm.so 0002dfa3 <unknown> /system/lib/libdvm.so 0002b63b <unknown> /system/lib/libdvm.so 0006057f <unknown> /system/lib/libdvm.so 00049d0d <unknown> /system/lib/libdvm.so 0004d2b3 <unknown> /system/lib/libandroid_runtime.so 0004dfd9 <unknown> /system/lib/libandroid_runtime.so 0000105d <unknown> /system/bin/app_process 0000e34b <unknown> /system/lib/libc.so 00000d7f <unknown> /system/bin/app_process
,
Nov 20 2017
According to the dashboard https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=content_browsertests&tests=ParallelDownloadTest.ParallelDownloadComplete it's flaky on - KitKat Phone Tester (dbg) - KitKat Phone Tester (rel) - KitKat Tablet Tester - Lollipop Phone Tester - Lollipop Tablet Tester - Marshmallow 64 bit Tester - Marshmallow Tablet Tester - Linux ASan LSan Tests (1) - Linux TSan Tests - Linux Tests SANDBOX - Android N5 Swarm - Linux Swarm - android_n5x_swarming_rel - android_unswarmed_n5_rel - android_unswarmed_n5x_rel in addition to - linux_android_rel_ng
,
Nov 20 2017
,
Nov 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d0f630f5e3da65b62fe0f47e0c24670c3b44bb85 commit d0f630f5e3da65b62fe0f47e0c24670c3b44bb85 Author: Takeshi Yoshino <tyoshino@chromium.org> Date: Mon Nov 20 08:04:09 2017 Disable a flaky test ParallelDownloadTest.ParallelDownloadComplete on Android and Linux Bug: 786626 Change-Id: I8a5a3d5673f7a1fc57bb87791fb93411ae16adf2 Tbr: qinmin@chromium.org Reviewed-on: https://chromium-review.googlesource.com/778342 Reviewed-by: Takeshi Yoshino <tyoshino@chromium.org> Commit-Queue: Takeshi Yoshino <tyoshino@chromium.org> Cr-Commit-Position: refs/heads/master@{#517780} [modify] https://crrev.com/d0f630f5e3da65b62fe0f47e0c24670c3b44bb85/content/browser/download/download_browsertest.cc
,
Nov 20 2017
Disabled. Removing Sheriff-Chromium.
,
Nov 30 2017
,
Nov 30 2017
This test just caused a failure in my CQ run: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/316722 The test was re-enabled in https://chromium-review.googlesource.com/c/chromium/src/+/772191.
,
Nov 30 2017
,
Dec 1 2017
,
Dec 1 2017
xingliu@ any update on this? It looks quite flaky to me, and as Sheriff I feel I should re-apply the change in #4 that somehow was rolled back unless you have a better suggestion. (flakes: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=content_browsertests&tests=ParallelDownloadComplete)
,
Dec 1 2017
Let me disable this test soon.
,
Dec 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2049eab9c7aebd498738ef8375d927b7ecc30b34 commit 2049eab9c7aebd498738ef8375d927b7ecc30b34 Author: Xing Liu <xingliu@chromium.org> Date: Fri Dec 01 19:24:37 2017 Disable ParallelDownloadTest.ParallelDownloadComplete on Android. Currently it's flaky on Android, other platforms seem fine. Disable for now for further investigation. Bug: 786626 Change-Id: I1399f8212ab3a6c58959032313301b6f8d4a3714 Reviewed-on: https://chromium-review.googlesource.com/804134 Reviewed-by: Min Qin <qinmin@chromium.org> Commit-Queue: Xing Liu <xingliu@chromium.org> Cr-Commit-Position: refs/heads/master@{#521013} [modify] https://crrev.com/2049eab9c7aebd498738ef8375d927b7ecc30b34/content/browser/download/download_browsertest.cc
,
Dec 4 2017
,
Dec 5 2017
,
Dec 11 2017
Can't repro on local device, but found sometimes the test will take long time to finish, suspecting that it will cause the flake on trybots. Switch to other things for now.
,
Jan 20 2018
,
Jan 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e0f2031db87e7ca20ae60f80f87593018da338de commit e0f2031db87e7ca20ae60f80f87593018da338de Author: Xing Liu <xingliu@chromium.org> Date: Fri Jan 26 18:06:37 2018 Parallel Download: Fix flaky test ParallelDownloadComplete on Android. Refactor RunLoop::RunUntilIdle call in a while loop into an observer class that only hold one RunLoop. RunLoop::RunUntilIdle in a while loop may never return in multithread context on Android platform. Bug: 786626 Change-Id: Ic5deb53e892d9c6303346eb6a7f803c89eddd26d Reviewed-on: https://chromium-review.googlesource.com/882183 Commit-Queue: Xing Liu <xingliu@chromium.org> Reviewed-by: Scott Violet <sky@chromium.org> Reviewed-by: Min Qin <qinmin@chromium.org> Reviewed-by: David Trainor <dtrainor@chromium.org> Cr-Commit-Position: refs/heads/master@{#532011} [modify] https://crrev.com/e0f2031db87e7ca20ae60f80f87593018da338de/content/browser/download/download_browsertest.cc [modify] https://crrev.com/e0f2031db87e7ca20ae60f80f87593018da338de/content/public/test/test_download_http_response.cc [modify] https://crrev.com/e0f2031db87e7ca20ae60f80f87593018da338de/content/public/test/test_download_http_response.h
,
Jan 29 2018
|
|||||||||||||
►
Sign in to add a comment |
|||||||||||||
Comment 1 by kbr@chromium.org
, Nov 18 2017