New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 791218 link

Starred by 2 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocking:
issue 685856



Sign in to add a comment

EddystoneEncoderBridgeTest.TestValidUrl flaky in unit_tests on android_n5x_swarming_rel

Project Member Reported by kbr@chromium.org, Dec 2 2017

Issue description

Failure seen here:

https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/317966

Log excerpt below.

It doesn't look like any tombstones were collected for this failure and there's no crash stack in the result_details. Marking Infra>Client>Android for this reason.

I  852.770s run_tests_on_device(0241d7a25994378a)  [ RUN      ] EddystoneEncoderBridgeTest.TestValidUrl
I  852.770s run_tests_on_device(0241d7a25994378a)  [ CRASHED      ]
I  852.773s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( test -e /data/tombstones );echo %$?'
I  852.804s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=utc ls -a -l /data/tombstones/ );echo %$?'
I  852.868s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=UTC date );echo %$?'
I  852.933s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=utc ls -a -l /data/tombstones/ );echo %$?'
I  852.984s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a pull /data/tombstones/tombstone_00 /b/swarming/w/itsFxLOa/tmpGzZzZI/tmp_ReadFileWithPull
I  853.165s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( rm /data/tombstones/tombstone_00 );echo %$?'
I  853.229s run_tests_on_device(0241d7a25994378a)  tombstone_00 created on 2017-12-02 00:53:00, about this long ago: 0:01:01 Device: 0241d7a25994378a
I  853.229s run_tests_on_device(0241d7a25994378a)  Resolving...
I  853.229s run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/build/android/pylib/symbols/../../../../third_party/android_platform/development/scripts/stack --arch arm64 --output-directory /b/swarming/w/ir/out/Release --more-info /b/swarming/w/itsFxLOa/tmpn05kUb
I  886.728s run_tests_on_device(0241d7a25994378a)  Writing text to logdog stream, tombstones_20171202T005436_0241d7a25994378a
I  886.729s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( test -e /data/tombstones );echo %$?'
I  886.793s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=utc ls -a -l /data/tombstones/ );echo %$?'
W  886.856s run_tests_on_device(0241d7a25994378a)  No tombstones to clear.
I  886.856s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( pm dump org.chromium.native_test | grep dataDir=; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?'
I  887.821s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( mkdir -p /sdcard/tmp-7bad3f010ee32 );echo %$?'
I  887.884s run_tests_on_device(0241d7a25994378a)  flags:
I  887.885s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a logcat -c
I  887.949s record_to_file  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a logcat -v threadtime chromium:v 'cr_*:v' DEBUG:I StrictMode:D
I  887.950s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a push /b/swarming/w/itsFxLOa/tmpfqiIsz /data/local/tmp/temp_file-76d658b68567d
I  887.965s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( ( p=org.chromium.native_test;am instrument -w -e "$p".NativeTestInstrumentationTestRunner.ShardNanoTimeout 120000000000 -e "$p".NativeTestInstrumentationTestRunner.NativeTestActivity "$p".NativeUnitTestActivity -e "$p".NativeTestInstrumentationTestRunner.TestList /data/local/tmp/temp_file-76d658b68567d -e "$p".NativeTestInstrumentationTestRunner.StdoutFile /sdcard/temp_file-a771e34a54b87.gtest_out "$p"/"$p".NativeTestInstrumentationTestRunner )>/data/local/tmp/temp_file-62a60f9f7230e 2>&1 );echo %$?'
I  903.665s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a pull /data/local/tmp/temp_file-62a60f9f7230e /b/swarming/w/itsFxLOa/tmpjiPxSM/tmp_ReadFileWithPull
I  903.711s TimeoutThread-1-for-delete_temporary_file(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell 'rm -f /data/local/tmp/temp_file-62a60f9f7230e'
I  903.739s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=utc ls -a -l /sdcard/ );echo %$?'
I  903.803s run_tests_on_device(0241d7a25994378a)  /sdcard/temp_file-a771e34a54b87.gtest_out size on device: 62025
I  903.803s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( TZ=utc ls -a -l /sdcard/ );echo %$?'
I  903.852s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a pull /sdcard/temp_file-a771e34a54b87.gtest_out /b/swarming/w/itsFxLOa/tmphUMU3d/tmp_ReadFileWithPull
I  903.919s TimeoutThread-1-for-delete_temporary_file(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell 'rm -f /sdcard/temp_file-a771e34a54b87.gtest_out'
I  903.919s TimeoutThread-1-for-delete_temporary_file(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell 'rm -f /data/local/tmp/temp_file-76d658b68567d'
I  903.983s run_tests_on_device(0241d7a25994378a)  Logcat saved to https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F3a2d7b893da5bc11%2F%2B%2Flogcat_logcat_-879327335997850541_20171202T005437-UTC_0241d7a25994378a
I  903.984s archive  Writing text to logdog stream, logcat_logcat_-879327335997850541_20171202T005437-UTC_0241d7a25994378a
I  903.984s TimeoutThread-1-for-delete_temporary_file(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell 'rm -f /data/user/0/org.chromium.native_test/temp_file-9422cb7575964.xml'
I  903.986s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( pm clear org.chromium.native_test );echo %$?'
I  904.626s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  Setting permissions for org.chromium.native_test.
I  904.626s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell '( ( p=org.chromium.native_test;for q in android.permission.CAMERA android.permission.READ_EXTERNAL_STORAGE android.permission.RECORD_AUDIO android.permission.WRITE_EXTERNAL_STORAGE;do pm grant "$p" "$q";echo "~X~$q~X~$?~X~";done )>/data/local/tmp/temp_file-57ae386337b84 2>&1 );echo %$?'
I  907.056s TimeoutThread-1-for-run_tests_on_device(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a pull /data/local/tmp/temp_file-57ae386337b84 /b/swarming/w/itsFxLOa/tmpFKf5XA/tmp_ReadFileWithPull
I  907.109s TimeoutThread-1-for-delete_temporary_file(0241d7a25994378a)  [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 0241d7a25994378a shell 'rm -f /data/local/tmp/temp_file-57ae386337b84'

 

Comment 2 by kbr@chromium.org, Dec 2 2017

Components: Internals>Core
I'm wondering why the test suite timed out just because one of the tests crashed.

Thanks for the tombstone link. Here's the result:

Stack Trace:
  RELADDR   FUNCTION                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     FILE:LINE
  000000000006a144  tgkill+8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     /system/lib64/libc.so
  00000000000678d4  pthread_kill+68                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              /system/lib64/libc.so
  0000000000023838  raise+28                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     /system/lib64/libc.so
  000000000001dfd8  abort+60                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     /system/lib64/libc.so
  00000000004338a8  art::Runtime::Abort()+324                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    /system/lib64/libart.so
  0000000000138164  art::LogMessage::~LogMessage()+3136                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          /system/lib64/libart.so
  000000000030eb68  art::JavaVMExt::JniAbort(char const*, char const*)+2080                                                                                                                                                                                                                                                                                                                                                                                                                                                                      /system/lib64/libart.so
  000000000030efe8  art::JavaVMExt::JniAbortV(char const*, char const*, std::__va_list)+116                                                                                                                                                                                                                                                                                                                                                                                                                                                      /system/lib64/libart.so
  0000000000143f1c  art::ScopedCheck::AbortF(char const*, ...)+144                                                                                                                                                                                                                                                                                                                                                                                                                                                                               /system/lib64/libart.so
  000000000014adcc  art::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::JniValueType*) (.constprop.116)+6088                                                                                                                                                                                                                                                                                                                                                                                                               /system/lib64/libart.so
  000000000015dad8  art::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, std::__va_list, art::Primitive::Type, art::InvokeType)+948                                                                                                                                                                                                                                                                                                                                                                               /system/lib64/libart.so
  000000000015e868  art::CheckJNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, std::__va_list)+68                                                                                                                                                                                                                                                                                                                                                                                                                                     /system/lib64/libart.so
  0000000000ffbcf8  _JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+104                                                                                                                                                                                                                                                                                                                                                                                                                                                                /b/c/b/android_n5x_swarming_rel/src/third_party/android_tools/ndk/platforms/android-21/arch-arm64/usr/include/jni.h:807
  v------>  Java_MemoryPressureListener_registerSystemCallback(_JNIEnv*)                                                                                                                                                                                                                                                                                                                                                                                                                                                                 /b/c/b/android_n5x_swarming_rel/src/out/Release/gen/base/base_jni_headers/base/jni/MemoryPressureListener_jni.h:64
  000000000163e9d0  base::android::MemoryPressureListenerAndroid::RegisterSystemCallback(_JNIEnv*)+240                                                                                                                                                                                                                                                                                                                                                                                                                                           /b/c/b/android_n5x_swarming_rel/src/base/android/memory_pressure_listener_android.cc:26
  000000000356fc04  TestingBrowserProcessPlatformPart::TestingBrowserProcessPlatformPart()+16                                                                                                                                                                                                                                                                                                                                                                                                                                                    /b/c/b/android_n5x_swarming_rel/src/chrome/test/base/testing_browser_process_platform_part.cc:7
  000000000356f028  TestingBrowserProcess::TestingBrowserProcess()+100                                                                                                                                                                                                                                                                                                                                                                                                                                                                           /b/c/b/android_n5x_swarming_rel/src/chrome/test/base/testing_browser_process.cc:105
  000000000356ef70  TestingBrowserProcess::CreateInstance()+112                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  /b/c/b/android_n5x_swarming_rel/src/chrome/test/base/testing_browser_process.cc:86
  00000000018442b0  testing::internal::TestEventRepeater::OnTestStart(testing::TestInfo const&)+56                                                                                                                                                                                                                                                                                                                                                                                                                                               /b/c/b/android_n5x_swarming_rel/src/third_party/googletest/src/googletest/src/gtest.cc:3343
  000000000184284c  testing::TestInfo::Run()+80                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  /b/c/b/android_n5x_swarming_rel/src/third_party/googletest/src/googletest/src/gtest.cc:2638
  0000000001842bf4  testing::TestCase::Run()+160                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 /b/c/b/android_n5x_swarming_rel/src/third_party/googletest/src/googletest/src/gtest.cc:2772
  0000000001846728  testing::internal::UnitTestImpl::RunAllTests()+520                                                                                                                                                                                                                                                                                                                                                                                                                                                                           /b/c/b/android_n5x_swarming_rel/src/third_party/googletest/src/googletest/src/gtest.cc:4677
  00000000018464a8  testing::UnitTest::Run()+120                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 /b/c/b/android_n5x_swarming_rel/src/third_party/googletest/src/googletest/src/gtest.cc:4285
  0000000003578f78  base::TestSuite::Run()+120                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   /b/c/b/android_n5x_swarming_rel/src/base/test/test_suite.cc:272
  00000000035732e8  int base::internal::Invoker<base::internal::BindState<int (content::UnitTestTestSuite::*)(), base::internal::UnretainedWrapper<content::UnitTestTestSuite> >, int ()>::RunImpl<int (content::UnitTestTestSuite::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::UnitTestTestSuite> > const&, 0ul>(int (content::UnitTestTestSuite::* const&)(), std::__ndk1::tuple<base::internal::UnretainedWrapper<content::UnitTestTestSuite> > const&, std::__ndk1::integer_sequence<unsigned long, 0ul>)+24  /b/c/b/android_n5x_swarming_rel/src/base/bind_internal.h:351
  v------>  base::(anonymous namespace)::LaunchUnitTestsInternal(base::RepeatingCallback<int ()> const&, unsigned long, int, bool, base::RepeatingCallback<void ()> const&)                                                                                                                                                                                                                                                                                                                                                              /b/c/b/android_n5x_swarming_rel/src/base/test/launcher/unit_test_launcher.cc:196
  000000000357b800  base::LaunchUnitTests(int, char**, base::RepeatingCallback<int ()> const&)+76                                                                                                                                                                                                                                                                                                                                                                                                                                                /b/c/b/android_n5x_swarming_rel/src/base/test/launcher/unit_test_launcher.cc:558
  00000000035731fc  main+208                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     /b/c/b/android_n5x_swarming_rel/src/chrome/test/base/run_all_unittests.cc:30
  v------>  testing::android::JNI_NativeTest_RunTests(_JNIEnv*, base::android::JavaParamRef<_jobject*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jstring*> const&, base::android::JavaParamRef<_jobject*> const&, base::android::JavaParamRef<_jstring*> const&)                                                                                                                                                                                /b/c/b/android_n5x_swarming_rel/src/testing/android/native_test/native_test_launcher.cc:131
  000000000356e0c0  Java_org_chromium_native_1test_NativeTest_nativeRunTests+688                                                                                                                                                                                                                                                                                                                                                                                                                                                                 /b/c/b/android_n5x_swarming_rel/src/out/Release/gen/testing/android/native_test/native_test_jni_headers/testing/jni/NativeTest_jni.h:56
  00000000021ae708  offset 0x14cf000) (void org.chromium.native_test.NativeTest.nativeRunTests(java.lang.String, java.lang.String, java.lang.String, android.content.Context, java.lang.String)+268                                                                                                                                                                                                                                                                                                                                              /data/app/org.chromium.native_test-1/oat/arm64/base.odex
  00000000021af728  offset 0x14cf000) (void org.chromium.native_test.NativeTest.runTests(android.app.Activity)+300                                                                                                                                                                                                                                                                                                                                                                                                                               /data/app/org.chromium.native_test-1/oat/arm64/base.odex
  00000000021ae5b8  offset 0x14cf000) (void org.chromium.native_test.NativeTest.access$000(org.chromium.native_test.NativeTest, android.app.Activity)+76                                                                                                                                                                                                                                                                                                                                                                                         /data/app/org.chromium.native_test-1/oat/arm64/base.odex
  000000000323a2bc  offset 0x14cf000) (void org.chromium.native_test.NativeTest$1.run()+128                                                                                                                                                                                                                                                                                                                                                                                                                                                      /data/app/org.chromium.native_test-1/oat/arm64/base.odex
  00000000745a26e0  offset 0x1ed8000                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             /data/dalvik-cache/arm64/system@framework@boot.oat


Is there some sort of pervasive problem in the Android test suites where they'll get randomly killed due to the system OOM killer?

re your first question, on looking at the task log, the bot timed out on pushing data dependencies *twice* and didn't start running tests until almost 14 minutes in to the shard as a result:

  I  820.627s Main  Using external sharding settings. This is shard 0/2
  I  820.628s Main  STARTING TRY #1/3

Looking into why.
Components: Infra>Client>Chrome
Moving Infra>Client>Android -> Infra>Client>Chrome+OS=Android
Components: -Infra>Client>Android
Components: -Infra>Client>Chrome
Forked the data dependency pushing part into https://bugs.chromium.org/p/chromium/issues/detail?id=791632

Comment 7 by k...@chromium.org, Feb 15 2018

Cc: -k...@chromium.org

Sign in to add a comment