There's always an "UNKNOWN" result before it passes. But I couldn't find where in the logs the run failed, only seeing the "OK" result. This is only happening on this builder. flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=ipc_tests&tests=IPCChannelProxyTest.BadMessageOnIPCThread result details: https://00e9e64bac4c73df4b381381126a064a30e868a9a4585639cc-apidata.googleusercontent.com/download/storage/v1/b/chromium-result-details/o/html%2Fipc_tests_KitKat%20Phone%20Tester%20(dbg)_1147_2017_11_14_T21_36_35-UTC?qk=AD5uMEsWUGe8Ird4EN_lVLwQiAdpbpEjy1wkaq6DBJjWgi9db3o5W47GWm59f-A5Dd2hU3LMQa9atgDNRduPcIFZKZ5XcLPYr8Jb0lfaQslzZLcJUe5-YcB_TWWY6ffnmXnktB65X0-3sb6FyCakL_XdtFTYoK_G9zLOsx2vR3aeaQboNy8Lg7ate3HKu6sVouGCo1EuNC-y26NSy-FO27VZYtyG66OGq19Z7OAqBDDny-YAU2fP8rfe2knmGml5RvSY_VyxAUTWm7UAY7RbcxxTAuLKJeJtd0lbg5QHnNirCaghNa9V5A33VgUUN15kePh00WlbfBr8-BTXPnz3RL-G_FnI4LIuxixdJIJ3-SITZfY72lYryuGXbzkscAycX0lGtsNBPXrFifUK4IyT-w9P37rGi4o9zvIzHHP7G8czAcN3WjMuYJ8weC5S96v3-26-fBlu9GqyJKkgWF6HoB-DlanBz98lGd4PStTpRoA39O0maVJgIp-KD-Nv6iaYsqpRL_Fk19HMT4eoHHFU4WW75dgCsEXYrV_qwN0FBg9DEHPuknjFewtfwJgyxSI6_pxAJ5yEdgFQYKD5hPkIvjPCLBm16tEYPw9XgR_Wdf-GwJuUhrtrNYYRbmTA525uWNrqpShb5qZiCUCGS57o5RPT46gq8UJUmShm365_Vd2M2fFnhvhzEsccWifqjFP6zPxw3SLRpUKqblHdB_BEh5OMooZjqUqhZ4SdHBEyIWF2pqFzJLMhVwU6j66lLjOY8OQ9PZsBb5YJKoC8wqQ4O3ar4kfhzCMKZiehLba2a5o9MQN5fc28qVvy_ziOCbaF6dlZT7d_9b9gqpi46n-y8I_lXQM1JVfUdA logs: https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium.android%2FKitKat_Phone_Tester__dbg_%2F1147%2F%2B%2Frecipes%2Fsteps%2Fipc_tests_on_Android%2F0%2Fstdout
I see that the 1st try for all tests failed in the logs, error message is: I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 134217749 from logs: I 8.958s Main STARTING TRY #1/3 I 8.959s Main Will run 78 tests on 1 devices: 03852b9e003b9f11 I 8.959s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( test -e /data/tombstones );echo %$?' I 8.975s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /data/tombstones/ );echo %$?' W 8.990s run_tests_on_device(03852b9e003b9f11) No tombstones to clear. I 8.990s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( pm dump org.chromium.native_test | grep dataDir=; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?' I 10.507s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( mkdir -p /sdcard/tmp-d991ecb1fc101 );echo %$?' I 10.523s run_tests_on_device(03852b9e003b9f11) flags: I 10.523s run_tests_on_device(03852b9e003b9f11) Getting logdog client. I 10.523s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 logcat -c I 10.531s record_to_file [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 logcat -v threadtime chromium:v 'cr_*:v' DEBUG:I StrictMode:D I 10.532s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 push /b/swarming/w/itZnvKMr/tmp6ptI8l /data/local/tmp/temp_file-57cfef75b5acf I 10.539s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 push /b/swarming/w/itZnvKMr/tmpoDyfkg /data/local/tmp/temp_file-658b28cffbab.sh I 10.544s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) Large shell command will be run from file: ( p=org.chromium.native_test;am instrument -w -e "$p".NativeTestInstrumentationTestRunner.ShardNanoTimeout 120000000000 -e "$p".NativeTestInstrumentationTestRunner.NativeTestActivity "$p".NativeUnitTestActivity -e "$p".NativeTest.RunInSubThread 1 -e "$p".NativeTestInstrumentationTestRunner.TestList /data/local/tmp/temp_file-57cfef75b5acf -e "$p".NativeTestInstrumentationTestRunner.StdoutFile /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out "$p"/"$p".NativeTestInstrumentationTestRunner )>/data/loca ... I 10.544s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( sh /data/local/tmp/temp_file-658b28cffbab.sh );echo %$?' I 13.953s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-658b28cffbab.sh' I 13.953s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 pull /data/local/tmp/temp_file-ed048a4e9de73 /b/swarming/w/itZnvKMr/tmpNbIPej/tmp_ReadFileWithPull I 14.006s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-ed048a4e9de73' I 14.041s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?' I 14.057s run_tests_on_device(03852b9e003b9f11) /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out size on device: 308 I 14.057s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?' I 14.073s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( cat /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out );echo %$?' I 14.089s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out' I 14.090s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-57cfef75b5acf' I 14.154s run_tests_on_device(03852b9e003b9f11) Logcat saved to https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F39d369f22557ed11%2F%2B%2Flogcat_logcat_8575280885749269098_20171114T125451-UTC_03852b9e003b9f11 I 14.154s archive Writing text to logdog stream, logcat_logcat_8575280885749269098_20171114T125451-UTC_03852b9e003b9f11 I 14.154s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/data/org.chromium.native_test/temp_file-c7dc02d109aa9.xml' I 14.155s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( pm clear org.chromium.native_test );echo %$?' I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 134217749 I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 4294967295 I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 2147483667 I 14.758s run_tests_on_device(03852b9e003b9f11) [WARNING:ipc_message_attachment_set.cc(68)] Cannot add file descriptor. MessageAttachmentSet full. I 14.759s run_tests_on_device(03852b9e003b9f11) Finished running tests on this device. I 14.763s Main FINISHED TRY #1/3 Assigning to owner of test for investigation.
Comment 1 by aluo@chromium.org
, Nov 16 2017Owner: jam@chromium.org
I see that the 1st try for all tests failed in the logs, error message is: I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 134217749 from logs: I 8.958s Main STARTING TRY #1/3 I 8.959s Main Will run 78 tests on 1 devices: 03852b9e003b9f11 I 8.959s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( test -e /data/tombstones );echo %$?' I 8.975s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /data/tombstones/ );echo %$?' W 8.990s run_tests_on_device(03852b9e003b9f11) No tombstones to clear. I 8.990s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( pm dump org.chromium.native_test | grep dataDir=; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?' I 10.507s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( mkdir -p /sdcard/tmp-d991ecb1fc101 );echo %$?' I 10.523s run_tests_on_device(03852b9e003b9f11) flags: I 10.523s run_tests_on_device(03852b9e003b9f11) Getting logdog client. I 10.523s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 logcat -c I 10.531s record_to_file [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 logcat -v threadtime chromium:v 'cr_*:v' DEBUG:I StrictMode:D I 10.532s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 push /b/swarming/w/itZnvKMr/tmp6ptI8l /data/local/tmp/temp_file-57cfef75b5acf I 10.539s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 push /b/swarming/w/itZnvKMr/tmpoDyfkg /data/local/tmp/temp_file-658b28cffbab.sh I 10.544s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) Large shell command will be run from file: ( p=org.chromium.native_test;am instrument -w -e "$p".NativeTestInstrumentationTestRunner.ShardNanoTimeout 120000000000 -e "$p".NativeTestInstrumentationTestRunner.NativeTestActivity "$p".NativeUnitTestActivity -e "$p".NativeTest.RunInSubThread 1 -e "$p".NativeTestInstrumentationTestRunner.TestList /data/local/tmp/temp_file-57cfef75b5acf -e "$p".NativeTestInstrumentationTestRunner.StdoutFile /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out "$p"/"$p".NativeTestInstrumentationTestRunner )>/data/loca ... I 10.544s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( sh /data/local/tmp/temp_file-658b28cffbab.sh );echo %$?' I 13.953s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-658b28cffbab.sh' I 13.953s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 pull /data/local/tmp/temp_file-ed048a4e9de73 /b/swarming/w/itZnvKMr/tmpNbIPej/tmp_ReadFileWithPull I 14.006s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-ed048a4e9de73' I 14.041s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?' I 14.057s run_tests_on_device(03852b9e003b9f11) /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out size on device: 308 I 14.057s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?' I 14.073s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( cat /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out );echo %$?' I 14.089s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /storage/emulated/legacy/temp_file-4467b154140ec.gtest_out' I 14.090s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/local/tmp/temp_file-57cfef75b5acf' I 14.154s run_tests_on_device(03852b9e003b9f11) Logcat saved to https://luci-logdog.appspot.com/v/?s=chromium%2Fandroid%2Fswarming%2Flogcats%2F39d369f22557ed11%2F%2B%2Flogcat_logcat_8575280885749269098_20171114T125451-UTC_03852b9e003b9f11 I 14.154s archive Writing text to logdog stream, logcat_logcat_8575280885749269098_20171114T125451-UTC_03852b9e003b9f11 I 14.154s TimeoutThread-1-for-delete_temporary_file(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell 'rm -f /data/data/org.chromium.native_test/temp_file-c7dc02d109aa9.xml' I 14.155s TimeoutThread-1-for-run_tests_on_device(03852b9e003b9f11) [host]> /b/swarming/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 03852b9e003b9f11 shell '( pm clear org.chromium.native_test );echo %$?' I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 134217749 I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 4294967295 I 14.758s run_tests_on_device(03852b9e003b9f11) [ERROR:ipc_channel_reader.cc(212)] IPC message is too big: 2147483667 I 14.758s run_tests_on_device(03852b9e003b9f11) [WARNING:ipc_message_attachment_set.cc(68)] Cannot add file descriptor. MessageAttachmentSet full. I 14.759s run_tests_on_device(03852b9e003b9f11) Finished running tests on this device. I 14.763s Main FINISHED TRY #1/3 Assigning to owner of test for investigation.