Example, this run: https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20%28dbg%29/builds/34200 If you open the stdio of chrome_public_test_apk and search for "detected failure in": https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20%28dbg%29/builds/34200/steps/Instrumentation%20test%20chrome_public_test_apk/logs/stdio You'll find that CastStartStopTest#testStopFromNotification failed once and then succeeded on retry. Then going to the flakiness dashboard for the test: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=testStopFromNotification That run is reported as green and not red.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/999e16207089e4c849084033f74d6a1f2a95d00f commit 999e16207089e4c849084033f74d6a1f2a95d00f Author: jbudorick <jbudorick@chromium.org> Date: Mon May 23 15:23:45 2016 [Android] Expose each try result in test results JSON. BUG= 609588 Review-Url: https://codereview.chromium.org/1987763002 Cr-Commit-Position: refs/heads/master@{#395328} [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/PRESUBMIT.py [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/pylib/local/device/local_device_test_run.py [add] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/pylib/local/device/local_device_test_run_test.py [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/pylib/remote/device/remote_device_test_run.py [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/pylib/results/json_results.py [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/pylib/results/json_results_test.py [modify] https://crrev.com/999e16207089e4c849084033f74d6a1f2a95d00f/build/android/test_runner.py
Comment 1 by jbudorick@chromium.org
, May 10 2016Status: Started (was: Untriaged)