Findings: 
1. https://build.chromium.org/p/tryserver.chromium.android/builders/android_chromium_variable/builds/420/steps/steps/logs/err.text

2. Buildbucket set this try job as 'COMPLETED'

3. Although the try job exceptioned out, I can still get report from Logdog.


Need to fix the "File name too long" issue in 1 and also change on Findit side to avoid adding culprit for such try jobs.

2. If it is not expected to be COMPLETED, we may want to talk to nodir@ and check whether it is a bug on the buildbucket side.

3. There is no report log, why we still could retrieve the result?

For "File name too long", we could file a separate bug to track that and cc jbudorick@. I believe he will be able to help out or find someone else to fix it.

Possible workaround on our recipe side is to shorten the wrapping step names which contains the whole git hash like "test 011ef8d97d0b06877ca4cfc621d2e56f19aeef8c".

2. I'll contact nodir@ about this.

3. We added the report in finally block. I'm not sure why we don't see the step on build page, but the log exists.

3. Seems like a bug in Logdog integration with buildbot?

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/cb802a9f557b4b9031f212b2839f7bb66008e932

commit cb802a9f557b4b9031f212b2839f7bb66008e932
Author: Chan <chanli@chromium.org>
Date: Wed Apr 12 18:13:26 2017

[Findit] Only use culprits from report for test try jobs.

Bug:  710508 
Change-Id: Ie0f5e8458bcc55779cea62b67669bdcd87d9fe6b
Reviewed-on: https://chromium-review.googlesource.com/474972
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Commit-Queue: Chan Li <chanli@chromium.org>

[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/identify_try_job_culprit_pipeline.py
[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/start_try_job_on_demand_pipeline.py
[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/test/identify_try_job_culprit_pipeline_test.py

So the buildbucket should work normal, the result is "FAILURE" and failure_reason is "INFRA FAILURE". 

As for the report, I think Logdog works normal? It's more of a bug to me why there is no report and failure reason on build page like other build failures.

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/cb802a9f557b4b9031f212b2839f7bb66008e932

commit cb802a9f557b4b9031f212b2839f7bb66008e932
Author: Chan <chanli@chromium.org>
Date: Wed Apr 12 18:13:26 2017

[Findit] Only use culprits from report for test try jobs.

Bug:  710508 
Change-Id: Ie0f5e8458bcc55779cea62b67669bdcd87d9fe6b
Reviewed-on: https://chromium-review.googlesource.com/474972
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Commit-Queue: Chan Li <chanli@chromium.org>

[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/identify_try_job_culprit_pipeline.py
[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/start_try_job_on_demand_pipeline.py
[modify] https://crrev.com/cb802a9f557b4b9031f212b2839f7bb66008e932/appengine/findit/waterfall/test/identify_try_job_culprit_pipeline_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e

commit 1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e
Author: Chan <chanli@chromium.org>
Date: Fri Apr 14 20:36:07 2017

[Findit] Change test.py to deflake by running on good_revision.

1. Add a flag to allow defalke by running on good_revision.
  a. Only runs on tests that failed on the first revision in range.
  b. If some of the tests also failed on good_revision, remove them from culprits and add to flakes.
2. Also, only add culprits to report if the try job completes.
3. Add a field for iterations to run a test, default to 20.

Bug:  710508 
Change-Id: I92bb840ba62cee7d1b021eb3382eba82ef98fa5c
Reviewed-on: https://chromium-review.googlesource.com/474988
Commit-Queue: Chan Li <chanli@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_steps_multiple_culprits.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/unaffected_test_skipped_by_analyze.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_consecutive_culprits.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/only_one_test_passed.json
[add] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/flaky_tests.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/use_analyze_set_to_False_for_non_linear_try_job.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_culprit_in_last_sub_range.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/all_test_failed.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.py
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_tests_multiple_culprits.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/use_build_parameter_for_tests.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/none_swarming_tests.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_culprit_in_first_sub_range.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/all_test_passed.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/findit_culprit_in_middle_sub_range.json
[modify] https://crrev.com/1d58c5e2f2b5afbf0c163d49d52f352df8c0ca5e/scripts/slave/recipes/findit/chromium/test.expected/swarming_tests.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/4f31c979995894fc757210198e9cb8b2d8a77002

commit 4f31c979995894fc757210198e9cb8b2d8a77002
Author: Chan <chanli@chromium.org>
Date: Fri Apr 14 21:42:01 2017

[Findit] truncate revision in step name to prevent 'file name too long' error


Bug:710508

Change-Id: Ia01cb6c4c02c5ac81e0513bfc4a287b955666692
Reviewed-on: https://chromium-review.googlesource.com/476075
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>
Commit-Queue: Chan Li <chanli@chromium.org>

[modify] https://crrev.com/4f31c979995894fc757210198e9cb8b2d8a77002/scripts/slave/recipe_modules/findit/api.py
[add] https://crrev.com/4f31c979995894fc757210198e9cb8b2d8a77002/scripts/slave/recipes/findit/chromium/test.expected/use_abbreviated_revision_in_step_name.json
[modify] https://crrev.com/4f31c979995894fc757210198e9cb8b2d8a77002/scripts/slave/recipes/findit/chromium/test.py