The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/421df92ae3d4853247a09280144b1467f07d8f7d

commit 421df92ae3d4853247a09280144b1467f07d8f7d
Author: Nghia Nguyen <nednguyen@google.com>
Date: Mon May 21 14:58:04 2018

Improve error handling when browser cannot be found in serially_executed_browser_test_case

Bug:  chromium:845147 
Change-Id: Id5c483a8cea1042766ddc8beee57ae235b50f923
Reviewed-on: https://chromium-review.googlesource.com/1067020
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/421df92ae3d4853247a09280144b1467f07d8f7d/telemetry/telemetry/testing/serially_executed_browser_test_case.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/203aae348ec33b6847f75aae47b914311471151e

commit 203aae348ec33b6847f75aae47b914311471151e
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Mon May 21 18:19:11 2018

Roll src/third_party/catapult/ 7ca7a59f0..421df92ae (1 commit)

https://chromium.googlesource.com/catapult.git/+log/7ca7a59f0202..421df92ae3d4

$ git log 7ca7a59f0..421df92ae --date=short --no-merges --format='%ad %ae %s'
2018-05-21 nednguyen Improve error handling when browser cannot be found in serially_executed_browser_test_case

Created with:
  roll-dep src/third_party/catapult
BUG= chromium:845147 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I8be4845e37314ec28da9d60ed848ddaeacb54626
Reviewed-on: https://chromium-review.googlesource.com/1067438
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#560307}
[modify] https://crrev.com/203aae348ec33b6847f75aae47b914311471151e/DEPS

Thanks Ned!

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/3020 has the improved message, but this is still reported as test failure. Is it possible to report this as an infra failure?

ynovikov@ I would file another bug for it as it's in the domain of swarming. It looks like it was still trying to trigger job even though there are no healthy bots available

7 bots could possibly run this task (0 busy, 4 dead, 3 quarantined, 0 maintenance)
(https://chromium-swarm.appspot.com/task?id=3da5723034773010&refresh=10&show_raw=1)

It's kind of between domains, so I'm not sure who to assign this to.

"7 bots could possibly run this task (0 busy, 4 dead, 3 quarantined, 0 maintenance)" is irrelevant, looks like "dead" and "quarantined" are not mutually exclusive, so there are 3 bots still available.

Anyway, you can see that this task was run on a swarmed bot, so no capacity issues here. However, the state of the task is "COMPLETED (FAILURE)", which is shown as test failure on the bot webpage. I would like the task have a state similar to BOT_DIED. Maybe something like BAD_USB. Which will show as infra failure on the bot page. Not sure if anything is required from telemetry to report this to swarming, or this should be analyzed from the logs somehow.

Re #7: What you're describing is something I suggested in https://groups.google.com/a/google.com/d/msg/luci-eng/qskxPXIqy-g/__9zAay2CAAJ

My original design (ie: marking these tasks as infra failures) never got buy-in from owners, and whatever middle ground we had agreed on was too insufficient and/or non-trivial that I never got around to implementing it :/

I hope to *someday* revisit that effort with the end-goal of having these types of tasks auto-retried. But for now, we have to settle for normal failures.