Thanks for filing. Here's what happened:

In the initial run of 12 shards, 1 shard failed very early:
https://chromium-swarm.appspot.com/task?id=41c3262912e67010&refresh=10&show_raw=1

"""
17:33:50: ERROR: Reboot has not completed after 180 seconds; giving up.
cros_run_vm_test: Unhandled exception:
Traceback (most recent call last):
  File "/b/s/w/ir/third_party/chromite/bin/cros_run_vm_test", line 169, in <module>
    DoMain()
  File "/b/s/w/ir/third_party/chromite/bin/cros_run_vm_test", line 165, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/b/s/w/ir/third_party/chromite/lib/commandline.py", line 912, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/b/s/w/ir/third_party/chromite/scripts/cros_run_vm_test.py", line 15, in main
    return cros_test.CrOSTest(opts).Run()
  File "/b/s/w/ir/third_party/chromite/lib/cros_test.py", line 73, in Run
    self._Deploy()
  File "/b/s/w/ir/third_party/chromite/lib/cros_test.py", line 136, in _Deploy
    self._device.RunCommand(deploy_cmd)
  File "/b/s/w/ir/third_party/chromite/lib/vm.py", line 331, in RunCommand
    return cros_build_lib.RunCommand(*args, **kwargs)
  File "/b/s/w/ir/third_party/chromite/lib/cros_build_lib.py", line 646, in RunCommand
    raise RunCommandError(msg, cmd_result)
chromite.lib.cros_build_lib.RunCommandError: return code: 1; command: deploy_chrome --force --build-dir /b/s/w/ir/out_amd64-generic/Release --process-timeout 180 --to localhost --port 9222 --board amd64-generic --cache-dir /b/s/w/ir/build/cros_cache
cmd=['deploy_chrome', '--force', '--build-dir', '/b/s/w/ir/out_amd64-generic/Release', '--process-timeout', '180', '--to', 'localhost', '--port', '9222', '--board', 'amd64-generic', '--cache-dir', '/b/s/w/ir/build/cros_cache']
"""

This task had no isolated output. This should have resulted in TEST_INVALID_RESULTS for the entire test suite. However, instead we marked the test suite as a failure with no failing tests:
"""
* Passed: 933 (933 expected, 0 unexpected)
* Skipped: 119 (119 expected, 0 unexpected)
* Failed: 0 (0 expected, 0 unexpected)
* Flaky: 0 (0 expected, 0 unexpected)
"""

This causes us to run 'retry with patch'. All tests passed. Since there were no failing tests, we skipped 'retry with patch' and marked the build as a success. 

The bug here is that the chromium_tests recipe is failing to handle the case where a swarming task fails and has no output.

The swarming summary for the task shows 12 shards, where shard 7 has 

failure:true
state='COMPLETED'

https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8927206925038076096/+/steps/telemetry_unittests__with_patch_/0/logs/swarming.summary/0

I would expect load_share_json to return None:
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/swarming/resources/standard_gtest_merge.py?q=scripts/slave/recipe_modules/swarming/resources/standard_gtest_merge.py&sq=package:chromium&g=0&l=87

thus causing merged['missing_shards'].append(index) to trigger. However, we don't see "missing_shards', so we know that this logic is not correctly triggering.

This suggests that load_shard_json() is somehow returning a non-empty JSON dictionary for a shard that has no swarming output.

tikuta, could you investigate?

Yes, I take a look.

At first, the test is not gtest, it is processed by standard_isolated_script_merge.py.
But we don't detect missing shards in the script.
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/swarming/resources/standard_isolated_script_merge.py?l=23&rcl=20af509a4bc473a8367d230b35839cd9477d2f1c

And we don't pass shard json of task that does not have any isolated output.
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/swarming/resources/collect_task.py?l=113&rcl=20af509a4bc473a8367d230b35839cd9477d2f1c

So, in this case, I think we need to detect missing shards in standard_isolated_script_merge.py like we do in standard_gtest_merge.py from summary.json.

jbudorick, could you take this?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/6a83fce3c0dc225cfa5ad336a74c3931c4354aae

commit 6a83fce3c0dc225cfa5ad336a74c3931c4354aae
Author: erikchen <erikchen@chromium.org>
Date: Mon Dec 17 21:18:44 2018

Handle missing swarming output in standard_isolated_script_merge.py.

Previously, if swarming output was missing for a shard, then
standard_isolated_script_merge would simply consider the shard a success [with
no tests run]. This CL updates standard_isolated_script_merge to notice that
there is missing shard output, and to set the correct output flags
'missing_shards' and UNRELIABLE_RESULTS.

These flags are not yet parsed by swarming/api.py:_default_collect_step.

Bug: 915310
Change-Id: Iab6e52becd8f77225c54a8c4c7f2932ed368d830
Reviewed-on: https://chromium-review.googlesource.com/c/1380492
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/6a83fce3c0dc225cfa5ad336a74c3931c4354aae/scripts/slave/recipe_modules/swarming/resources/standard_isolated_script_merge.py
[modify] https://crrev.com/6a83fce3c0dc225cfa5ad336a74c3931c4354aae/scripts/slave/recipe_modules/swarming/unittests/standard_isolated_script_merge_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/4d78a7ce476b11ed4b6bf2a9876803532402e37d

commit 4d78a7ce476b11ed4b6bf2a9876803532402e37d
Author: erikchen <erikchen@chromium.org>
Date: Tue Dec 18 14:55:35 2018

Update presentation for isolated swarming tasks with missing shards.

Previously, isoalted scripts with missing shards were treated as successes. This
CL updates the presentation [but not functionality] of these tasks to show that
there is missing shard output.

Bug: 915310
Change-Id: I1afb4cd9d8af5a539807172c4564b0c77999be29
Reviewed-on: https://chromium-review.googlesource.com/c/1381053
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Erik Chen <erikchen@chromium.org>

[modify] https://crrev.com/4d78a7ce476b11ed4b6bf2a9876803532402e37d/scripts/slave/README.recipes.md
[modify] https://crrev.com/4d78a7ce476b11ed4b6bf2a9876803532402e37d/scripts/slave/recipe_modules/swarming/api.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/e2b47d2331bdf81c66d1ee53264d3373d72150c2

commit e2b47d2331bdf81c66d1ee53264d3373d72150c2
Author: erikchen <erikchen@chromium.org>
Date: Wed Dec 19 16:31:28 2018

Fix class hierarchy of SwarmingAndroidPerfTest.

This CL is a refactor with no intended behavior change.

Previously, SwarmingAndroidPerfTest inherited from SwarmingGTestTest, despite
overriding create_task(). It was incorrectly using SwarmingGTestTest's
implementation of validate_task_results(), which always returned valid=False.
This worked because SwarmingGTestTest inherits from SwarmingTest, which was
incorrectly marking valid=False results as a successful test step.

We could have fixed this by also overriding validate_task_results(). However, at
that point, there's really no point in inheriting from SwarmingGTestTest at all,
as that just creates confusion. This CL simply allows SwarmingAndroidPerfTest to
directly inherit from SwarmingTest.

Bug: 915310
Change-Id: I6a012f489b038a1ff68ce0c7a448e20847986486
Reviewed-on: https://chromium-review.googlesource.com/c/1382802
Reviewed-by: Oleh Prypin <oprypin@chromium.org>
Commit-Queue: Erik Chen <erikchen@chromium.org>

[modify] https://crrev.com/e2b47d2331bdf81c66d1ee53264d3373d72150c2/scripts/slave/recipe_modules/webrtc/steps.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/dd3830941096af85163e32e7c47b475fd21e031e

commit dd3830941096af85163e32e7c47b475fd21e031e
Author: erikchen <erikchen@chromium.org>
Date: Wed Dec 19 17:17:20 2018

Correctly handle UNRELIABLE results from swarming isolated scripts.

Previously, the global tag UNRELIABLE_RESULTS was ignored when returned by
swarming isolated scripts. Furthermore, the invalid test runs were not
generating a step failure.

Change-Id: Idd933859a3ffce40808238327e80132b78059d45
Bug: 915310
Reviewed-on: https://chromium-review.googlesource.com/c/1381055
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/dd3830941096af85163e32e7c47b475fd21e031e/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.py
[modify] https://crrev.com/dd3830941096af85163e32e7c47b475fd21e031e/scripts/slave/recipe_modules/chromium_tests/steps.py