Based on the log in https://chrome-swarming.appspot.com/task?id=3ece17e32a46a710&refresh=10&show_raw=1, the failure of memory.desktop benchmark caused the whole benchmark run loop to stop. 

The failing stack is:
Traceback (most recent call last):
  File "..\..\testing\scripts\run_performance_tests.py", line 268, in <module>
    sys.exit(main())
  File "..\..\testing\scripts\run_performance_tests.py", line 244, in main
    False, stories=stories) or return_code)
  File "..\..\testing\scripts\run_performance_tests.py", line 132, in execute_benchmark
    args, per_benchmark_args, is_histograms))
  File "c:\b\s\w\ir\testing\scripts\run_telemetry_benchmark_as_googletest.py", line 170, in run_benchmark
    shutil.rmtree(tempfile_dir)
  File "c:\b\s\w\ir\.swarming_module\bin\Lib\shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "c:\b\s\w\ir\.swarming_module\bin\Lib\shutil.py", line 250, in rmtree
    os.remove(fullname)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\b\\s\\w\\itebecpm\\tmp6yz3tctelemetry\\benchmark_log.txt'
Failed to delete c:\b\s\w\ir (32 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete c:\b\s\w\ir (32 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 4 seconds.


So the root cause is even the benchmark run subprocess crashes, it has some leaky subprocesses that are holding opened file handles to the temp directory that we want to remove

The short term fix for this is to add ignore_errors to rmtree function call. Longer term, we should make sure run_telemetry_benchmark_as_googletest clean up all subprocesses of test