+Dirk: The stack came from typ. Using pdb, I found the error originates from https://github.com/dpranke/typ/blob/master/typ/runner.py#L391

Found the problem:
Name collision between https://github.com/catapult-project/catapult/blob/a8a6fb7ea065c2df8ae129ad05ac80ba6ae7cc3a/tracing/tracing_build/benchmarks.py & tools/perf/benchmarks/

This happens because of Randy's CL that add tracing_build to telemetry's sys.path (https://codereview.chromium.org/1980773002). For now, I will rename tracing/tracing_build/benchmarks.py to run_benchmarks

Yeah, I'm not sure typ could do much better in this situation, but if you have suggestions I'm open to them.

@Dirk: I think this is a problem with python module management. Though do you think it's possible for typ to blow up early (or warn) if it discovers duplicate package/module names? It took me a while to debug because of the error "ImportError: No module named benchmark_smoke_unittest" seems quite cryptic.

Catapult dep roller still stuck because of another set of failure in telemetry_perf_unittest.

  Traceback (most recent call last):
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\tools\perf\benchmarks\benchmark_smoke_unittest.py", line 73, in BenchmarkSmokeTest
      self.assertEqual(0, SinglePageBenchmark().Run(options),
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\third_party\catapult\telemetry\telemetry\benchmark.py", line 91, in Run
      return story_runner.RunBenchmark(self, finder_options)
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 324, in RunBenchmark
      results.PrintSummary()
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\third_party\catapult\telemetry\telemetry\internal\results\page_test_results.py", line 240, in PrintSummary
      output_formatter.Format(self)
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\third_party\catapult\telemetry\telemetry\internal\results\html_output_formatter.py", line 173, in Format
      self._SaveResults(html)
    File "e:\b\swarm_slave\work\isolated\runq5_gvs\third_party\catapult\telemetry\telemetry\internal\results\html_output_formatter.py", line 93, in _SaveResults
      self._output_stream.write(results)
    File "e:\b\depot_tools\python276_bin\lib\codecs.py", line 691, in write
      return self.writer.write(data)
    File "e:\b\depot_tools\python276_bin\lib\codecs.py", line 351, in write
      data, consumed = self.encode(object, self.errors)
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 55035: ordinal not in range(128)

https://codereview.chromium.org/1998943002/ should fix it.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/461a6b771643f6095a980b1f0605fd7480ab108b

commit 461a6b771643f6095a980b1f0605fd7480ab108b
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri May 20 07:44:57 2016

Roll src/third_party/catapult/ caaf30d6a..127940067 (11 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/caaf30d6a5c5..1279400675a3

$ git log caaf30d6a..127940067 --date=short --no-merges --format='%ad %ae %s'

BUG= 613421 , 613050 , 352807 , 352807 

TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/1995213002
Cr-Commit-Position: refs/heads/master@{#395028}

[modify] https://crrev.com/461a6b771643f6095a980b1f0605fd7480ab108b/DEPS

@nednguyen - I'm not sure that typ can do that check, since it doesn't necessarily know what names modules are going to be imported as? Maybe there's some way to do that that I don't know about, though ...