I should be pretty simple. Juan can give the guidance while I am out

There is a generate_system_health_csv script [1], I think here [2] you should be able to add e.g. a new column with

  '|'.join(t.name for r in s.TAGS)

to get all tags of a story s.

[1]: https://cs.chromium.org/chromium/src/tools/perf/generate_system_health_csv
[2]: https://cs.chromium.org/chromium/src/tools/perf/core/system_health_csv_generator.py?rcl=d1471ff1795de86c9e4853fbc6e854a503c8e920&l=74

Thanks perezju@. Is this only for system_health, or applicable to all benchmarks?

It's supposed to be only for system health. But it might work with a bit of effort (mostly removing or commenting out lines that fail) for other benchmarks by tweaking the list over here:
https://cs.chromium.org/chromium/src/tools/perf/core/system_health_csv_generator.py?rcl=d1471ff1795de86c9e4853fbc6e854a503c8e920&l=61

Unfortunately not all stories expose the same properties as system health stories; although this is something we're hoping to clean up.

Actually what we need is to query the tags and/or story names of all benchmarks reported by `run_benchmark list`. Can you make it work?

The discussion on https://chromium-review.googlesource.com/c/chromium/src/+/963770 becomes more general than the CL itself so probably let's continue the discussion here?

So...
1. Are we OK to land find_story_tags first and refactor it later?
2. Do we want to integrate it into run_benchmark or an integrated script with list_benchmarks, and generate_system_health_csv?

For 1, I prefer to land it first and refactor later because 2 may take a considerable amount of time; The interface hasn't be decided yet.

I generally want us to figure out the right way to do things before implementing them. We should always strive to make the code healthy at first try, once it's in, it would be harder to fix them.

I would reuse existing `run_benchmark list` command. There are two steps:
1) Make sure `run_benchmark list` can list all available benchmarks regardless of current platform. Right now ``run_benchmark list` only list runnable benchmarks for the host platform.
2) Add an option like `--show-story-tags` so that the command will list the benchmarks & a comma separated list of tags for each benchmark

My opinion is that "run_benchmark" is the CLI for developers to run benchmarks and maybe get some quick information about them, to figure out what they can and want to run (in this context, e.g. showing only benchmarks for the current host platform is a good idea). And yeah, it would be great to implement even more options to make it easier for the developer to interactively explore and learn about the stories or tags available for a benchmark through that interface in a human readable way.

But I don't think that is what laszio needs for their use case, which is more like getting a machine-readable snapshot of all benchmarks/stories/tags available for the whole of tools/perf. And that's why I believe that some hypothetical "export_benchmark_data" script (with no options!) which just spits out a csv with all the data for further processing would be more useful.

#9: run_benchmark list --json used to output the machine readable form of the benchmarks, so I don't see why we shouldn't make the script output support different type format. 

I prefer this option for better code sharing. If we already have a place that does these type of benchmark info display, we should expand it. It's also good for usability, I think we have a bit too many utility scripts in tools/perf at the moment: https://cs.chromium.org/search/?q=%22env+vpython%22+f:tools/perf&type=cs

Ok, +1 to have this as "run_benchmark list --json" if that would fit the needs that started this discussion.

"run_benchmark list --json" sounds perfect to me. In fact, we have to parse the human oriented output in our tools, which is subject to changes and is fragile.

We are interested in when this will be available. I assume that deciding the fields may take some time, where I can't really help.

laszio: you can add that support to https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py?rcl=16322a374abb6c805d8d3fd7ed2daf11a2579292&l=124

You can see the how the old code look like in https://chromium.googlesource.com/catapult.git/+/7298e9f90ac7ec7e6f9a644b4f4d270fa195657d/telemetry/telemetry/benchmark_runner.py#135

Looks like Ting-Yuan is already on this :)

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/6ab2bba1a3cae99d1da26ddcf53c6b33a5bb48e4

commit 6ab2bba1a3cae99d1da26ddcf53c6b33a5bb48e4
Author: Ting-Yuan Huang <laszio@chromium.org>
Date: Tue Mar 20 23:06:59 2018

[Telemetry] Support outputing json in benchmark_runner

The default is unchanged and still in a human readable format.

Bug:  chromium:819969 
Change-Id: Id511540395eefbbf4e324d8657cfc51d12c69e69
Reviewed-on: https://chromium-review.googlesource.com/967453
Commit-Queue: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/6ab2bba1a3cae99d1da26ddcf53c6b33a5bb48e4/telemetry/telemetry/benchmark_runner.py
[modify] https://crrev.com/6ab2bba1a3cae99d1da26ddcf53c6b33a5bb48e4/telemetry/telemetry/benchmark_runner_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6aea1cac122502499713bfc182d0454d0f325fa9

commit 6aea1cac122502499713bfc182d0454d0f325fa9
Author: catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Mar 23 14:48:06 2018

Roll src/third_party/catapult/ 734f737c6..c4e9b1332 (14 commits)

https://chromium.googlesource.com/catapult.git/+log/734f737c6b57..c4e9b1332818

$ git log 734f737c6..c4e9b1332 --date=short --no-merges --format='%ad %ae %s'
2018-03-23 nednguyen Revert "Add --enable-automation flag in GetFromBrowserOptions()"
2018-03-22 dtu [pinpoint] Set auto_explore == True for "patch jobs".
2018-03-21 ynovikov Remove a note on android_optional_gpu_tests_rel from manual rolls doc.
2018-03-15 dproy Reland "More precise self time calculation"
2018-03-21 nednguyen Ignore fetchts file
2018-03-20 laszio [Telemetry] Support outputing json in benchmark_runner
2018-03-19 kjharland [Dashboard] Whitelist prod Fuchsia Garnet builder.
2018-03-20 benjhayden Fix sorting results.html
2018-03-19 horo Add --enable-automation flag in GetFromBrowserOptions()
2018-03-19 eakuefner [Dashboard] Avoid creating empty rows/histograms/tests if histograms are empty
2018-03-16 dtu [pinpoint] Infer "repository" parameter from configuration.
2018-03-19 chiniforooshan First set of thread times metrics in TBMv2
2018-03-16 erikchen Replace --enable-heap-profiling with --memlog equivalent.
2018-03-19 wangxianzhu Fix visual rect support for cc::DisplayItemList

Created with:
  roll-dep src/third_party/catapult
BUG=chromium:822258, chromium:786572 , chromium:731979 ,chromium:821531,chromium:821521, chromium:819969 ,chromium:822258, chromium:627461 , chromium:822843 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I8c0025df6198bf9b1662bb8d78e45741d66f6aff
Reviewed-on: https://chromium-review.googlesource.com/977884
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#545441}
[modify] https://crrev.com/6aea1cac122502499713bfc182d0454d0f325fa9/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/bce75ec93af71aa34b1718b73503a7781b79318f

commit bce75ec93af71aa34b1718b73503a7781b79318f
Author: nednguyen <nednguyen@google.com>
Date: Mon Mar 26 15:29:58 2018

Supress Telemetry pylint error

Bug:  chromium:819969 

TBR=perezju@chromium.org, iannuci@chromium.org

Change-Id: Ib707a83b345067de606c477c9dc239032b150892
Reviewed-on: https://chromium-review.googlesource.com/980238
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>

[modify] https://crrev.com/bce75ec93af71aa34b1718b73503a7781b79318f/telemetry/telemetry/internal/util/external_modules.py
[modify] https://crrev.com/bce75ec93af71aa34b1718b73503a7781b79318f/telemetry/telemetry/internal/platform/posix_platform_backend.py

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/2943ee372877a1133027d036d6dc54232277bcc4

commit 2943ee372877a1133027d036d6dc54232277bcc4
Author: Ting-Yuan Huang <laszio@chromium.org>
Date: Mon Mar 26 16:42:08 2018

[Telemetry] include story tags in benchmark_runner list

Bug:  chromium:819969 
Change-Id: Iebdfaa00c4055270e6a5590f947f02897d9cfce6
Reviewed-on: https://chromium-review.googlesource.com/972630
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>

[modify] https://crrev.com/2943ee372877a1133027d036d6dc54232277bcc4/telemetry/telemetry/benchmark_runner.py
[modify] https://crrev.com/2943ee372877a1133027d036d6dc54232277bcc4/telemetry/telemetry/benchmark_runner_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/13822b93deb44b969f5aa180d18f75ea81b28b31

commit 13822b93deb44b969f5aa180d18f75ea81b28b31
Author: catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Mar 27 03:32:28 2018

Roll src/third_party/catapult/ bffbf166f..767f070f1 (6 commits)

https://chromium.googlesource.com/catapult.git/+log/bffbf166f004..767f070f1df4

$ git log bffbf166f..767f070f1 --date=short --no-merges --format='%ad %ae %s'
2018-03-26 eakuefner [Telemetry] Add more diagnostics to benchmark_total_duration
2018-03-26 dtu [pinpoint] Add descriptions to try job dialogs.
2018-03-22 laszio [Telemetry] include story tags in benchmark_runner list
2018-03-26 dproy Get navigation info without using FrameLoader snapshots
2018-03-26 nednguyen Supress Telemetry pylint error
2018-03-26 eakuefner Revert "HistogramSet - Only merge histograms with stories."

Created with:
  roll-dep src/third_party/catapult
BUG= chromium:819969 , chromium:824761 , chromium:819969 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I9b5458944cce2430a96f4d92854bed716b4b23ed
Reviewed-on: https://chromium-review.googlesource.com/981594
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#545975}
[modify] https://crrev.com/13822b93deb44b969f5aa180d18f75ea81b28b31/DEPS