Agree with the need of some better documentation.

About your particular question, yes the benchmark can be broken up by "stories". Alerts and bugs from perf sheriffs are now being filed at the story level, and the output of bisect jobs shows the command required to run a single story.

Random example from https://bugs.chromium.org/p/chromium/issues/detail?id=660154#c5:

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 660154

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.stackoverflow system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_tools/load_tools_stackoverflow
Relative Change: 10.73%
Score: 99.9

You might still want to tweak that command a bit, e.g. set --pageset-repeat 3 at least, but the important part is the --story-filter argument.

Depending on the story with a 3x repetition the benchmark will take about 5 minutes to run, or 15 minutes for the "longer" running stories.

context for this is me trying to look into crbug.com/661504

> and the output of bisect jobs shows the command required to run a single story.

And if the bisect job fails? I think the command should be included when the bug is filed.

So looking at the results.html from yesterday comparing values before and after. The "name" column is not wide enough to display the entire name, and the differences are all at the end of the name, so all I see is a column of "memory:webview:all_processes:rep...". There is no way to resize the column afaict. Is there a better way to present the results right now?

> And if the bisect job fails? I think the command should be included when the bug is filed.

Also agree. Annie may be able to comment more on this, but this is one of the many things that we would like to see improved with respect to the perf sheriffing and regression diagnosing experience.

> Is there a better way to present the results right now?

Have you tried the new results2 output format? I think that if you have a recent catapult checkout this should be now enabled by default (when saying --output-format html), but Ethan and Ben can give you more details about it.

I don't suppose there is a way to convert results.html to results2?

> Have you tried the new results2 output format?

oh it's on by default already and the one I've been using, and that's the one causing problems because the width of the table cannot be adjsuted and is too short to display everything

Oh I think the latest version of results2 has a bug (was working fine last week..)

TypeError: Cannot set property 'group' of undefined
    at HTMLElement.set currentGroupKeys [as currentGroupKeys] (results.html:20284)
    at HTMLElement.restoreSetting_ (results.html:20215)
    at HTMLElement.set possibleGroups [as possibleGroups] (results.html:20232)
    at HTMLElement.updateGroups_ (results.html:27164)
    at HTMLElement.updateContents_ (results.html:27187)
    at HTMLElement.set histograms [as histograms] (results.html:27083)
    at HTMLElement.set histograms [as histograms] (results.html:27702)
    at HistogramImporter.displayHistograms (results.html:27785)
    at runTask (results.html:15956)
    at processRequests (results.html:15975)
results.html:15949 Cannot set property 'group' of undefined TypeError: Cannot set property 'group' of undefined
    at HTMLElement.set currentGroupKeys [as currentGroupKeys] (file:///android/chromium/src/tools/perf/results.html:20284:27)
    at HTMLElement.restoreSetting_ (file:///android/chromium/src/tools/perf/results.html:20215:29)
    at HTMLElement.set possibleGroups [as possibleGroups] (file:///android/chromium/src/tools/perf/results.html:20232:12)
    at HTMLElement.updateGroups_ (file:///android/chromium/src/tools/perf/results.html:27164:36)
    at HTMLElement.updateContents_ (file:///android/chromium/src/tools/perf/results.html:27187:12)
    at HTMLElement.set histograms [as histograms] (file:///android/chromium/src/tools/perf/results.html:27083:12)
    at HTMLElement.set histograms [as histograms] (file:///android/chromium/src/tools/perf/results.html:27702:36)
    at HistogramImporter.displayHistograms (file:///android/chromium/src/tools/perf/results.html:27785:27)
    at runTask (file:///android/chromium/src/tools/perf/results.html:15956:21)
    at processRequests (file:///android/chromium/src/tools/perf/results.html:15975:7)

Bo, I can't repro this exception at ToT. Can you please attach the results2 where you're seeing this issue?

As to the issue with column widths, this turns out to be a core bug in tracing's table widget, which there is an ongoing effort to fix: https://github.com/catapult-project/catapult/issues/2946

attached results.html that doesn't work for me on linux 55.0.2883.35. stuck at "Displaying Histogram table..."

short term workaround is just make the column wider then?

Deleted: results.html 1.0 MB

results.html 1.0 MB View Download

Looking.

It doesn't load locally for me either, even after trying our upgrader tracing/bin/valueset2html (located in Catapult). Hopefully Ben can chime in with some idea about the root cause, but otherwise maybe you can just run your job again, which should use Catapult at ToT, which doesn't seem to show this issue.

To the point about the column width, I can't tell whether you're making a feature request or asking whether fixing this in devtools would be a workaround; to the latter, I'd say yes, but to the former, I think this is just a case of "can't please everybody" -- the columns were full-length before but that made folks have to scroll horizontally.

If this helps, I'm at chromium, refs/heads/master@{#430278}, and calling tools/perf/run_benchmark <args>.

third_party/capapult is at
commit 8ac279fd8db3eb9eea84ea14816320f41b91b186
Author: hjd <hjd@chromium.org>
Date:   Mon Nov 7 04:39:22 2016 -0800

    [tracing] Fix typo in layer tree view error
    
    Review-Url: https://codereview.chromium.org/2378773005

Fix: https://codereview.chromium.org/2481373003

Sorry bout that!

thanks!

Filed go/catabug/3035 about improving documentation.

Bo, I think your other concerns have been addressed, otherwise let us know/file new bugs.