IIRC, the flakiness dashboard seem to support taking a json for test runtime in the form of:
{"a": {
    "b": [100, 200],
    "c": ..
  }
  "d": [...]
}

Should typ's test runtime output follow this so we can also have test timing info uploaded the flakiness server?

I think you're referring to the times_ms.json file. I don't think we should recreate this file in typ, and should either add the times to the json_results format or have you get the info from the trace file. 

I'm fine w/ kbr@'s leaning that we should add it to the json_results format. At some point I'd like to switch us all over to an augmented version of the trace file format, but now is not the time for that.

Supporting timing information in full_results.json instead of in times_ms.json sounds fine to me. I wasn't around when the decision was made to use separate files so I'm not sure if I'm missing a reason why we aren't already doing that.

Who can take this? I would be happy to add it to typ but know nothing about the harness. I'm guessing this is half an hour of work, tops. Could we please prioritize it? I'll handle the extraction of these run times from the JSON file once they're present.

It is an easy thing to implement, but I have a lot on my plate at the moment. I'll do what I can.

If you're too busy, if you could point me to the appropriate code in typ I will be happy to implement it.

I think I can do this quickly enough and it'll just delay some other stuff for the telemetry team, which sullivan said she was okay with ...

Where will the timing data be used? If in test-results the server will need to be modified to read the new fields, too.

I'll only be using them in the extraction script https://cs.chromium.org/chromium/src/content/test/gpu/gather_swarming_json_results.py .

Ok, sgtm.

To #9, if the test-results server can be modified to read the timing data, it would be great too. But that is not urgent.

Posted a fix to https://codereview.chromium.org/2693503003/ and updated the file format spec at https://www.chromium.org/developers/the-json-test-results-format .

Once that change is approved I can roll that (and another change) into chromium.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/83ab204d13a2058652855c18fd5ac5467853c105

commit 83ab204d13a2058652855c18fd5ac5467853c105
Author: dpranke <dpranke@chromium.org>
Date: Sat Feb 11 03:28:44 2017

Roll typ v0.9.9 -> v0.9.11 (5822dda0..f6afa2bb).

This primarily adds support for test times in the json test results
files.

  f6afa2b bump version to 0.9.11
  afcd16c fix upload to still use http servers
  1197d56 checkpoint
  96da555 add test times to the JSON results

TBR=kbr@chromium.org
BUG= 689788 

Review-Url: https://codereview.chromium.org/2690743002
Cr-Commit-Position: refs/heads/master@{#449838}

[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/README.chromium
[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/typ/fakes/tests/test_result_server_fake_test.py
[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/typ/json_results.py
[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/typ/tests/json_results_test.py
[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/typ/tests/main_test.py
[modify] https://crrev.com/83ab204d13a2058652855c18fd5ac5467853c105/third_party/typ/typ/version.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e86fbe70bbe580990957eca6a00531bd421358c9

commit e86fbe70bbe580990957eca6a00531bd421358c9
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Sun Feb 19 03:45:52 2017

Roll src/third_party/catapult/ 36a508280..84a7af661 (27 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/36a50828018d..84a7af661032

$ git log 36a508280..84a7af661 --date=short --no-merges --format='%ad %ae %s'
2017-02-17 dtu Use third-party libraries in Catapult instead of the App Engine SDK.
2017-02-17 jessimb Adjusting css to make group by counter work for 3 numbers.
2017-02-17 jessimb Allow a user to enter a milestone param to view the table.
2017-02-17 jessimb Added new index for Anomaly.
2017-02-17 nednguyen Revert of [Telemetry] Migrate browser_test_runner to use typ as the test runner (patchset #1 id:1 of https://codereview.chromium.org/2700563004/ )
2017-02-17 sullivan First version of benchmark health report.
2017-02-17 benjhayden Remove storyRepeatCounter.
2017-02-17 benjhayden Merge RelatedHistogram diagnostics.
2017-02-17 mikecase Fix for improper call to ps on Android versions greater than N.
2017-02-17 ulan Make estimatedInputLatency metric more friendly for traces without TTI.
2017-02-17 erikchen Use absolute pcs and load locations to symbolize on macOS.
2017-02-17 erikchen Fix a bug introduced in a refactor to the ELFSymbolizer.
2017-02-17 nednguyen [Telemetry] Remove --page-repeat flag
2017-02-17 hjd [dashboard] Avoid double XHR in /report
2017-02-16 dtu [pinpoint] Add Job.job_id property.
2017-02-16 nednguyen Roll typ v0.9.5 -> v0.9.11 (a277897604718c..f6afa2bbd1765b21e8)
2017-02-16 eakuefner [Dashboard] Only validate row_id for first row in add_point
2017-02-16 eakuefner [Dashboard] Use list_tests.GetTestsForTestPathDict in /graph_json
2017-02-16 eakuefner [Dashboard] Add test for listing tests with empty selection in test_path_dict
2017-02-16 benjhayden Fix merging Diagnostics.
2017-02-16 benjhayden Add CollectedRelatedEventSet diagnostic.
2017-02-16 eakuefner [Dashboard] Add capability for querying unselected tests by test_path_dict
2017-02-16 nednguyen [Telemetry] Migrate browser_test_runner to use typ as the test runner
2017-02-16 eakuefner [Dashboard] Add 'test_path_dict' request type to /list_tests
2017-02-16 benjhayden Replace values with histograms in metric_map_function.
2017-02-16 rnephew [WPR] Add log message when downloading archives.
2017-02-16 alexandermont Separate "time interval" from "metric collection" logic in power_metrics.html.

Created with:
  roll-dep src/third_party/catapult
BUG= 636153 ,625701, 677302 , 677302 , 689788 , 636153 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2705883002
Cr-Commit-Position: refs/heads/master@{#451509}

[modify] https://crrev.com/e86fbe70bbe580990957eca6a00531bd421358c9/DEPS