All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=918601

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=8f95bccfeedf2e1ade43c1536bf0cbb4765602c37dddb257974eda21200d49ae


Bot(s) for this bug's original alert(s):

Win 7 Perf

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14b7eac4940000

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/14b7eac4940000

[Reland #4] Use the SequenceManager in ScopedTaskEnvironment by alexclarke@chromium.org
https://chromium.googlesource.com/chromium/src/+/2ded172c0daa913d3d6e0b118d86ed38210cf06f
clockless_playback: 9498 → 7092 (-2405)

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Currently this regression is affecting tests only rather than production code, although we are planning to use this in production so we need to get to the bottom of this.

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12c77382540000

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/12c77382540000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12d83d48540000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14b77ba2540000

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/12d83d48540000

The Performance Dashboard labels the axis as "unitless_smallerIsBetter (lower is better)". The value went down, so if anything the patch made things better.

The pin pointed patch (https://crrev.com/c/1371899) changes the task posting infrastructure, and we were expecting this to have influence on benchmarks sensitive to task posting. To be honest we were not expecting such a big improvement though. 

It troubles me a bit that doing less runs per second is a good thing. I would have expected that to be bad. I am going to loop in the benchmark owners in case I am missing something.

Looks like the test is measuring runs/s [1], so higher is presumably better and the benchmark unit has been mislabeled somehow.

[1] https://cs.chromium.org/chromium/src/media/test/pipeline_integration_perftest.cc?rcl=0e6e4575bef92f075c44933afd78117487ef0f46&l=41

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/13b45022540000

Yes higher is better, it's probably not a huge deal for this test though. Clockless playback is driven by 0-delay repeated PostTask, so a change here seems expected given the CL in question and isn't very relevant to what this test is trying to measure (media pipeline performance).

So if you expected back-to-back PostTasks to be impacted you can just close this as WontFix. We should probably fix the units though, wherever that's coming from??

+Ben, +Brian,

How can we could make this result largerIsBetter rather than smallerIsBetter?

I think this could be done by doing the following:

1. Update the conversion script to not automatically append _smallerIsBetter if the unit already ends with _smallerIsBetter or _biggerIsBetter, and instead append whatever the existing suffix is https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/value/gtest_json_converter.py?l=97

2. Update your metric name to be clockless_playback_biggerIsBetter, which should then be converted to unitless_biggerIsBetter.

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/13b45022540000

Brian, instead of requiring cc perf tests like media_perftests to change their metric names, could you translate legacy unit info to python and reference that in gtest_json_converter.py?
https://github.com/catapult-project/catapult/blob/master/tracing/tracing/value/legacy_unit_info.html
Related discussion: https://chromium-review.googlesource.com/c/catapult/+/1422677

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14b77ba2540000

The swarming task expired. The bots are likely overloaded, dead, or misconfigured.

Using the legacy unit info should work, although we'll have to see what we decide on doing in the meeting today since we'll likely have to change stuff in tests to fix the data formatting issue, anyways.