I'm going to be working around the logic added by kbr@ in https://codereview.chromium.org/1527123002, which is being triggered on the perf waterfall, because they manually set the gpu dimension. CL is https://chromium-review.googlesource.com/c/419708/. 

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/6bd183c35d9b955fe999e0fbf915fb0dfeb777d2

commit 6bd183c35d9b955fe999e0fbf915fb0dfeb777d2
Author: Stephen Martinis <martiniss@chromium.org>
Date: Sat Dec 17 00:07:08 2016

AD: Analyze tests even when test results fails

Changes the analyzer logic to continue to try to analyze data
even when test results has no data. This is important to
take into account the test suite detection logic.

Also adds some special casing for chromium.perf's benchmark names.

BUG= 674708 

Change-Id: I6253bf9f6a57a97ad7ff295f982d3d84eb5f2202
Reviewed-on: https://chromium-review.googlesource.com/421048
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/6bd183c35d9b955fe999e0fbf915fb0dfeb777d2/go/src/infra/monitoring/analyzer/analyzer.go
[modify] https://crrev.com/6bd183c35d9b955fe999e0fbf915fb0dfeb777d2/go/src/infra/monitoring/analyzer/step/test_step.go
[modify] https://crrev.com/6bd183c35d9b955fe999e0fbf915fb0dfeb777d2/go/src/infra/monitoring/analyzer/step/test_step_test.go

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/81ff0a4cbfede0c84a1d13d3d106757bf5186022

commit 81ff0a4cbfede0c84a1d13d3d106757bf5186022
Author: Henrik Kjellander <kjellander@chromium.org>
Date: Sun Dec 18 19:52:31 2016

Revert "AD: Analyze tests even when test results fails"

This reverts commit 6bd183c35d9b955fe999e0fbf915fb0dfeb777d2.

Reason for revert: Most likely made "infra go tests" flaky.
BUG= 675472 

Original change's description:
> AD: Analyze tests even when test results fails
> 
> Changes the analyzer logic to continue to try to analyze data
> even when test results has no data. This is important to
> take into account the test suite detection logic.
> 
> Also adds some special casing for chromium.perf's benchmark names.
> 
> BUG= 674708 
> 
> Change-Id: I6253bf9f6a57a97ad7ff295f982d3d84eb5f2202
> Reviewed-on: https://chromium-review.googlesource.com/421048
> Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
> Commit-Queue: Stephen Martinis <martiniss@chromium.org>
> 

TBR=seanmccullough@chromium.org,martiniss@chromium.org,zhangtiff@chromium.org
BUG= 674708 
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Change-Id: Ic83ad5a150eb479f14e9a874460ecd6fe2415f5c
Reviewed-on: https://chromium-review.googlesource.com/420791
Reviewed-by: Henrik Kjellander <kjellander@chromium.org>
Commit-Queue: Henrik Kjellander <kjellander@chromium.org>

[modify] https://crrev.com/81ff0a4cbfede0c84a1d13d3d106757bf5186022/go/src/infra/monitoring/analyzer/analyzer.go
[modify] https://crrev.com/81ff0a4cbfede0c84a1d13d3d106757bf5186022/go/src/infra/monitoring/analyzer/step/test_step.go
[modify] https://crrev.com/81ff0a4cbfede0c84a1d13d3d106757bf5186022/go/src/infra/monitoring/analyzer/step/test_step_test.go

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/13c0093ddaac5cc4ff9f3efde64b04220ef94e5d

commit 13c0093ddaac5cc4ff9f3efde64b04220ef94e5d
Author: Stephen Martinis <martiniss@chromium.org>
Date: Mon Dec 19 18:47:31 2016

Reland of "AD: Analyze tests even when test results fails""

This is a reland of commit 6bd183c35d9b955fe999e0fbf915fb0dfeb777d2 (
https://chromium-review.googlesource.com/c/421048/)

Includes fix for flaky test.

BUG= 674708 

Change-Id: I01b32b17e4c045a764e73a03a52386a64337e2d8
Reviewed-on: https://chromium-review.googlesource.com/422102
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/13c0093ddaac5cc4ff9f3efde64b04220ef94e5d/go/src/infra/monitoring/analyzer/analyzer.go
[modify] https://crrev.com/13c0093ddaac5cc4ff9f3efde64b04220ef94e5d/go/src/infra/monitoring/analyzer/step/test_step.go
[modify] https://crrev.com/13c0093ddaac5cc4ff9f3efde64b04220ef94e5d/go/src/infra/monitoring/analyzer/step/test_step_test.go

If this is really a Pri-1, find an owner and update the priority.

This is the result of a bulk edit that moved high priority available bugs to a lower priority in an attempt to be more honest with bug filers.

Stephen - is this work done?

No, not really. I dealt with it some by hacking around it in alerts dispatcher.

We could probably make this really happen soon. Not sure how high of a priority it is though.

It would be really great to get this working! stdio links from perf dashboard don't work until this is fixed, which makes it really hard to debug when data stops coming in.

This is a rather annoying problem we have yet to fix. I think there are rough plans for how to fix this, but nothing very concrete we can implement yet, AFAIK. cc-ing estaab because I feel like he was talking about this a bit? Also if there's another bug that we should use instead of this, it's fine to merge.

Did you see the naming doc I wrote up and sent out last week (go/chromium-build-naming)?

Ah yes, that does include build steps. I had forgotten. That should solve this problem then.

What are the next steps for this? I take it someone needs to change some scripts and/or configs to conform to go/chromium-build-naming ?

I think we need to define what "this" is. As per that spec, the step name and the "test suite" are two different things:

gn_name ("-" descriptor)* "+" swarming_configuration

Obviously, the current steps don't match that format, but I'm not sure that even if we did fix that, we would address the bug, because I'm not sure what we need to do to address the bug. Would you also need to be able to extract the gn_name (i.e., GN label / ninja target) from the total step name?

We (sheriffing tooling) wouldn't necessarily have to extract it from the step name.

Other options I believe would work for us:
- Have the test step add that formatted name to the results json.
- Write that name to stdout in some easily extractable way.
- Add it as extra step metadata visible in Milo (if there's a way to do this already)

Should we WontFix this, maybe? Bouncing back to martiniss@ to figure out what we're actually trying to do/fix here.

I'm going to mark this as WontFix becasue this is old and it doesn't seem to be super actionable. @martiniss: Please feel free to reopen/file a new bug if additional action is needed.