Dirk: how can we communicate to typ the fact that the current test is expected to fail? Thanks. We can add an abstraction to serially_executed_browser_test_case to communicate this information.

I think maybe I'm missing the bigger picture here. Why do you have tests that are expected to fail? Is there some way to rewrite them so that they aren't failing?

One usage example is that a test might be passing on some platforms, but fail on others, where a feature is not implemented yet.

Correct. The layout tests from which https://www.chromium.org/developers/the-json-test-results-format came had the possibility of expected failures, too. Is there a way to communicate this to typ so that we can more automatically discover tests which were previously failing and are now passing?

There is no mechanism for having expected failures in typ at the moment.

telemetry-based tests use unittests' mechanisms (more or less) to annotate individual test cases to skip tests if they are expected to not work and there's no point in running them, so that works fine for the "not on this platform yet" use case.

layout tests would discover the tests, skip the ones it is expected to skip, run the rest, and process the results. However, in this case, we want to capture and display the actual results.

It sounds like in this case we actually want to run the test but discard the results, and that's a bit more unusual, and actually kinda unrelated to whether a test passes or fails (though typ will not print the captured data for tests that pass by default).

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

this should still be marked available.

I'm going to mark this as blocked on  bug 835690 , which is where I'm adding support for test expectations to TYP itself. Once that's implemented, we can then use this bug to add the needed support to the gpu test frameworks.

Discussed offline, Rakib will take ownership of this bug

 Issue 916434  has been merged into this issue.