James, would you take a look?

The flake has not reoccurred since. I don't know if this is fixed or just rare, but I'm removing the sheriff label for now.

My CL renamed this test and moved it to the browser_tests suite - that's why it appears like new flake.

It continues to flake at a low level:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=ChromeVoxOutputE2ETest.Audio

However, various OutputE2ETest tests were flaky even when part of chromevox_tests:
https://findit-for-me.appspot.com/waterfall/list-flakes?step_name=chromevox_tests

I can't repro the flake locally. I don't see any obvious problems in the test code. I'm going to disable the test.

I also wonder if this could be related to recent media session work - the test exercises an <audio> tag.

Hi James,

To add on the history here,

https://chromium-review.googlesource.com/1225984

resolved major flakes in the ChromeVox test suite.

That change landed on 9/15. Thereafter, the number of flakes actually was pretty much 0. I do think the Output audio test is most likely a result of moving these tests to browser tests, having no other reason why the test fails.

steimel, is it possible media control changes in https://chromium-review.googlesource.com/c/chromium/src/+/1268663 are related to this flake? I see a mention of layout test flake in the CL description.

Notes from my CL to disable the test (https://chromium-review.googlesource.com/c/chromium/src/+/1324389):

My CL moving the test to browser_tests was blamed for this flake. I don't think the runtime environment is significantly different in browser_tests. I don't see anything obviously wrong in the test, other than this TODO:

      // TODO(dmazzoni/dtseng): Replace with a query.
      el = el.nextSibling.nextSibling.nextSibling;

The failure is this:

TypeError: Cannot read property 'role' of null
    at cursors.WrappingCursor.cursors.Cursor (chrome-extension://mndnfokpggljbaajbnioimlmbfngpief/chromeVox2ChromeBackgroundScript.js:211:792)
    at new cursors.WrappingCursor (chrome-extension://mndnfokpggljbaajbnioimlmbfngpief/chromeVox2ChromeBackgroundScript.js:248:127)
    at Function.cursors.WrappingCursor.fromNode (chrome-extension://mndnfokpggljbaajbnioimlmbfngpief/chromeVox2ChromeBackgroundScript.js:248:205)
    at Function.cursors.Range.fromNode (chrome-extension://mndnfokpggljbaajbnioimlmbfngpief/chromeVox2ChromeBackgroundScript.js:258:204)
    at ChromeVoxOutputE2ETest.<anonymous> (file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:9041:33)
    at CallbackHelper.<anonymous> (file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:8555:16)
    at CallFunctionAction.invoke (file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:7933:22)
    at RunAllAction.invoke (file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:8094:35)
    at file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:8565:14
    at Object.listener [as callback] (file:///b/s/w/ir/out/Release/test_data/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs:8769:21)
I see that the line of code in the test that uses cursors reference an element:

     range = cursors.Range.fromNode(el);

So maybe that TODO is related? I don't know what a "query" means in that context.

There have been some changes in media focus recently, but I have no idea if they impact the <audio> tag.

In particular, I see calls to a method enableTestMode() in the LayoutTests. The underlying method has a comment saying it disables animations. Could those animations affect accessibility tests?

https://cs.chromium.org/chromium/src/third_party/blink/renderer/core/html/media/media_controls.h?rcl=76401fe87d5c5ec4b752787994bd4475c07862dc&l=72

Seems like this really only occurs on viz_browser_tests?

Seems happening a few times with single_process_mash_browser_tests;
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-rel/17825
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-rel/17800
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-rel/17794

As seeing the flakiness dashboard, this seems not specific to a single configuration, rather equally flaky on browser_tests, single_process_mash_browser_tests, and viz_browser_tests. But only flaky on linux-chromeos-rel bot.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/799745e842310619e7b13c8260019d318dbf095d

commit 799745e842310619e7b13c8260019d318dbf095d
Author: Scott Violet <sky@chromium.org>
Date: Wed Jan 09 00:01:24 2019

browser_tests: disables ChromeVoxOutputE2ETest.Audio

as it is flaky

BUG=901725
TEST=test only change

Change-Id: I1c80611a93924b21dfdd4f518e24f597838e7747
Reviewed-on: https://chromium-review.googlesource.com/c/1401118
Reviewed-by: David Tseng <dtseng@chromium.org>
Commit-Queue: Scott Violet <sky@chromium.org>
Cr-Commit-Position: refs/heads/master@{#620952}
[modify] https://crrev.com/799745e842310619e7b13c8260019d318dbf095d/chrome/browser/resources/chromeos/chromevox/cvox2/background/output_test.extjs