This is flaky, with an extremely high failure rate (25–50%). Two sets of tests appear to have started flaking around the same time. First failure that I can find with this set of tests:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win%207%20Tests%20x64%20%281%29/41488

failures:
All/PasswordManagerBrowserTestWithViewsFeature.PromptForPushStateWhenFormDisappears/1
All/PasswordManagerBrowserTestWithViewsFeature.ChangePwdFormPushStateBubbleShown/1
SettingsA11yManagePasswords.MANAGE_PASSWORDS_aria_required_children
SettingsA11yManagePasswords.MANAGE_PASSWORDS_frame_title_unique
CrControlledButtonTest.All
SettingsA11yManagePasswords.MANAGE_PASSWORDS_valid_lang
CrSettingsSiteListTest.EditExceptionDialog
BrowsingDataRemoverBrowserTest.HistoryDeletion
CrExtensionsNavigationHelperTest.SupportedRoutes
CrSettingsCategoryDefaultSettingTest.All
MaterialHistoryRoutingTest.All
All/PasswordManagerBrowserTestWithViewsFeature.PromptForPushStateWhenFormDisappears_ParametersInOrigins/1
WebUIImplBrowserTest.SameDocumentNavigationsAndReload
SecurityStateTabHelperTest.ConsoleMessageNotPrintedForPushStateNavigation/1
SettingsA11yManagePasswords.MANAGE_PASSWORDS_link_name

You can see recurring tests SettingsA11yManagePasswords.MANAGE_PASSWORDS* and Cr*.All (some mocha tests). I believe these are all timeouts.

This started about 6 days ago. No obvious cause. The same tests (or tests matching those patterns) are still failing today.

Oh, the common thing between SettingsA11yManagePasswords.MANAGE_PASSWORDS* and Cr*.All are that they are Mocha tests.

I have ... no idea how these tests are set up (seem to be generated), or how to disable them.

For SettingsA11yManagePasswords, it looks like hcarmona has already disabled a bunch of these for flake in r502632 (which was almost a year ago).

chrome/test/data/webui/settings/a11y/passwords_a11y_test.js

It looks like those disabled are the suffixes to MANAGE_PASSWORDS_* but there are lots more failing.

here are the suffixes to MANAGE_PASSWORDS_* that I'm seeing:

aria_allowed_attr
aria_required_children
aria_valid_attr
blink
document_title
frame_title_unique
heading_order
html_has_lang
html_lang_valid
link_name
object_alt
radiogroup
region
table_fake_caption
th_has_data_cells
valid_lang

I stopped after investigating four failures... every single one of these is unique. I'm going to disable the whole test.

Haven't looked into Cr*.All tests which also look like Mocha tests.

I can't figure out how to disable this entire test. Adding DISABLED_ to the front of SettingsA11yManagePasswords, as in:


--- a/chrome/test/data/webui/settings/a11y/passwords_a11y_test.js
+++ b/chrome/test/data/webui/settings/a11y/passwords_a11y_test.js
@@ -25,7 +25,8 @@ SettingsA11yManagePasswords.prototype = {
   ]),
 };
 
-AccessibilityTest.define('SettingsA11yManagePasswords', {
+// Disabled for flaky timeouts.  https://crbug.com/872575 
+AccessibilityTest.define('DISABLED_SettingsA11yManagePasswords', {
   /** @override */
   name: 'MANAGE_PASSWORDS',
   /** @type {PasswordManager} */

results in this compile failure:


FAILED: test_data/chrome/test/data/webui/settings/a11y/passwords_a11y_test.js gen/chrome/test/data/webui/settings/a11y/passwords_a11y_test-gen.cc 
python ../../tools/gypv8sh.py ./v8_shell ../../chrome/third_party/mock4js/mock4js.js ../../chrome/test/data/webui/test_api.js ../../chrome/test/base/js2gtest.js webui --external=y ../../chrome/test/data/webui/settings/a11y/passwords_a11y_test.js chrome/test/data/webui/settings/a11y/passwords_a11y_test.js gen/chrome/test/data/webui/settings/a11y/passwords_a11y_test-gen.cc test_data/chrome/test/data/webui/settings/a11y/passwords_a11y_test.js
../../chrome/test/base/js2gtest.js:386: TypeError: Cannot read property 'prototype' of undefined
  var browsePreload = this[testFixture].prototype.browsePreload;
                                        ^
TypeError: Cannot read property 'prototype' of undefined
    at TEST_F (../../chrome/test/base/js2gtest.js:386:41)
    at rules.forEach (../../chrome/test/data/webui/settings/a11y/../../../../../../chrome/test/data/webui/a11y/accessibility_test.js:138:5)
    at Array.forEach (<anonymous>)
    at Object.AccessibilityTest.define (../../chrome/test/data/webui/settings/a11y/../../../../../../chrome/test/data/webui/a11y/accessibility_test.js:124:9)
    at eval (../../chrome/test/data/webui/settings/a11y/passwords_a11y_test.js:29:19)
    at ../../chrome/test/base/js2gtest.js:517:1
ninja: build stopped: subcommand failed.


I have no idea how these Mocha tests work or how to disable them (other than commenting out the test or removing the file from BUILD.gn). Hector, I see you previously disabled some of the tests. I'm assigning to you to figure out how to disable this file.

 Issue 872499  has been merged into this issue.

Not removing from sheriff queue until tests are disabled or current owner confirms active investigation

This is getting flagged by TryFlakes as well (note incoming dupes).

I'm just going to comment out the test for the time being.

 Issue 872962  has been merged into this issue.

 Issue 872963  has been merged into this issue.

 Issue 872965  has been merged into this issue.

 Issue 872966  has been merged into this issue.

 Issue 872967  has been merged into this issue.

I've spun off the non-SettingsA11yManagePasswords failures into a separate bug,  Issue 873041 .

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7e723bb3ea6853ae1dfa8d5c9a9db87d89487974

commit 7e723bb3ea6853ae1dfa8d5c9a9db87d89487974
Author: Matt Giuca <mgiuca@chromium.org>
Date: Fri Aug 10 06:04:33 2018

Disable SettingsA11yManagePasswords test for flakiness.

Bug:  872575 
Change-Id: I5d94177e46ede0b1107da0a253ed58f450c2c7ff
No-Tree-Checks: true
Tbr: mgiuca@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1170161
Reviewed-by: Matt Giuca <mgiuca@chromium.org>
Commit-Queue: Matt Giuca <mgiuca@chromium.org>
Cr-Commit-Position: refs/heads/master@{#582072}
[modify] https://crrev.com/7e723bb3ea6853ae1dfa8d5c9a9db87d89487974/chrome/test/data/webui/settings/a11y/passwords_a11y_test.js

If this is mocha, let's dupe against the mocha bug.