I'm unable to repro with --repeat-each=10 --iterations=10 

storage/domstorage/events/resources/eventTestHarness.js looks a bit sketchy -- runAfterNStorageEvents() relies on the N events coming in within 20ms which I could see failing on a slow system, but I'm not sure that would be flaky in "random order" per this report.

qyearsley@ - are you able to capture the failure, e.g. the test results diff? I can probably diagnose more from that.

Ah, via: 

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=storage%2Fdomstorage%2Fevents 


basic.html:
+ FAIL got at least 1, expected only 0 events

case-insensitive.html:
+ FAIL got at least 2, expected only 1 events

.... which is interesting - *extra* events, not missing events (which is what I'd expect from a timeout happening too early.)

basic.html and case-sensitive.html differ from the other two tests in that they use about:blank as the iframe and the listener assigned via contentWindow; the other two tests in the dir use explicit html files assign the handler themselves.

it may be best to ditch the use of about:blank and use dummy html files

Seems worth changing if just for consistency, although it would be helpful to be able to reproduce the flakiness in order to confirm whether that fixes it.

I just ran Tools/Scripts/run-webkit-tests storage/domstorage --order=random 100 times and got 0/100 failures.

Maybe it's not flaky or my workstation the way I'm running it? Or maybe I just didn't catch it in those 100 runs? Using the script in https://docs.google.com/document/d/11-628Ny6hY__kfIv__zVD0m3rA0OB9tQLLC_gJI_PNg/edit to run it more times might be able to reproduce it.

If we're unable to reproduce it then we could try removing it from RandomOrderExpectations as a trial.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2c15205a5a85afaabc36afec03d92ad069e2e7a8

commit 2c15205a5a85afaabc36afec03d92ad069e2e7a8
Author: jsbell <jsbell@chromium.org>
Date: Wed Nov 30 23:50:35 2016

Deflake storage/domstorage/events tests when run in random order

Unable to repro the flakiness locally but via code inspection it
appeared to be due a pattern used in 2 of the 4 tests (the flaky ones)
not navigating the iframe used as the source of the events between the
sessionStorage and localStorage passes to ensure a fresh context.
Changed those 2 tests to navigate the iframe before each pass.
Speculative fix, so fingers crossed.

The tests also used a strange mix of js-test helpers and direct
testRunner access leading to redundant "TEST PASSED" output. This was
simplified to follow usual js-test usage.

BUG=664848
R=jeffcarp@chromium.org

Review-Url: https://codereview.chromium.org/2538773003
Cr-Commit-Position: refs/heads/master@{#435482}

[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/RandomOrderExpectations
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/basic-body-attribute-expected.txt
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/basic-expected.txt
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/basic-setattribute-expected.txt
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/case-sensitive-expected.txt
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/resources/eventTestHarness.js
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/script-tests/basic.js
[modify] https://crrev.com/2c15205a5a85afaabc36afec03d92ad069e2e7a8/third_party/WebKit/LayoutTests/storage/domstorage/events/script-tests/case-sensitive.js

Not closing this until the bots report consistently green.

Hrm, still flaky. Will need to dig further.

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot