CL: https://codereview.chromium.org/2570553002/

Yutaka marked the test as flaky, but the flakiness did not drop. I've analyzed various scenarios in which the test has failed and it's unclear to me how to test launcher behaves. In particular, according the the results of this query (https://plx.corp.google.com/script/#a=qo%7Ci=google%253A%253Ascript_85._13f7e1_eaa0_4837_b9d0_a1a80e9124ee):

1) It's not clear what the test is expected to be doing when it's not specified in the TestExpectations file. Most of the time test launcher was expecting it to PASS, but sometimes it expected it to be SLOW or be SKIPped.
2) After the test was added to the TestExpectations, it was expected to either PASS or FAIL, but was also sometimes expected to be SLOW or SKIPped in addition to PASS or FAIL.
3) The launcher stopped retry-ing the test after it was marked as expected to PASS or FAIL even though the test has produced TEXT or TIMEOUT.

I'll try to figure out what is the logic used by the test launcher, because it seems it's more complicated than I expected. Removing additional test result types for layout tests as tracked in 654500, would make this much simpler.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1b1f643260924f93e19015e44ac61c97190e64b0

commit 1b1f643260924f93e19015e44ac61c97190e64b0
Author: allada <allada@chromium.org>
Date: Tue Jan 10 19:48:22 2017

[Devtools] Clear cache before running the test

This should help reducing the flakiness of this test, which is currently ~65%:
http://test-results.appspot.com/flakiness/dir/http/tests/inspector/network. The
text diff shows that actual number of items that are cached is different from
the expected: https://justpaste.it/11bq3.

R=allada@chromium.org
BUG= 673378 , 678486 

patch from issue 2570553002 at patchset 180001 (http://crrev.com/2570553002#ps180001)

Review-Url: https://codereview.chromium.org/2620793002
Cr-Commit-Position: refs/heads/master@{#442663}

[modify] https://crrev.com/1b1f643260924f93e19015e44ac61c97190e64b0/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/1b1f643260924f93e19015e44ac61c97190e64b0/third_party/WebKit/LayoutTests/http/tests/inspector/network-test.js
[modify] https://crrev.com/1b1f643260924f93e19015e44ac61c97190e64b0/third_party/WebKit/LayoutTests/http/tests/inspector/network/network-filters.html

Looks like last commit did not fix the test.

Trying to fix another test: DesktopWindowTreeHostX11HighDPITest. LocatedEventDispatchWithCapture. It currently has 65.15% flakiness.

CL: http://crrev.com/2633473002

After hours of unsuccessful attempts to understand code under test, I gave up and decide to find some test that was reported as flaky by chromium-try-flakes, then fixed and follow it's flakiness as reported by Flakiness Surface.

I've discovered  issue 680770  and wrote an experimental query [1] to check whether Flakiness Surface has correctly reported increasing and then decreasing flakiness over specific days: before it was flaky, when it became flake and after it was fixed. The query confirmed the hypothesis and showed that stats shown on Flakiness Surface do reflect flakiness of tests correctly.

However, one lesson that I have learned as part of this is that some very flaky tests may not actually affect developer productivity since they are only flaky when run after some other tests (share global state) and thus always pass on second retry when the other test is not run. To address this, we need to report the actual false rejections in CQ, but this is already tracked in  issue 674084 , therefore I am closing this issue as Fixed.

[1]: https://plx.corp.google.com/script/#a=cp%7Ci=google%253A%253Aconfigs_monitoring_chrome_infra_plx_chrome_infra_flaky_tests_with_layout_team_dir_info%7Cn=Copy+of+chrome_infra.flaky_tests_with_layout_team_dir_info%7Cv=draft