foolip@, please triage

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f0db89b5b96c96af7b2ae2975c25eaba712ed791

commit f0db89b5b96c96af7b2ae2975c25eaba712ed791
Author: Vasilii Sukhanov <vasilii@chromium.org>
Date: Mon Jul 09 09:20:55 2018

Mark external/wpt/css/mediaqueries/device-aspect-ratio-003.html as flaky

TBR=foolip@chromium.org

Bug: 861682
Change-Id: I7985c8c2fe6f48be237d8a9cbab398efbb0416b1
Reviewed-on: https://chromium-review.googlesource.com/1128751
Reviewed-by: Vasilii Sukhanov <vasilii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#573252}
[modify] https://crrev.com/f0db89b5b96c96af7b2ae2975c25eaba712ed791/third_party/WebKit/LayoutTests/TestExpectations

I've taken a look, the test hasn't changed since it was imported in //chromium-review.googlesource.com/813970 and the flakiness seems to Win7, and started recently. My first suspicion is therefore a Chromium change at or before commit bf7893388e2900e0a7bd0feb0139e55f99e3da45. But it's too infrequent to narrow down the suspect range to anything small, so without a tool like FindIt I don't know how to proceed.

chanli@, robertma@, how's the LayoutTests support for FindIt coming along? Is it possible to run a case like this through it to see if it identifies a cause?

Findit already starts to find culprits for some webkit_layout_tests flakiness. I will run an analysis for this particular test and update this bug later.

chanli@, do you have results from the analysis yet?

foolip@, sorry for not updating the bug. There was a bug in Findit that blocked manually triggering analyses. I just landed a fix for it and deployed a new version yesterday. Hopefully I should be able to manually trigger the analysis and get the results later.

foolip@, I got the analysis result back (https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVyvQELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKGAWNocm9taXVtLndpbi9XaW43IFRlc3RzICgxKS84MTYwMy93ZWJraXRfbGF5b3V0X3Rlc3RzL1pYaDBaWEp1WVd3dmQzQjBMMk56Y3k5dFpXUnBZWEYxWlhKcFpYTXZaR1YyYVdObExXRnpjR1ZqZEMxeVlYUnBieTB3TURNdWFIUnRiQT09DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw) but unfortunately when we rerun this particular test, all runs pass so Findit decides this flake in unreproducible. 

Is it possible that this test only flakes when it runs with other tests?

That is certainly possible, but this isn't the type of test where I'd suspect it since there's no global state involved. Was the analysis done using Win7?

Based on the analysis, the analysis was indeed run on Win7. Is there any other builder/platform I can try to run the analysis on? And it's confirmed the test is still flaky?

In https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_layout_tests&tests=device-aspect-ratio-003.html the last flake was https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win7%20Tests%20%28dbg%29%281%29/70856 which was only 2 days ago.

Although, clicking through, I can't actually see it failing, not even under "Flaky" or "Unexpected flaky" in https://test-results.appspot.com/data/layout_results/Win7_Tests__dbg__1_/70856/layout-test-results/results.html.

Would that be a bug in the flakiness dashboard, or how can that happen?