A large portion of this flakiness is from:

02-06 00:42:22.449 11931 11945 F chromium: [11931:11945:0206/004222.451807:FATAL:layouttest_support.cc(297)] Check failed: false. did not swap for reason 0

jonross@ you were last in this code. Is this something you can investigate?

That message means that the compositor swap itself failed. There's not enough details with that message alone to know what the source of the error is.

If there are a few tests which flake more often it would be best to debug where SwapPromise::SWAP_FAILS is being set during the failures. Then we could triage the error source better.

Could someone with knowledge of the failing tests take an initial look?

Starting at https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_unit_tests&builder=tryserver.chromium.android%3Aandroid_blink_rel, clicking any gray box (hovering says CRASH PASS) the Build Log is all green and the Failure log is a 404.

https://logs.chromium.org/v/?s=chromium%2Fbb%2Ftryserver.chromium.android%2Fandroid_blink_rel%2F5323%2F%2B%2Frecipes%2Fsteps%2Fwebkit_tests__without_patch_%2F0%2Fstdout never finishes loading for me, but in devtools there's still network activity every few seconds even after 5 minutes, and the page says "Loading streams (has the build crashed?)"

Does this manifest as failed tryjobs on chromium-review.googlesource.com, as failed builds on build.chromium.org, or where does the failure show up?

This is marked as a P1, but with nobody assigned. dpranke@, do you know who has experience with tryserver.chromium.android?

+jbudorick, +martiniss for starters.

I don't know why we're getting the 404 for the failure log; it might be an android-specific thing?

I will note that your second link is for webkit_tests, not webkit_unit_tests; going to a build log and then clicking on stdout for webkit_unit_tests produced a valid file for me.

#0: I'm curious how you stumbled on this. Perhaps the dashboard looked very different a few weeks back?

#4, re the gray boxes: the failure modes you see on the flakiness dashboard do not manifest as failed builds or steps anywhere (w/ the exception of the current CRASH/CRASH/CRASH build, which would be a failure). They're retried and pass on the retry. As such, I don't think this should be a P1 at all.

#4, re 5323: that's webkit_tests, not webkit_unit_tests; is that in the scope of this bug? (No idea what's up w/ logdog there.)

#4/#5, re the 404: we don't post build logs for tests that pass on retry (i.e., that the build overall thinks succeeded). You'd see a 404 for the failure log on any gray box, android or not.

> #4/#5, re the 404: we don't post build logs for tests that pass on retry
> (i.e., that the build overall thinks succeeded). You'd see a 404 for the
> failure log on any gray box, android or not.

Good point, that's probably a bug we should fix in test-results.

re c6: I'm currently making a dashboard to look at flaky test results in chrome_infra.test_results and found it while looking for flaky tests on the trybots.  I'm going to split this into multiple bugs since there are quite a few tests that are flaky, for example:

re c5: yes I used the wrong link, oops sorry!

02-21 16:22:43.129  4215  4215 E chromium: [ERROR:weburl_loader_mock.cc(155)] Not implemented reached in virtual void blink::WebURLLoaderMock::SetDefersLoading(bool)


I   95.040s run_tests_on_device(0bcafaf4032fe6aa)  [ RUN      ] WebDocumentFirstPartyTest.NestedOriginAInOriginBWithFirstPartyOverride
I   95.040s run_tests_on_device(0bcafaf4032fe6aa)  [FATAL:SchemeRegistry.cpp(105)] Check failed: WTF::IsBeforeThreadCreated(). 



I   47.159s run_tests_on_device(0bcb0b0f032fe5aa)  [ RUN      ] WebEmbeddedWorkerImplTest.PauseAfterDownload
I   47.159s run_tests_on_device(0bcb0b0f032fe5aa)  [FATAL:v8_remote_playback_availability_callback.cc(34)] Check failed: !CallbackFunction().IsEmpty().


I   67.891s run_tests_on_device(0bcd20ea0c371254)  [ RUN      ] StyleEngineTest.TextToSheetCache
I   67.891s run_tests_on_device(0bcd20ea0c371254)  [FATAL:GCInfo.cpp(74)] Check failed: HeapObjectHeader::FromPayload(payload)->GcInfoIndex() == gc_info_index (6732 vs. 65)


I   64.949s run_tests_on_device(06b6ef690062085b)  [ RUN      ] FontResourceTest.ResourceFetcherRevalidateDeferedResourceFromTwoInitiators
I   64.949s run_tests_on_device(06b6ef690062085b)  ../../third_party/WebKit/Source/core/loader/FrameFetchContextTest.cpp:1166: Failure
I   64.949s run_tests_on_device(06b6ef690062085b)  Actual function call count doesn't match EXPECT_CALL(*client, DidDisplayContentWithCertificateErrors())...
I   64.949s run_tests_on_device(06b6ef690062085b)           Expected: to be called once
I   64.949s run_tests_on_device(06b6ef690062085b)             Actual: never called - unsatisfied and active
I   64.949s run_tests_on_device(06b6ef690062085b)  [ CRASHED      ]

Filed https://bugs.chromium.org/p/chromium/issues/detail?id=814557
 for one of the crashes, blink::PaintPropertyTreeBuilderTest_CachedProperties_Test::TestBody()

Filed https://bugs.chromium.org/p/chromium/issues/detail?id=815330 for Check failed: WTF::IsBeforeThreadCreated() failure.  I suspect it's the same root cause for these flakes due to when they started and the same pattern of flakes for many of the tests.  Leaving this bug open to track the flakes in this suite.

Marking this as available re the tracking. Assigning @aluo as the owner for now.

Ping from the ecosystem-infra sheriff. Is this still a P2? Perhaps someone would also be willing to look at bug 824539?

Ping aluo@, are you working on this?

nednguyen@, perhaps you have context on this as well? Or who has been working on Android lately?