Backtrace:
	content::WebContentsObserverSanityChecker::DidStartLoading [0x06C5A686+38] (C:\b\c\b\win_asan\src\content\test\web_contents_observer_sanity_checker.cc:306)
	content::WebContentsImpl::LoadingStateChanged [0x02ED7C33+601] (C:\b\c\b\win_asan\src\content\browser\web_contents\web_contents_impl.cc:4957)
	content::WebContentsImpl::DidStartLoading [0x02EDD609+41] (C:\b\c\b\win_asan\src\content\browser\web_contents\web_contents_impl.cc:5587)

Looking at the win64-clang builder the first related failure is in 
https://uberchromegw.corp.google.com/i/official.desktop/builders/win64-clang/builds/1731 (Fri Nov 30 20:23:15 2018)
every "browser_tests" failure since then has contained the CrSettingsChromeCleanupPageTest.All failure.

However, https://uberchromegw.corp.google.com/i/official.desktop/builders/win64-clang/builds/1747 has succeeded.


For the win-asan builder the first related failure is in
https://uberchromegw.corp.google.com/i/official.desktop/builders/win-asan/builds/1181 (Sat Dec 1 20:39:45 2018)

This *might* be related to https://chromium-review.googlesource.com/c/chromium/src/+/1352822. Hard to tell since the flakiness dashboard doesn't seem to work for Chrome-official buildbots. Re-assigning to joenotcharles who wrote that CL. Please re-assign back to me if https://chromium-review.googlesource.com/c/chromium/src/+/1352822 isn't responsible.

I can reproduce this on my desktop. There seems to be some start being kept between test runs because with --gtest_repeat=2 the first test always succeeds and the second test always fails.

This does seem to have been triggered by https://crrev.com/c/1352822. Replacing the <embed> from that CL with <span> stops the failure on my desktop. But I think the cause is the resource leak in the test harness, not the patch itself.

Actually seems to be timing related. When I add some debug logs it goes down to about 1 in 3 runs fail. Must just be coincidence that it was always repeat #2 that failed.

Added some debug output:

[36672:5736:1206/162429.140:ERROR:web_contents_impl.cc(5625)] DidStartLoading 0 frame_name '' unique_name 'chrome://settings/partner-logo.svg'
[36672:5736:1206/162429.140:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 1 different_document 0 due_to_interstital 0 url NULL
[36672:5736:1206/162429.142:ERROR:web_contents_impl.cc(5664)] DidStopLoading
[36672:5736:1206/162429.143:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 0 different_document 1 due_to_interstital 0 url chrome://settings/
[36672:5736:1206/162429.205:ERROR:web_contents_impl.cc(5625)] DidStartLoading 0 frame_name '' unique_name 'chrome://settings/partner-logo.svg'
[36672:5736:1206/162429.206:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 1 different_document 0 due_to_interstital 0 url NULL
[36672:5736:1206/162429.210:ERROR:web_contents_impl.cc(5664)] DidStopLoading
[36672:5736:1206/162429.221:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 0 different_document 1 due_to_interstital 0 url chrome://settings/
[36672:5736:1206/162429.267:ERROR:web_contents_impl.cc(5625)] DidStartLoading 0 frame_name '' unique_name 'chrome://settings/partner-logo.svg'
[36672:5736:1206/162429.267:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 1 different_document 0 due_to_interstital 0 url NULL
[36672:5736:1206/162429.331:ERROR:web_contents_impl.cc(5625)] DidStartLoading 0 frame_name '' unique_name 'chrome://settings/partner-logo.svg'
[36672:5736:1206/162429.331:ERROR:web_contents_impl.cc(4993)] LoadingStateChanged start 1 different_document 0 due_to_interstital 0 url NULL
[36672:5736:1206/162429.331:FATAL:web_contents_observer_sanity_checker.cc(307)] Check failed: !is_loading_.

There's a random number of valid DidStart / DidStop pairs before it dies, so it looks like several sequences are calling DidStart and DidStop unsynchronized and they occasionally collide. Looks like a real race condition in the loader?

Nope, these are all being called from the same sequence. So whatever is messing up the order of DidStart and DidStop is at the other end of the mojo pipe.

Found a fix in the test suite - it was lacking a teardown() function that removes the current page, so every test case was adding more and more copies of the same child to the DOM.

I'd argue that there's still an underlying loader bug since the missing teardown() shouldn't cause the loader to get in an invalid state. But the teardown is clearly the right thing to do so I'll add it and follow up with the loader team to see if they think it's worth investigating further.

In the setup() method there is a PolymerTest.clearBody() call which should run before each test and empty the DOM anyway, so not 100% sure if the lack of teardown() is the issue.

Uploaded https://crrev.com/c/1367924.

I've confirmed it fixes the issue on my desktop (without the fix the test will fail about 50% of the time, with the fix it's run over 10 repeats without a crash). I'm following the pattern of about_page_tests.js.

I don't know enough about the JS infrastructure to get any further on this. If you don't think this is the correct fix (or if you want to dig into it further to find out why clearBody isn't enough) I'll have to throw this over to you.

Today https://crrev.com/c/1367924 ran for 4 repeats and then crashed again, so I didn't submit it.

If the investigation takes a while, can we revert https://chromium-review.googlesource.com/c/1352822 in the meantime? Having known-broken tests in the tree causes needless duplicate work.

Just to update:

Still seeing the same issue (CrSettingsChromeCleanupPageTest) on official of 73.0.3636.0/73.0.3636.1 & continuous builder as below.

Link to the builder:
------------------
https://uberchromegw.corp.google.com/i/official.desktop/builders/win-asan/builds/1189
https://uberchromegw.corp.google.com/i/official.desktop/builders/win-clang/builds/1764

https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/win64%20beta/builds/5793
https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/win%20trunk/builds/98894
https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/win64%20trunk/builds/47506


Link to error log:
-----------------
https://logs.chromium.org/logs/chrome/bb/official.desktop/win-asan/1189/+/recipes/steps/browser_tests/0/stdout
https://logs.chromium.org/logs/chrome/bb/official.desktop/win-clang/1764/+/recipes/steps/browser_tests/0/stdout

https://logs.chromium.org/logs/chrome/bb/official.desktop.continuous/win64_beta/5793/+/recipes/steps/browser_tests/0/stdout
https://logs.chromium.org/logs/chrome/bb/official.desktop.continuous/win_trunk/98894/+/recipes/steps/browser_tests/0/stdout
https://logs.chromium.org/logs/chrome/bb/official.desktop.continuous/win64_trunk/47506/+/recipes/steps/browser_tests/0/stdout

Thanks..!

@10: The revert is in the cq.

Revert committed as https://crrev.com/c/1370201

Just to update:

Same issue(CrSettingsChromeCleanupPageTest) still seen on Win 64 beta of official continuous builder.

Link to the builder:
-------------------
https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/win64%20beta/builds/5826

Link to error log:
-------------------
https://logs.chromium.org/logs/chrome/bb/official.desktop.continuous/win64_beta/5826/+/recipes/steps/browser_tests/0/stdout

Thanks..!

Thanks for the update. Will need to merge the fix to M72 after it's been in Canary for 24 hours.

Just to update:

Same issue(CrSettingsChromeCleanupPageTest) still seen on Win 64 beta of official continuous builder.

Link to the builder:
-------------------
https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/win64%20beta/builds/5884

Link to error log:
-------------------
https://logs.chromium.org/logs/chrome/bb/official.desktop.continuous/win64_beta/5884/+/recipes/steps/browser_tests/0/stdout

joenotcharles@, Please take a look into it.
Thanks..!

This

jmukthavaram@: this is waiting on TPM approval to merge the fix in https://crbug.com/767130 to M72.

Just to update:

Also seeing the same issue on Win Clang official builder.

Link to the builder:
--------------------
https://uberchromegw.corp.google.com/i/official.desktop/builders/win-clang/builds/1792

Link to error log:
------------------
https://logs.chromium.org/logs/chrome/bb/official.desktop/win-clang/1792/+/recipes/steps/browser_tests/0/stdout

Thanks!

That's building branch 3626, which is M72.

Fix has been merged to M72.