I don't see any obvious recent changes in //chrome/browser/metrics that would have affected this. Disabling the test and assigning to asvitkine@ as the one with the most knowledge here based on https://codereview.chromium.org/2939943004.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4a714378954d3ce803f1b08d864d13da83f2fce7

commit 4a714378954d3ce803f1b08d864d13da83f2fce7
Author: Colin Blundell <blundell@chromium.org>
Date: Wed Feb 28 11:14:14 2018

Disable flaky ChromeVisibilityObserverBrowserTest.VisibilityTest

There are no obvious recent changes to have turned this test flaky. The
best hypothesis that I have is that something in the changes in
https://codereview.chromium.org/2939943004 was problematic, although
in that case it's unclear why it would only start turning up now.

TBR=asvitkine@chromium.org

Bug:  817172 
Change-Id: Ic0665e7d8d606b5ecb01d11f641bcfe2ec82a1ab
Reviewed-on: https://chromium-review.googlesource.com/941121
Reviewed-by: Colin Blundell <blundell@chromium.org>
Commit-Queue: Colin Blundell <blundell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539784}
[modify] https://crrev.com/4a714378954d3ce803f1b08d864d13da83f2fce7/chrome/browser/metrics/desktop_session_duration/chrome_visibility_observer_browsertest.cc

I don't believe it would be https://codereview.chromium.org/2939943004 - as it wouldn't explain why we're starting to see the problem more recently.

Gayane, could you take a look?

Here's some thoughts:
  - Maybe we should put this test in interactive_uitests since it requires for the browser to be active and if multiple browser tests are running at the same time, this could be the cause of flakyness. For interactive_uitests, I believe the test harness ensures there's no concurrent tests.
  - I also noticed BrowserActivationWaiter helper class in chrome/test/base/ui_test_utils.h. Maybe we should use that class when creating the window initially? Right now we just RunUntilIdle() on the RunLoop and assume that's sufficient. But if there's a helper class specifically for this purpose, maybe we should use it?

I have made some changes to this test here https://chromium-review.googlesource.com/c/chromium/src/+/951409

I will close this bug for now. If still flaky, feel free to re-open it.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/07f44e3e242f03f35ccbc49034c44ce2c18ef700

commit 07f44e3e242f03f35ccbc49034c44ce2c18ef700
Author: Gayane Petrosyan <gayane@chromium.org>
Date: Fri Mar 23 01:47:42 2018

Use interactive UI tests for Chrome visibility observer tests.

Use interactive UI tests for Chrome visibility observer tests and change the logic of the test:
- Now test class inherits both from InProcessBrowserTest and ChromeVisibilityObserver,
Previously observer was created after first browser creation that why observer and browser states were not in sync in the beginning of the test
- Deactivate doesn't works for mac
- For tests only, when browser is deactivated observer is notified immediately instead of delayed task.

Bug:  817172 
Change-Id: I71c0e8072210af9999e9acee7910ca435d7e1d80
Reviewed-on: https://chromium-review.googlesource.com/951409
Reviewed-by: Scott Violet <sky@chromium.org>
Commit-Queue: Gayane Petrosyan <gayane@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545335}
[modify] https://crrev.com/07f44e3e242f03f35ccbc49034c44ce2c18ef700/chrome/browser/metrics/desktop_session_duration/chrome_visibility_observer.cc
[modify] https://crrev.com/07f44e3e242f03f35ccbc49034c44ce2c18ef700/chrome/browser/metrics/desktop_session_duration/chrome_visibility_observer.h
[delete] https://crrev.com/4a520b324361dd58d564b82b1d502747f46c67dd/chrome/browser/metrics/desktop_session_duration/chrome_visibility_observer_browsertest.cc
[add] https://crrev.com/07f44e3e242f03f35ccbc49034c44ce2c18ef700/chrome/browser/metrics/desktop_session_duration/chrome_visibility_observer_interactive_uitest.cc
[modify] https://crrev.com/07f44e3e242f03f35ccbc49034c44ce2c18ef700/chrome/test/BUILD.gn