New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 816643 link

Starred by 3 users

Issue metadata

Status: Fixed
Merged: issue 523255
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

chromium-try-flakes files bugs about failures in tryjobs with patches applied.

Project Member Reported by chromium...@appspot.gserviceaccount.com, Feb 26 2018

Issue description

"BrowserFocusTest.ClickingMovesFocus" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLgsSBUZsYWtlIiNCcm93c2VyRm9jdXNUZXN0LkNsaWNraW5nTW92ZXNGb2N1cww.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 
Cc: isherman@chromium.org
Labels: OS-Chrome
Owner: msw@chromium.org
Status: Assigned (was: Untriaged)
Disabling the flaky test on Chrome OS in https://chromium-review.googlesource.com/c/chromium/src/+/938486.
Priority must get down to P2 when this CL lands.
Assigning to msw@ for triage.
Cc: -isherman@chromium.org
Project Member

Comment 3 by bugdroid1@chromium.org, Feb 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/da7df3358bd5750c6c3a684649a4e43232e8318c

commit da7df3358bd5750c6c3a684649a4e43232e8318c
Author: Reza.Zakerinasab <zakerinasab@chromium.org>
Date: Tue Feb 27 00:25:33 2018

Disable BrowserFocusTest.ClickingMovesFocus on Chrome OS

Bug:816643
Change-Id: I0b3f4cfaf67794e9904d8f269c69c3eaa98b9c05

TBR=msw@chromium.org

Change-Id: I0b3f4cfaf67794e9904d8f269c69c3eaa98b9c05
Reviewed-on: https://chromium-review.googlesource.com/938486
Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org>
Reviewed-by: Mohammad Reza Zakerinasab <zakerinasab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539341}
[modify] https://crrev.com/da7df3358bd5750c6c3a684649a4e43232e8318c/chrome/browser/ui/browser_focus_uitest.cc

Labels: -Pri-1 -Sheriff-Chromium Pri-2

Comment 5 by msw@chromium.org, Feb 27 2018

Components: Test
Labels: -Pri-2 Pri-1
Mergedinto: 523255
Status: Duplicate (was: Assigned)
Reza, in each case where this one test failed, all these interactive_ui_tests failed with focus/activation/input issues:

StickyKeysBrowserTest.CtrlClickHomeButton
SameSiteSubframe/DragAndDropBrowserTest.DragImageFromDisappearingFrame/0
TabDragging/DetachToBrowserTabDragControllerTest.DragWithMaskedWindows/0
TabDragging/DetachToBrowserInSeparateDisplayTabDragControllerTest.DragMaxTabToNonMaxWindowInSeparateDisplay/0
CrossSiteSubframe/DragAndDropBrowserTest.DragImageBetweenFrames/0
TabDragging/DetachToBrowserTabDragControllerTest.DragInSameWindow/0
FindInPageTest.CrashEscHandlers
TabDragging/DetachToBrowserInSeparateDisplayTabDragControllerTest.DragBrowserWindowWhenMajorityOfBoundsInSecondDisplay/0
LocationIconViewTest.HideOnSecondClick
CrossSiteSubframe/DragAndDropBrowserTest.CrossSiteDrag/0
MouseLeaveTest.ContextMenu
BrowserFocusTest.FindFocusTest
TabDragging/DetachToBrowserTabDragControllerTest.DragToSeparateWindow/0
TabDragging/DetachToBrowserTabDragControllerTest.DragToSeparateWindow/1
BrowserFocusTest.ClickingMovesFocus
TabDragging/DetachToBrowserInSeparateDisplayTabDragControllerTest.DragSingleTabToSeparateWindowInSecondDisplay/0
DetachToBrowserInSeparateDisplayAndCancelTabDragControllerTest.CancelDragTabToWindowIn1stDisplay
CrossSiteSubframe/DragAndDropBrowserTest.DragImageFromDisappearingFrame/0
CrossSiteSubframe/DragAndDropBrowserTest.DragStartInFrame/0
SameSiteSubframe/DragAndDropBrowserTest.CrossSiteDrag/0
LocalNTPUITest.FakeboxRedirectsToOmnibox
TabDragging/DetachToBrowserTabDragControllerTest.DeleteBeforeStartedDragging/0
OmniboxViewTest.DoesNotUpdateAutocompleteOnBlur
SameSiteSubframe/DragAndDropBrowserTest.DragStartInFrame/0
TabDragging/DetachToBrowserInSeparateDisplayTabDragControllerTest.DragTabToWindowInSeparateDisplay/0
OmniboxViewViewsTest.SelectAllOnClick
OmniboxViewViewsTest.TextElideStatus
TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindowWhileInImmersiveFullscreenMode/0
SameSiteSubframe/DragAndDropBrowserTest.DragImageBetweenFrames/0
TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindowFromMaximizedWindow/0
DetachToBrowserInSeparateDisplayAndCancelTabDragControllerTest.CancelDragTabToWindowIn2ndDisplay
WebViewInteractiveTests/WebViewInteractiveTest.KeyboardFocusSimple/0
TabDragging/DetachToBrowserInSeparateDisplayTabDragControllerTest.DragTabToWindowOnSecondDisplay/0
WebViewInteractiveTests/WebViewInteractiveTest.KeyboardFocusWindowCycle/0

See the logs at:
https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/linux-chromeos-rel/67654
https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/linux-chromeos-rel/67723
https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/linux-chromeos-rel/67809
https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/linux-chromeos-rel/67872

Were all the other tests disabled? Did you notice the comment immediately below your change?
  // If this flakes, disable and log details in  http://crbug.com/523255 .
This seems to be a more pervasive issue than a single flaky test, and seems like it's not entirely fixed.

Comment 6 by sky@chromium.org, Feb 27 2018

Status: Available (was: Duplicate)

Comment 7 by msw@chromium.org, Feb 27 2018

Components: Infra>Flakiness>Dashboard Infra>Platform>Buildbot>TryServer
Labels: -OS-Chrome
Owner: dpranke@chromium.org
Status: Untriaged (was: Available)
Summary: chromium-try-flakes files bugs about failures in tryjobs with patches applied. (was: "BrowserFocusTest.ClickingMovesFocus" is flaky)
These failures are from a trybot *with a patch applied*; this bug never should have been filed, the test should not have been disabled!
All the cited failures are from this WIP CL's trybots (with patch): https://chromium-review.googlesource.com/c/chromium/src/+/932821
zakerinasab@, did you disable any other tests like this? Hopefully we can track down any other such bugs and changes.

I'm morphing this bug into a high-priority bug for chromium-try-flakes. Dirk, can you triage?
This tool should not be filing bugs about cq trybot failures with patches applied.
(or it should know these were all for one CL (no flake), and not recommend the disable/revert actions in the bug!)
Someone should probably audit chromium-try-flakes bugs for other inappropriate CLs disabling healthy tests.

As an aside,  Issue 523255  was Windows-specific; it was showing a Windows-only dialog blocking the tests from passing.

Comment 8 by msw@chromium.org, Feb 27 2018

Cc: zakerinasab@chromium.org
Owner: wylieb@chromium.org
Status: Assigned (was: Untriaged)
strange that we'd analyze failurs for an uncommitted CL. 

wylieb@, can you take a look?
There are three other similar bugs that I'm aware of:

 crbug.com/816645 
 crbug.com/816646 
crbug.com/816647

Project Member

Comment 11 by bugdroid1@chromium.org, Feb 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b106b538720231fc92472c6203bc79a7800b468d

commit b106b538720231fc92472c6203bc79a7800b468d
Author: Michael Wasserman <msw@chromium.org>
Date: Tue Feb 27 22:18:55 2018

Revert "Disable BrowserFocusTest.ClickingMovesFocus on Chrome OS"

This reverts commit da7df3358bd5750c6c3a684649a4e43232e8318c.

Reason for revert:
It only failed with a cq trybot patch applied, see the bug...

Original change's description:
> Disable BrowserFocusTest.ClickingMovesFocus on Chrome OS
> 
> Bug:816643
> Change-Id: I0b3f4cfaf67794e9904d8f269c69c3eaa98b9c05
> 
> TBR=msw@chromium.org
> 
> Change-Id: I0b3f4cfaf67794e9904d8f269c69c3eaa98b9c05
> Reviewed-on: https://chromium-review.googlesource.com/938486
> Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org>
> Reviewed-by: Mohammad Reza Zakerinasab <zakerinasab@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#539341}

TBR=msw@chromium.org,zakerinasab@chromium.org

Change-Id: Id966bb3d987533af026061e11bbabc5e7100a21e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  816643 
Reviewed-on: https://chromium-review.googlesource.com/939655
Reviewed-by: Michael Wasserman <msw@chromium.org>
Commit-Queue: Michael Wasserman <msw@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539559}
[modify] https://crrev.com/b106b538720231fc92472c6203bc79a7800b468d/chrome/browser/ui/browser_focus_uitest.cc

Comment 12 by msw@chromium.org, Feb 27 2018

Cc: sky@chromium.org
 Issue 816645  has been merged into this issue.

Comment 13 by msw@chromium.org, Feb 27 2018

Cc: arthurso...@chromium.org
 Issue 816646  has been merged into this issue.

Comment 14 by msw@chromium.org, Feb 27 2018

 Issue 816645  looks like a dup of this, I'm reverting that disable CL too.
The other two bugs have failures across numerous CLs (with patch...), so they might be legit flakes? (I de-dupped 816646)
I guess checking for failures with patches applied might reveal flakes, but it shouldn't be flagged if all the failures are from one CL.
Status: Started (was: Assigned)
+1 msw@

I would argue that there's no basis for an issue to be filed if all the data only comes from one CL.

If Chromium-try-flakes uses committed+uncommitted patchset data (which it does), then the occurrences tracked should be distinct to the CL (as opposed to CL+Patchset). This will avoid filing a bug for a flaky test that is only flaky in a WIP cl.

I'll get a fix in the works.

Comment 16 by sky@chromium.org, Feb 28 2018

It doesn't matter if the data comes from more than one cl. The "with patch" test results should always be suspect because who knows what changes may be in a patch. If the "with patch" and "without patch" both fail the same set of tests there is a high likelihood the test is flaky on the main waterfall, but there is no guarantee that is true. It is entirely possible the patch may have truly broke or made things worse for the tests.
Good point. 

Uncommitted (with patch) data isn't as useful, but if the patchset is eventually committed then that data is useful imo.

I agree there's no guarantees, but as it happens across different cls the probability that a flake has occurred increases. There's definitely room for improvement here!
Project Member

Comment 18 by bugdroid1@chromium.org, Mar 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/6bafff0da9d209b92abdc6598b497fcea75bf0aa

commit 6bafff0da9d209b92abdc6598b497fcea75bf0aa
Author: Brandon Wylie <wylieb@chromium.org>
Date: Fri Mar 02 00:11:46 2018

[CTF] Only allow one occurrence per changelist

CTF looks at committed and uncommitted cq attempts. A WIP CL caused
an issue to be filed from the cq attempts. This change only allows
one occurrence to exist per changelist.

Bug:816643
Change-Id: Ibfe0c7ea5ea47a848c6fe3feca184c91ae25e571
Reviewed-on: https://chromium-review.googlesource.com/940622
Commit-Queue: Brandon Wylie <wylieb@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/6bafff0da9d209b92abdc6598b497fcea75bf0aa/appengine/chromium_try_flakes/handlers/test/flake_issues_test.py
[modify] https://crrev.com/6bafff0da9d209b92abdc6598b497fcea75bf0aa/appengine/chromium_try_flakes/handlers/flake_issues.py

Status: Fixed (was: Started)

Sign in to add a comment