The culprit CL isn't clear to me here. Will disable the test.

https://codereview.chromium.org/2631313004 disables the test.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/00320b068a7e061eb3634293c19a0401357fa66a

commit 00320b068a7e061eb3634293c19a0401357fa66a
Author: meade <meade@chromium.org>
Date: Wed Jan 18 02:43:47 2017

Disable ImeTest testLongPressInputWhileComposingText due to flakiness

BUG= 682080 
TBR=changwan@chromium.org

Review-Url: https://codereview.chromium.org/2631313004
Cr-Commit-Position: refs/heads/master@{#444251}

[modify] https://crrev.com/00320b068a7e061eb3634293c19a0401357fa66a/content/public/android/javatests/src/org/chromium/content/browser/input/ImeTest.java

It's all green without patch.

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=content_shell_test_apk&tests=org.chromium.content.browser.input.ImeTest%23testLongPressInputWhileComposingText

As for the with-patch runs, only two CLs were tested and one of them was IME CL: https://codereview.chromium.org/2370663002

Let me first check if the test failure was genuine failure or not.

Removing from sheriff queue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/047691b2cea1a7a8444e3b54ecd4827d87da2995

commit 047691b2cea1a7a8444e3b54ecd4827d87da2995
Author: changwan <changwan@chromium.org>
Date: Tue Jan 24 02:37:15 2017

Revert of Disable ImeTest testLongPressInputWhileComposingText due to flakiness (patchset #1 id:1 of https://codereview.chromium.org/2631313004/ )

Reason for revert:
test failures observed only with random patchset, re-enabling until we get more evidence that this is flaky.

Original issue's description:
> Disable ImeTest testLongPressInputWhileComposingText due to flakiness
>
> BUG= 682080 
> TBR=changwan@chromium.org
>
> Review-Url: https://codereview.chromium.org/2631313004
> Cr-Commit-Position: refs/heads/master@{#444251}
> Committed: https://chromium.googlesource.com/chromium/src/+/00320b068a7e061eb3634293c19a0401357fa66a

TBR=meade@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 682080 

Review-Url: https://codereview.chromium.org/2657513003
Cr-Commit-Position: refs/heads/master@{#445613}

[modify] https://crrev.com/047691b2cea1a7a8444e3b54ecd4827d87da2995/content/public/android/javatests/src/org/chromium/content/browser/input/ImeTest.java

jbudorick@, is there any way to turn off try flakes reports for with-patch runs?

> jbudorick@, is there any way to turn off try flakes reports for with-patch runs?

that's how it's supposed to work I *think*. I'm guessing that tryflakes look for flakes by looking for:
one bot run where test failed (obviously with patch)
subsequent bot run with *exact same patch set* passed

At least that's been the case for all tryflake links I looked at in the past (which admittedly has been awhile..)

AFAIK Bo is correct. (+sergiyb just in case, though)

Hmm... But does it still make sense to disable tests for with-patch runs if flakiness is introduced by a random patch?

> Hmm... But does it still make sense to disable tests for with-patch runs if flakiness is introduced by a random patch?

that's basically turning off flake detection on trybots. imo there aren't enough main waterfall runs to effectively detect flakes.

Did you mean disable *tests* or disable *try-flakes* for with-patch runs?

Bo addressed the latter case; in the former case, I think it depends on several factors:
 - how flaky the test has become after the random patch
 - how soon someone will actually be able to investigate
 - how widespread the newfound flakiness is

For single tests, it typically makes sense to disable until someone has time to investigate if it's flaking at a rate high enough to be reported by try-flakes.

My assumption was that trybot always runs two sets: one with the given patch and one without the patch for each trybot run. So I wondered if half of the runs are enough to detect flakiness. I didn't know that without-patch actually comes from the waterfall. I'm fine with leaving it as it is, then.

Re #14: I mean disable *try-flakes*. My bad.

(to clarify: without-patch runs on the try bots only when with-patch fails s.t. the trybot can determine whether the patchset is responsible for the failure or not; this is why you'll occasionally see green try bot runs that have test failures.)

> My assumption was that trybot always runs two sets: one with the given patch and one without the patch for each trybot run.

The "without patch run" only happens if a test fails.

Thanks for the clarification! I agree that we need to keep it as is, then.

Chromium Try Flakes does have occasional false positives, but we minimize this number by requiring at least 3 flakes of a given test to happen in the last 24 hours, which is rather unlikely to happen on the same patch. The user would have to CQ the patchset at least twice.

Also we only consider a test flaky, when it has also passed in a different build on the same patchset, thus making it likely that CQ will commit the patchset to the repo after all and flakiness will end up affecting other developers.