The failures are timeouts on Windows.

I will disable this test.

(According to the flakiness dashboard, it appears this test has always been flaky on Windows since it was landed.)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ee44456b15b6a68c56d6227032a4812145227e01

commit ee44456b15b6a68c56d6227032a4812145227e01
Author: Mark Pearson <mpearson@chromium.org>
Date: Fri Jun 08 22:01:56 2018

Disable Flaky Vertical Scroll Touch Block Test on Windows

Don't run
virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html
on Windows.

TBR=ekaramad
NOTRY=true

Bug: 850964
Change-Id: I613ec9919482aec151738b0350c951ece4b9531d
Reviewed-on: https://chromium-review.googlesource.com/1093834
Commit-Queue: Mark Pearson <mpearson@chromium.org>
Reviewed-by: Mark Pearson <mpearson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#565753}
[modify] https://crrev.com/ee44456b15b6a68c56d6227032a4812145227e01/third_party/WebKit/LayoutTests/TestExpectations

Removing from sheriff queue

Detected 5 new flakes for test/step "virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfwsSBUZsYWtlInR2aXJ0dWFsL3RocmVhZGVkL2V4dGVybmFsL3dwdC9mZWF0dXJlLXBvbGljeS9leHBlcmltZW50YWwtZmVhdHVyZXMvdmVydGljYWwtc2Nyb2xsLXRvdWNoLWJsb2NrLW1hbnVhbC50ZW50YXRpdmUuaHRtbAw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

Detected 4 new flakes for test/step "virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfwsSBUZsYWtlInR2aXJ0dWFsL3RocmVhZGVkL2V4dGVybmFsL3dwdC9mZWF0dXJlLXBvbGljeS9leHBlcmltZW50YWwtZmVhdHVyZXMvdmVydGljYWwtc2Nyb2xsLXRvdWNoLWJsb2NrLW1hbnVhbC50ZW50YXRpdmUuaHRtbAw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

mpearson@: Tests are still failing despite that you disabled it. And it seems to still be. I can't tell why, can you have a look?

The test seems disabled on Win and Mac now (Skip). But FYI, my patch in CQ will re-eanble (test modified a bit) and hopefully won't flake on Windows again.

The test will also remain skipped on Mac.

Hmm there are still flakes on mac and win:


try run at 2018-06-26 00:35:21 UTC	(milo)	patchset	mac_chromium_rel_ng	
try run at 2018-06-26 00:26:48 UTC	(milo)	patchset	win7_chromium_rel_ng	
try run at 2018-06-25 23:33:00 UTC	(milo)	patchset	mac_chromium_rel_ng	(reported in 850964)
try run at 2018-06-25 23:24:29 UTC	(milo)	patchset	mac_chromium_rel_ng	(reported in 850964)
try run at 2018-06-25 23:16:19 UTC	(milo)	patchset	mac_chromium_rel_ng	(reported in 850964)
try run at 2018-06-25 23:15:29 UTC	(milo)	patchset	win7_chromium_rel_ng	(reported in 850964)
try run at 2018-06-25 23:12:32 UTC

https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfwsSBUZsYWtlInR2aXJ0dWFsL3RocmVhZGVkL2V4dGVybmFsL3dwdC9mZWF0dXJlLXBvbGljeS9leHBlcmltZW50YWwtZmVhdHVyZXMvdmVydGljYWwtc2Nyb2xsLXRvdWNoLWJsb2NrLW1hbnVhbC50ZW50YXRpdmUuaHRtbAw

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f63aa4eb4791b8810dea24fc70860c292bb1b409

commit f63aa4eb4791b8810dea24fc70860c292bb1b409
Author: Ehsan Karamad <ekaramad@chromium.org>
Date: Thu Jun 28 14:39:53 2018

Mark a vertical-scroll test as 'slow'

The test has been previously disabled (skipped) on Windows due to
recurrent timeouts. Running the test locally leads to an average of ~
3.7 sec runtime on a high-performance Windows 10 rig (Z840). The test is
not skipped on Linux right now, but since its runtime exceeds the 2 sec
rule of thumb (from the comments in SlowTests), this CL will make it
slow on Linux as well.

Bug: 850964
Change-Id: Ib478c0f59313da4302325578d9b768e1070bdcb6
Reviewed-on: https://chromium-review.googlesource.com/1115214
Reviewed-by: Ehsan Karamad <ekaramad@chromium.org>
Reviewed-by: Kent Tamura <tkent@chromium.org>
Commit-Queue: Ehsan Karamad <ekaramad@chromium.org>
Cr-Commit-Position: refs/heads/master@{#571116}
[modify] https://crrev.com/f63aa4eb4791b8810dea24fc70860c292bb1b409/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/f63aa4eb4791b8810dea24fc70860c292bb1b409/third_party/WebKit/LayoutTests/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-action-manual.tentative.html

Removing myself from thread; I'm no longer sheriff.  (And, no, I don't know why they were still running despite the changelist in comment #5.)

Making this generic about external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html as the test is now failing fairly consistently on virtual/video-surface-layer/ even though the test doesn't use a video.

Detected 3 new flakes for test/step "virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfwsSBUZsYWtlInR2aXJ0dWFsL3RocmVhZGVkL2V4dGVybmFsL3dwdC9mZWF0dXJlLXBvbGljeS9leHBlcmltZW50YWwtZmVhdHVyZXMvdmVydGljYWwtc2Nyb2xsLXRvdWNoLWJsb2NrLW1hbnVhbC50ZW50YXRpdmUuaHRtbAw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/214b3b07634a6788c02a4c2a79a57f8bb429c0d5

commit 214b3b07634a6788c02a4c2a79a57f8bb429c0d5
Author: Yi Gu <yigu@chromium.org>
Date: Sat Jun 30 04:01:23 2018

Mark virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html. timeout on Win7

TBR=ekaramad@chromium.org
NOTRY=True

Bug: 850964
Change-Id: I9ae6e227f3ab158c7aa93217358cbae11ba2ede4
Reviewed-on: https://chromium-review.googlesource.com/1121633
Reviewed-by: Yi Gu <yigu@chromium.org>
Commit-Queue: Yi Gu <yigu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#571757}
[modify] https://crrev.com/214b3b07634a6788c02a4c2a79a57f8bb429c0d5/third_party/WebKit/LayoutTests/TestExpectations

Detected 3 new flakes for test/step "virtual/threaded/external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfwsSBUZsYWtlInR2aXJ0dWFsL3RocmVhZGVkL2V4dGVybmFsL3dwdC9mZWF0dXJlLXBvbGljeS9leHBlcmltZW50YWwtZmVhdHVyZXMvdmVydGljYWwtc2Nyb2xsLXRvdWNoLWJsb2NrLW1hbnVhbC50ZW50YXRpdmUuaHRtbAw. This message was posted automatically by the chromium-try-flakes app.

Still timing out on Windows, I'm changing to Skip on Windows again.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e86d26e91b185df03681d212ff6c1d12b6af956b

commit e86d26e91b185df03681d212ff6c1d12b6af956b
Author: Luciano Pacheco <lucmult@chromium.org>
Date: Tue Jul 03 01:59:40 2018

Mark flaky test as skip on Win due to timeout

Test external/wpt/feature-policy/experimental-features/vertical-scroll-touch-block-manual.tentative.html
has been marked as flaky by Try-Flakes, skipping until further
investigation.

Bug: 850964
Tbr: ekaramad@chromium.org
Change-Id: I13ef5fd0520a2e04e1c7166fff2fd2a52a8d0add
Reviewed-on: https://chromium-review.googlesource.com/1123739
Reviewed-by: Luciano Pacheco <lucmult@chromium.org>
Commit-Queue: Luciano Pacheco <lucmult@chromium.org>
Cr-Commit-Position: refs/heads/master@{#572083}
[modify] https://crrev.com/e86d26e91b185df03681d212ff6c1d12b6af956b/third_party/WebKit/LayoutTests/TestExpectations

Looking at the link in comment #19 for "All flaky occurrences" of the test I see around 16 flakes on June 7-8 for the earlier version of the test. This is before adding <meta name="timeout" content="long"> to the test.

From June 29 till July 3rd I can see 12 flakes (over a span of 5 days). Of course the sample size is not reliably large, but still it hints at a reduction in the flakes from 8/day to 2.4 per day. So perhaps the timeouts are actually the test taking too long to finish.

I noticed the meta tag for WPT sets the timeout to 10 sec which is slightly less than 3x the average runtime on my local machine...so supposedly if the sole reason for failures are tests taking too long (as opposed to never ending) then maybe I should mark the test as SlowTest as well.

..actually, my mistake. The "long" tests have a timeout of 60 seconds so perhaps something is wrong with the test itself.
https://web-platform-tests.org/writing-tests/testharness-api.html#harness-timeout

This is perhaps due to the input API for testing. I will make another attempt at this when testdriver supports touch-scroll gestures.