The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e40478bf67a1995acf352543b09018f0de6763c5

commit e40478bf67a1995acf352543b09018f0de6763c5
Author: Finnur Thorarinsson <finnur@chromium.org>
Date: Fri Nov 03 13:48:33 2017

Disable four webkit tests (NOTIMPLEMENTED in shell_android.cc)

TBR: awdf
Bug:  781171 
Change-Id: I68f6d458c4246fbc99a43c0b6cec37b28dd1bbb0
Reviewed-on: https://chromium-review.googlesource.com/753221
Reviewed-by: Finnur Thorarinsson <finnur@chromium.org>
Commit-Queue: Finnur Thorarinsson <finnur@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513762}
[modify] https://crrev.com/e40478bf67a1995acf352543b09018f0de6763c5/third_party/WebKit/LayoutTests/TestExpectations

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6925e54574810433f8cce3bd507d95c594dac658

commit 6925e54574810433f8cce3bd507d95c594dac658
Author: Finnur Thorarinsson <finnur@chromium.org>
Date: Fri Nov 03 15:39:16 2017

Revert "Disable four webkit tests (NOTIMPLEMENTED in shell_android.cc)"

This reverts commit e40478bf67a1995acf352543b09018f0de6763c5.

Reason for revert: The exceptions didn't seem to work as intended.

Original change's description:
> Disable four webkit tests (NOTIMPLEMENTED in shell_android.cc)
> 
> TBR: awdf
> Bug:  781171 
> Change-Id: I68f6d458c4246fbc99a43c0b6cec37b28dd1bbb0
> Reviewed-on: https://chromium-review.googlesource.com/753221
> Reviewed-by: Finnur Thorarinsson <finnur@chromium.org>
> Commit-Queue: Finnur Thorarinsson <finnur@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#513762}

TBR=finnur@chromium.org,awdf@chromium.org

Change-Id: I8d3b86e05b00d2b013597c43e38b65183bc3dc42
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  781171 
Reviewed-on: https://chromium-review.googlesource.com/753901
Reviewed-by: Finnur Thorarinsson <finnur@chromium.org>
Commit-Queue: Finnur Thorarinsson <finnur@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513790}
[modify] https://crrev.com/6925e54574810433f8cce3bd507d95c594dac658/third_party/WebKit/LayoutTests/TestExpectations

 Issue 781306  has been merged into this issue.

Do you think maybe the expectations didn't work because we're crashing, not failing (a like expect/assert)? Maybe [ Crash Failure ] or [ Skip ] would work?

Actually looking at the log, it seems to clearly be a failure, not a crash, I have no idea how these webkit tests work, feel free to disregard my previous comment.

When the NOTIMPLEMENTED occurs, it's a crash (see https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_layout_tests&tests=external%2Fwpt%2Fcss-timing%2Fstep-timing-functions-output.html). The tests sometimes also fail because of other reasons, e.g. the test is not properly loaded, causing it to produce extra output instead of running the harness test. Re #5: Yes.

xhwang@ (who made the latest change to the NOTIMPLEMENTED) can you check why the NOTIMPLEMENTED is reached? Should the function be just no-op?

I only added "<< title" to the line, and I didn't touch NOTIMPLEMENTED() :)

https://chromium.googlesource.com/chromium/src/+/68722de845c749dfc8dbac5f629b6cb40ea5a11a%5E%21/#F0

But, I don't understand why a NOTIMPLEMENTED would cause a crash. It should only produce some logs in certain builds.

There seems other reason causing the flaky crash.

Alright, well lets expect the crash then. We don't want to leave the builders red while we figure this out. I'll create CL in a minute that includes Crash and see if that works.

https://chromium-review.googlesource.com/c/chromium/src/+/754051

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/040b8372894fc90864847c540a5cb5ea3d0f00b1

commit 040b8372894fc90864847c540a5cb5ea3d0f00b1
Author: Sky Malice <skym@chromium.org>
Date: Fri Nov 03 23:08:59 2017

Supress failing css-timing webkit layout tests.

Tests:
* external/wpt/css/css-timing/cubic-bezier-timing-functions-output.html
* external/wpt/css/css-timing/frames-timing-functions-output.html
* external/wpt/css/css-timing/frames-timing-functions-syntax.html
* external/wpt/css/css-timing/step-timing-functions-output.html

TBR=wangxianzhu@chromium.org

Bug:  781171 
Change-Id: Id6d2891886f10eddc81053e206a5a5a40d2329a4
Reviewed-on: https://chromium-review.googlesource.com/754051
Reviewed-by: Sky Malice <skym@chromium.org>
Commit-Queue: Sky Malice <skym@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513945}
[modify] https://crrev.com/040b8372894fc90864847c540a5cb5ea3d0f00b1/third_party/WebKit/LayoutTests/TestExpectations

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa6e017b0fb87bc373294f28f5bf2541d240210b

commit aa6e017b0fb87bc373294f28f5bf2541d240210b
Author: Eddy Mead <meade@chromium.org>
Date: Mon Nov 06 00:55:48 2017

Skip failing css-timing LayoutTests

Tests:
* external/wpt/css/css-timing/frames-timing-functions-output.html
* external/wpt/css/css-timing/frames-timing-functions-syntax.html
* external/wpt/css/css-timing/step-timing-functions-output.html

TBR=rjwright

Bug:  781171 
Change-Id: I0799893780778304acb62a33b4694c58de0a6de0
Reviewed-on: https://chromium-review.googlesource.com/754143
Commit-Queue: meade_UTC10 <meade@chromium.org>
Reviewed-by: Renée Wright <rjwright@chromium.org>
Reviewed-by: meade_UTC10 <meade@chromium.org>
Cr-Commit-Position: refs/heads/master@{#514076}
[modify] https://crrev.com/aa6e017b0fb87bc373294f28f5bf2541d240210b/third_party/WebKit/LayoutTests/TestExpectations

With the exclusion these are now passing.

This is the first build that showed the failure: https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Android%20%28Nexus4%29/builds/71009 . Range is 513620-513623. https://chromium.googlesource.com/chromium/src/+/8614045a44169ecb2501ff63f91fe667f625353f and https://chromium.googlesource.com/chromium/src/+/a081bb535fd5e4ef21e88e994afcd9e57084a930 look like the two possibilities. Xianzhu, is it worth trying to revert your change (and the exclusion) to see what happens?

It's unrelated to my patch. The tests started to fail since that build because the tests were newly added into SmokeTests in https://chromium.googlesource.com/chromium/src/+/a081bb535fd5e4ef21e88e994afcd9e57084a930.

However, not only just the several tests pointing this bug, but also a lot of other tests are failing because of the same error. See https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium.webkit%2FWebKit_Android__Nexus4_%2F71012%2F%2B%2Frecipes%2Fsteps%2Fwebkit_tests%2F0%2Fstdout.

The tests produced the following text dump:

layer at (0,0) size 980x735
  LayoutView at (0,0) size 980x735
layer at (0,0) size 980x735
  LayoutBlockFlow {HTML} at (0,0) size 980x735
    LayoutBlockFlow {BODY} at (8,8) size 964x719
      LayoutText {#text} at (0,0) size 242x19
        text run at (0,0) width 242: "Could not load the requested resource."
      LayoutBR {BR} at (242,15) size 0x0
      LayoutText {#text} at (0,20) size 373x19
        text run at (0,20) width 373: "Error code: -102 (net::ERR_CONNECTION_REFUSED)"

It seems that the test forwarder on the device is refusing HTTP connection for the tests which might be because the device_forwarder crashed.

The NOTIMPLEMENTED is reached because the "Could not load the requested resource" page sets title.

jbudorick@ can you take a look?

Removing Sheriff-Chromium now that the bug is assigned and being investigated.

Regarding comment 15, I think it is the same as  issue 786550  that I'm working on. Turns out we forgot to forward ports used by wptserve in Android port forwarder so all external/wpt/... tests are producing layout dumps instead of harness output (PASS/FAIL, etc.) on Android due to the lack of testharness{report}.js. Generally speaking, WPT should never output layout dump; otherwise, something is wrong with the test runner.

The four css-timing tests in the original port also failed in the same pattern (layout dump in output): https://storage.googleapis.com/chromium-layout-test-archives/WebKit_Android__Nexus4_/71009/layout-test-results/results.html

My WIP CL for  issue 786550  seems to work. Therefore, I'm merging this issue into 786550; and I'll remove the related test expectations and rebaseline all WPT on Android.