Disable CL is up https://codereview.chromium.org/2402703002/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/15ca7de391cdb9731e00426e9c524160ae2d7347

commit 15ca7de391cdb9731e00426e9c524160ae2d7347
Author: zhenw <zhenw@chromium.org>
Date: Sat Oct 08 02:16:30 2016

Disable service_worker.service_worker_micro_benchmark on Android WebView Perf

TBR=nednguyen@google.com
BUG= chromium:653924 

Review-Url: https://codereview.chromium.org/2402703002
Cr-Commit-Position: refs/heads/master@{#424047}

[modify] https://crrev.com/15ca7de391cdb9731e00426e9c524160ae2d7347/tools/perf/benchmarks/service_worker.py

shimazu@
Can you handle this?

According to the following dashboard, something broke the perf test on Sep 15.
https://chromeperf.appspot.com/report?sid=2998962a3c6cf831f715dd6dba8753f5ba03e98de8716d2229243b969e65fde8

but I'm not familiar with the current status of the service workers on WebView.
timvolodine@: Do you have any idea about this?

Normally service workers should work in webview as well. Not sure if there were any changes to service workers themselves recently though.

Thanks.
I manually did bisect and found the first bad commit is https://crrev.com/440493088d060522b1d1a35346edec1dba12fac8 . 

This issue seems a regression from a catapult change ( https://crrev.com/3cc40b37372927435ed69f91aec2117488b0825c ).

nednguyen@: Could you take a look?

Ah, this used to break webview because webview didn't support --proxy flag. May be it's fine to reenable now.

Revert of the disabling CL in https://codereview.chromium.org/2412213002/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2b1cd31e7d0a45eac02a7fce8c792484f528b4cc

commit 2b1cd31e7d0a45eac02a7fce8c792484f528b4cc
Author: nednguyen <nednguyen@google.com>
Date: Wed Oct 12 19:23:12 2016

Revert of Disable service_worker.service_worker_micro_benchmark on Android WebView Perf (patchset #1 id:1 of https://codereview.chromium.org/2402703002/ )

Reason for revert:
The benchmark should be runnable on webview now.

Original issue's description:
> Disable service_worker.service_worker_micro_benchmark on Android WebView Perf
>
> TBR=nednguyen@google.com
> BUG= chromium:653924 
>
> Committed: https://crrev.com/15ca7de391cdb9731e00426e9c524160ae2d7347
> Cr-Commit-Position: refs/heads/master@{#424047}

TBR=zhenw@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= chromium:653924 

Review-Url: https://codereview.chromium.org/2412213002
Cr-Commit-Position: refs/heads/master@{#424818}

[modify] https://crrev.com/2b1cd31e7d0a45eac02a7fce8c792484f528b4cc/tools/perf/benchmarks/service_worker.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ae502f1b55708a8501f40446ff6413cece546040

commit ae502f1b55708a8501f40446ff6413cece546040
Author: shimazu <shimazu@chromium.org>
Date: Thu Oct 13 04:14:49 2016

Reland of Disable service_worker.service_worker_micro_benchmark on Android WebView Perf (patchset #1 id:1 of https://codereview.chromium.org/2412213002/ )

Reason for revert:
This still breaks the tests.

Original issue's description:
> Revert of Disable service_worker.service_worker_micro_benchmark on Android WebView Perf (patchset #1 id:1 of https://codereview.chromium.org/2402703002/ )
>
> Reason for revert:
> The benchmark should be runnable on webview now.
>
> Original issue's description:
> > Disable service_worker.service_worker_micro_benchmark on Android WebView Perf
> >
> > TBR=nednguyen@google.com
> > BUG= chromium:653924 
> >
> > Committed: https://crrev.com/15ca7de391cdb9731e00426e9c524160ae2d7347
> > Cr-Commit-Position: refs/heads/master@{#424047}
>
> TBR=zhenw@chromium.org
> # Not skipping CQ checks because original CL landed more than 1 days ago.
> BUG= chromium:653924 
>
> Committed: https://crrev.com/2b1cd31e7d0a45eac02a7fce8c792484f528b4cc
> Cr-Commit-Position: refs/heads/master@{#424818}

TBR=zhenw@chromium.org,nednguyen@google.com
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= chromium:653924 

Review-Url: https://codereview.chromium.org/2412983005
Cr-Commit-Position: refs/heads/master@{#424954}

[modify] https://crrev.com/ae502f1b55708a8501f40446ff6413cece546040/tools/perf/benchmarks/service_worker.py

Perf tests failed again, so I reverted the revert patch.
See build #427 - #431: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus6%20WebView%20Perf%20%282%29

FYI, I put a command I used for bisecting:
$ gclient sync && ninja -C out/AndroidMaster -j1000 android_webview_apk && ./build/android/adb_install_apk.py out/AndroidMaster/apks/AndroidWebView.apk && ./tools/perf/run_benchmark run  --browser=android-webview-shell service_worker.service_worker_micro_benchmark

This issue was created > 6 months ago. The perf waterfall has changed significantly since then. If this bug is still relevant, please re-open.

Will re-enable benchmark and see if it can be disabled on individual failing pages instead.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cb5d43903a7bb942cefc0122d8a49689a6018448

commit cb5d43903a7bb942cefc0122d8a49689a6018448
Author: Juan A. Navarro Perez <perezju@chromium.org>
Date: Tue Aug 08 16:55:33 2017

[tools/perf] Reenable service_worker.service_worker_micro_benchmark

Renable the benchmark on webview to see if individual pages can be
disabled instead.

This is part of the benchmark reenabling fix-it, where we're aiming to
bring the stories that are disabled in line with the stories that are
actually still failing. We're also migrating from wholesale benchmark
disabling to per-story disabling when a story fails.

NOTRY=true

Bug:  653924 ,713222
Change-Id: Ia1c2f0dbd9b6346bc9ea30680c20eb7d43638768
Reviewed-on: https://chromium-review.googlesource.com/606248
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#492662}
[modify] https://crrev.com/cb5d43903a7bb942cefc0122d8a49689a6018448/tools/perf/benchmarks/service_worker.py

The benchmark keeps failing on Webview, with the same as the original error, timing out after 60 seconds for:

    action_runner.WaitForJavaScriptCondition('window.done')

I'll see if I can reproduce locally.

Forgot to add, links to newly seen failures:

https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5X%20WebView%20Perf
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20WebView%20Perf

Yep, reproduces locally. The page fails to load and shows on screen:

  The webpage at http://localhost:8091/index.html could
  not be loaded because:

  net::ERR_SOCKS_CONNECTION_FAILED

As in the screenshot from the bots:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_0-2017-08-13_16-33-0449083.png

And the benchmark does work fine on Chrome.

Sadly we can't bisect now because who knows what range it broke when it started failing nearly a year ago.

WebView folks, any hints on how to further diagnose this?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/352393f5e715d43bb8b6bc3b0bc7c924e25c915a

commit 352393f5e715d43bb8b6bc3b0bc7c924e25c915a
Author: Juan A. Navarro Perez <perezju@chromium.org>
Date: Mon Aug 14 15:06:52 2017

[tools/perf] Disable service_worker_micro_benchmark on WebView

Story is still failing on WebView. Disable until a fix is found.

Bug:  653924 
Change-Id: I05ecec70c1955755ba54cfc807aebe89cf055cd8
Reviewed-on: https://chromium-review.googlesource.com/613262
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#494050}
[modify] https://crrev.com/352393f5e715d43bb8b6bc3b0bc7c924e25c915a/tools/perf/benchmarks/service_worker.py

Assigning to benchmark owner to triage.

The tests have been removed.
See   issue 736518  .