I forgot to note. In the chrome browser, everything still runs perfectly. It seems to be only the webview that is affected.

Here is a jsfiddle that can reproduce the issue.

On Phones Chrome browser this runs at 30-32 fps.

When launched via webview it is 10-12fps.

https://jsfiddle.net/titansoftime/kr20tqo2/3/embedded/result/

After running numerous tests, it seems this performance regression is not limited to WebGL.

Running the Octane 2.0 benchmark (5 times each) on both the Chrome Browser and the webview has shown me that js performance is 1.5 - 2x faster in the browser compared to the webview.

I am using a Samsung Galaxy s7 edge.

Thanks for the repro case. It's important to find the root of this issue. +P1 for now in case this is affecting a lot of content out there.

+Performance +Blink/JavaScript to take a look.

Submitter: do you have a shell app which embeds WebView and which can be used to demonstrate the performance slowdown with https://jsfiddle.net/titansoftime/kr20tqo2/3/embedded/result/ or in general to compare against the browser?

CC'ing a couple of WebView and V8 compiler folks. I don't know how to go about bisecting this.

Yea, I have an app on the google play store. It's a game called "Titans of Time".

This is a simple app I made in android studio that is pretty much the following:

        webview =(WebView)findViewById(R.id.webView);

        webview.setWebViewClient(new WebViewClient());
        webview.getSettings().setJavaScriptEnabled(true);
        webview.getSettings().setDomStorageEnabled(true);
        webview.setOverScrollMode(WebView.OVER_SCROLL_NEVER);
        webview.loadUrl("https://www.titansoftime.com");

Note to others: this is the app:
https://play.google.com/store/apps/details?id=com.titansoftime.hobbes.titansoftime

On my phone (a Pixel 2) I can see some hitching with this app that doesn't appear in Chrome Dev when viewing https://www.titansoftime.com . I still don't know how to go about bisecting this issue though.

Pixel 2XL, the jsfiddle is getting 43fps with webview 65, 66, and 67.

titansoftime@, are you able to test on a different device? And are you able to restore the previously observed performance by uninstalling chrome (or webview, if you've disabled chrome) updates?

I will do some additional testing.

I will say that just a few hours ago, a user signed up from the UK via the webview app with an old version of chrome (56), and was getting 40fps. Whereas the users signing up via the app with the current version (66) are getting closer to 10-12 fps (which is what I'm getting on my phone).

I will attempt to rollback the webview/chrome updates (didn't actually know I could do this), and see what happens.

I want to reiterate that everything was running just peachy on both the browser/webview on my s7 just a couple weeks ago.

Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

galaxy S7+, 7.0., webview (in shell) 65: 43fps 66: 35fps 67: 43fps

in app, anecdotally 66 and 67 feel more janky. systrace seems to confirm that on 67 a subset of frames take unexpectedly long to draw.

That was me. Is it possible to show an FPS counter in the app?

Others are already investigating, so I won't. I just want to point out that this change (merged to m66) affects throughput of main frames. But we've also got feedback on other bugs that it improved webgl in some other games, so ymmv.

https://chromium-review.googlesource.com/933292
https://bugs.chromium.org/p/chromium/issues/detail?id=822727#c9

I added an fps overlay (stats.js). I will leave it up until it is no longer needed. You can tap on the overlay to change mode (fps, ms, memory usage).

Unfortunately it seems that unlike the pixel 2, I cannot rollback my webview/ chrome updates without some serious tinkering.

In game framerate is a pretty consistent 30fps on a Pixel 2 XL on both 65 and 67.

Yea I just installed chrome dev and canary. Browser variants all perform relatively the same. 

Switched the webview to both dev and canary, same bad performance as stable.

Ok.....

SO I disabled chrome, switched webview to use "Google Webview" and it is running buttery smooth....

This is great, but of course users will not know (or be expected) to do this.

When you disable chrome you will initially switch back to an old version of webview. I presume that when that updates you'll find yourself back at square 1.

I tried switching back and forwards between 56 and 67 on an S7+, and couldn't see a difference in the measured FPS.

Sigh... yup, updated webview from 63 to 66 and as you predicted, back to square 1.

Glad the Pixel 2 is unaffected, however it seems at the very least mine is.

I also want to note that disabling chrome reset chrome back to version 63. This is also running great in app.

Clearly something post 63 had a negative effect on the s7.

I ran additional tests in Octane 2.0 with Chrome 63.

Both webview and browser performed roughly the same (appx score of 6k on both).

Whereas with chrome 66 I was getting around 6k in browser and roughly 3.5k in webview.

#12, jdarpinian: inconsistent framerate could be issue 835353?

After more and more testing I am really beginning to think this issue isn't specifically a WebGL issue but a javascript performance issue.

The octane tests definitely show this.

In addition, in my app there is a character select screen which contains only one skeletal animated mesh (high cpu cost). This runs ok in webview (66). The actual game scene contains roughly 80 skinned meshes and that's where performance drops massively (in webview, browser is fine).

Just retested Octane 2.0 on 66. Score of 6.0-6.5k in browser, 2.5-3k in webview.

octane regressing doesn't sound webgl specific then.. what android OS version are you running this on?

Yea, I mentioned that in comment 4. Couldn't say for certain though. Now it's looking like this is almost certainly on the CPU side.

I am running Android 7.0.

does things improve if you enable multiprocess webview in android's developer options, assuming you know where that's hidden

I've been hesitant to even test that as most users of course will not know about it but......... testing...... OMG yes. I am now getting an avg of 40fps!!!

Now to wait 10 years for this option to become defaulted to on =]

It's enabled for Android 8.0 and up, except on the lowest end devices.

sounds like we mark the in-process renderer process as low priority by mistake, happened before, looks like broke again :/

Ok, I don't know what's going on.

I double checked RenderThreadImpl::SetProcessBackgrounded for single process (the thing I suspected in #30) did not break for 66 stable.

Then I tried m63, m65, and m66 on https://jsfiddle.net/titansoftime/kr20tqo2/3/embedded/result/, and 63 is significantly worse than 65/66. I'm using an ancient nexus 5 here, but I'm comparing relative performance, so device shouldn't matter. And looks like that one is js/cpu bound anyway, so GPU shouldn't matter either.

And none of this explains your result from #28 that turning on multiprocess appear to fix it. In theory, there should be no negligible difference between single and multiprocess in in terms of raw throughput.

So I'm not even able to reproduce your result, so can't really investigate..

I compared octane 2 for 66 and 63 as well, and 66 was slightly higher. But in general it's kind of hard to make the octane consistent due to other background processing and thermals.

That's unfortunate.

I can confirm that with multiprocess webview enabled restores the pre 66 behavior on my s7. 

I can provide screenshots if need be. I'd mail you my phone if I didn't need it.

Tested Octane 2 with multiprocess webview enabled.

Results are same as browser, appx 6500.

With disabled, the results are the same miserable 2500-3000.

And yea I noticed Octane can vary + or - 500 or so. I've taken this into account.

So then, best theory is this is device specific..? Do you have other devices to check?

And what build fingerprint does your s7 have? Should be in adb shell dumpstate | head

I can't say that it is device dependent with certainty. Most users signing up with the app are getting around what I am (8-12fps) whereas they were getting 20-30fps.

Obviously something happened. When I roll back to Chrome 63, everything is fine.

And again, its only the webview, browser is as fine as always.

I do not have another android device unfortunately as my wife and inlaws are all apple people, and my android family lives 15oo miles away.

And build fingerprint? Samsung has many different device flavors, and it's probably important to try the right one.

Sorry forgot about that.

Build: NRD90M.G935VVRS4BRC3
Build fingerprint: 'Verizon/hero2qltevzw/hero2qltevzw:7.0/NRD90M/G935VVRS4BRC3:user/release-keys'
Bootloader: G935VVRS4BRC3
Radio: msm
Network: Verizon Wireless
Kernel: Linux version 3.18.31-12200908 (dpi@SWDG9809) (gcc version 4.9 20150123 (prerelease) (GCC) ) #1 SMP PREEMPT Wed Apr 4 10:38:14 KST 2018
uptime: write: Broken pipe
command 'uptime ' failed: No such file or directory
/sys/block/mmcblk0/: No such file or directory

I think we have confirmed that something is going on. Are you getting this results consistently? Maybe the scheduler puts the webview process sometimes on little cores and sometimes on big cores?

Is this Samsung specific (not device specific)?

This is extremely consistent. I don't think I've ever done so much testing with such reproducible results =]

Unfortunately, I have only this one device to test on, all other available devices are apple (I know...).

So reporter is using a galaxy s7 edge from verizon.

I tried a galaxy s7 from sprint, which is probably close enough. And essentially reproduced this. I ran octane 2 with each set up twice (not consecutively) to avoid noise. Here's the result:
M56 stable - 8688 8805
M63 beta - 6566 6882
M66 dev - 4627 4390
M63 stable - 7895 8145
M66 stable - 4119 3605

The stable/beta/dev here means which channel of chrome build I installed from archives; eg M63 beta and M63 stable are the exact same build, just packaged to different channels. So I think there's some shenanigans going here since M63 beta and stable differ by so much (I assume more than noise).

And yeah, clearly M66 is way worse no matter which channel it's running on. Also turning on multiprocess on M66 doubled the score. So should start bisecting this down to two point releases. Unfortunately can only install fully signed official builds on production devices, which means no debugging and best outcome is to bisect down to a single day worth of changes.

I'm travelling next week though, so won't be able to do that for awhile.

I'm glad you were able to reproduce the problem.

Thank you for your effort.

Enjoy your travels =]

Leszek/Mythri, do you have any idea what might cause this? See comment #41. I would assume it is related to a Finch experiment. 32/64 bit Chrome shouldn't have any impact because it is a WebView, right?

Webview does not support finch (yet)

Octane 2 results from an S7 edge don't repro:

samsung/hero2ltexx/hero2lte:7.0/NRD90M/G935FXXU1DQG1:user/release-keys

63.0.3239.0 11164
64.0.3282.0 11146
65.0.3325.0 9957
66.0.3359.0 10595
66.0.3359.168 11359

G935F is the international version with exynos chipset. US versions tend to have qualcomm chipset.

Re:#43, I am not sure it is related to code-caching finch trials. Octane score shouldn't be impacted much because of code-caching. We turned on code caching after execute by default in M66 (66.0.3359.106), but I don't think this could cause this problem. We don't spend too much time on compilation in Octane. Code-caching could only impact the compilation time. Also, the comment says the regression is fixed when turning on multiprocessor support which should not impact code-caching.

Agree with Mythri here, in general this feels like a scheduling issue, not a V8 one.

Bisected to https://chromium-review.googlesource.com/c/chromium/src/+/898435

The change referenced in #50 switches V8 over to Chrome's page_allocator. This would change most mmap/mprotect/munmap calls, so has the potential to cause problems. It is strange that it would only affect WebView and not Chrome.

It only affects *single process* webview. I guess if chrome had a single-process mode, it would also be affected, maybe.

As this seems to be related, adding this week's memory sheriffs so they can offer their opinion too.

New data point. This appears to only affect arm64, not arm32.

Forcing the embedding app to be 32bit fixed this even in single process. Since webview renderer process always runs in 32bit, turning on multiprocess effectively fixed this. It's also the reason why I couldn't reproduce this with a nexus 5.

Chrome doesn't ship arm64 to stable channel yet, so doesn't really affect chrome either

Thanks boliu@, that's very helpful. I'm investigating if ASLR changes with my CL might somehow be triggering this on arm64.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/475e3d5b2cc9e2649ffe8838cbebd53547b33c3a

commit 475e3d5b2cc9e2649ffe8838cbebd53547b33c3a
Author: Bill Budge <bbudge@chromium.org>
Date: Wed May 16 20:13:11 2018

[page_allocator] Restrict ASLR range on 64-bit ARM Android

- This is a speculative fix to address a large performance regression
  on Android WebView.
- Restricts the range on 64 bit ARM Android to 0x20000000 - 0x60000000.

Bug:  chromium:837640 
Change-Id: I913ed62a9d360ae96aeaab80f468c18f357f956f
Reviewed-on: https://chromium-review.googlesource.com/1062012
Reviewed-by: Chris Palmer <palmer@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559255}
[modify] https://crrev.com/475e3d5b2cc9e2649ffe8838cbebd53547b33c3a/base/allocator/partition_allocator/address_space_randomization.h

boliu@, when can we expect to see if this change helps?

I can try it tomorrow

With that change, 64bit is definitely back up in the same range as 32bit. On a pixel, from ~6500 to ~9000 for octane 2.

Is that safe enough to be merged back to m67?

Let's give this some bake time on Dev and Clusterfuzz first. 

Got enough bake time yet to consider merging back to m67?

I think this is a low risk change. For V8, it restores the restricted ASLR range that we had before. For Chrome, it restricts the range from 2^39 to 2^30 which is still 1GB so I don't think it will cause any problems.

I would like to figure out why the larger ASLR range caused the perf regression and will continue to track this down.

Merge request then. See #65

This bug requires manual review: We are only 6 days from stable.
Please contact the milestone owner if you have questions.
Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

bbudge: can you merge your CL to m67?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/73d0ff282254dd2d9bd5300cc3a83fb79e6ad7ec

commit 73d0ff282254dd2d9bd5300cc3a83fb79e6ad7ec
Author: Bill Budge <bbudge@chromium.org>
Date: Fri May 25 22:45:49 2018

[page_allocator] Restrict ASLR range on 64-bit ARM Android

- This is a speculative fix to address a large performance regression
  on Android WebView.
- Restricts the range on 64 bit ARM Android to 0x20000000 - 0x60000000.

Bug:  chromium:837640 
Change-Id: I913ed62a9d360ae96aeaab80f468c18f357f956f
Reviewed-on: https://chromium-review.googlesource.com/1062012
Reviewed-by: Chris Palmer <palmer@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#559255}(cherry picked from commit 475e3d5b2cc9e2649ffe8838cbebd53547b33c3a)
Reviewed-on: https://chromium-review.googlesource.com/1074107
Reviewed-by: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/branch-heads/3396@{#705}
Cr-Branched-From: 9ef2aa869bc7bc0c089e255d698cca6e47d6b038-refs/heads/master@{#550428}
[modify] https://crrev.com/73d0ff282254dd2d9bd5300cc3a83fb79e6ad7ec/base/allocator/partition_allocator/address_space_randomization.h

im getting 44fps. Looks like its been fixed. 
verified on: pixel  2 xl / p vs 67.0.3396.68

I do not believe the Pixel 2 was ever affected by this.

The issue remains in 67.0.3396.59.

I can confirm however that the issue appears to be resolved in 68.03440.8.

Thank you everyone.

Verified on Samusung S8/8.0. ,Pixel XL /8.1.0 I am seeing 37-43 fps having 68.0.3440.14 .