Please find the log & video @ http://go/chrome-androidlogs1/6/654369

Harr?

10-10 10:43:28.154 E/AndroidRuntime( 2828): Process: com.opera.android.browser, PID: 2828
10-10 10:43:28.154 E/AndroidRuntime( 2828): android.os.BadParcelableException: ClassNotFoundException when unmarshalling: org.chromium.content.browser.FileDescriptorInfo
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.os.Parcel.readException(Parcel.java:1685)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.os.Parcel.readException(Parcel.java:1636)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at flo.a(OperaSrc:105)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at fff.o(OperaSrc:370)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at fff.i(OperaSrc:32)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at ffg.onServiceConnected(OperaSrc:177)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.app.LoadedApk$ServiceDispatcher.doConnected(LoadedApk.java:1465)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.app.LoadedApk$ServiceDispatcher$RunConnection.run(LoadedApk.java:1482)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.os.Handler.handleCallback(Handler.java:751)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.os.Handler.dispatchMessage(Handler.java:95)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.os.Looper.loop(Looper.java:154)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at android.app.ActivityThread.main(ActivityThread.java:6077)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at java.lang.reflect.Method.invoke(Native Method)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
10-10 10:43:28.154 E/AndroidRuntime( 2828): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)

FileDescriptorInfo is something we send between our browser and renderer processes on start up. How did opera get hold of it??

Semi-random suspect: https://codereview.chromium.org/2384063005

If that CL is not the issue, then I think will need a bisect here. I dunno how this happens basically.

cc torne too since knows like all things android :p

Er, isn't opera its own browser with its own rendering engine? (which is chromium). This sounds like they have messed up the proguard config for their binary. Is there some reason you think this is webview-related?

> Er, isn't opera its own browser with its own rendering engine? (which is chromium). This sounds like they have messed up the proguard config for their binary. Is there some reason you think this is webview-related?

That could be true too, but then won't opera just be crashing all over the place, which seems unlikely as well. I dunno, this one is super confusing :/

I mean apparently we have good and bad *webview* builds here, so presumably it's something we changed, regardless of whose fault it is..

Opera news seems to do its own multiprocess:

marlin:/ # ps | grep sand                                                      
u0_a214   27472 658   1278316 70272 SyS_epoll_ 00eb343484 S com.opera.android.browser:sandboxed_process1

But it loads our plat_support:

marlin:/ # cat /proc/27472/maps | grep chrom                                   
db7f7000-db7fb000 r-xp 00000000 fd:00 1671                               /system/lib/libwebviewchromium_loader.so
db7fb000-db7fc000 r--p 00003000 fd:00 1671                               /system/lib/libwebviewchromium_loader.so
db7fc000-db7fd000 rw-p 00004000 fd:00 1671                               /system/lib/libwebviewchromium_loader.so
ebb3f000-ebb43000 r-xp 00000000 fd:00 1672                               /system/lib/libwebviewchromium_plat_support.so
ebb43000-ebb44000 r--p 00003000 fd:00 1672                               /system/lib/libwebviewchromium_plat_support.so
ebb44000-ebb45000 rw-p 00004000 fd:00 1672                               /system/lib/libwebviewchromium_plat_support.so

Whether it starts a renderer doesn't depend on the multiprocess setting (it always does, it seems).

Are you sure that the bisect held the app version constant? We broke FileDescriptorInfo marshalling for a while (proguard error), but it's fixed now. There's no sign of them loading webview, but plenty of evidence that they're using webview code, so maybe they just picked up a version that was broken in multiprocess?

> but plenty of evidence that they're using webview code, so maybe they just picked up a version that was broken in multiprocess?

They embed chromium at the content layer, so shares a lot of code with chrome/webview.

Maybe they use WebView just for their EULA?

> Maybe they use WebView just for their EULA?

Can we verify that?

That maps grep was when it was showing the EULA. It also doesn't show up in chrome://inspect

So then the bisect is just completely bogus?

No, not bogus. It worked for me for 55.0.2882.3 and crashed for 55.0.2883.0
https://chromium.googlesource.com/chromium/src/+log/55.0.2882.3..55.0.2883.0?pretty=fuller&n=10000

When it works, chrome://inspect shows nothing though.

it would be very interesting to get to the bottom of this

Reverting https://codereview.chromium.org/2390203006 on ToT build seems to fix the issue.

probably should just revert then, although this is some magic going on right here....

M-mm we can dig deeper. Especially now we know that we can just revert...
Chrome.apk is used in that scenario somehow, I believe this is not what we would expect?

Btw after it worked, in order to reproduce the problem it is not enough to modify the code, recompile and reinstall. I also had to force stop Opera and clear its data (and cache).

Also, when filing bugs, if the issue repros in the "normal" configuration (e.g. here, with just regular monochrome stable) please say so, because when bugs call out an unusual configuration (like chrome disabled and using standalone webview instead) it's easy to assume that is actually required to reproduce the bug, which can be misleading - in this case it fails no matter what config you have on 2883 and up.

So, fun things so far:

1) Doesn't matter whether it's monochrome or webview, behaves the same in them all.

2) For me, it crashes in this way literally on startup, I don't have to go anywhere in the app.

3) Opera's APK definitely uses WebView *as well* as having chromium linked into it.

4) Everyone has libwebviewchromium_plat_support.so loaded on N because of complex handwaving involving linker namespaces; that doesn't signify anything.

5) However opera's main process has actually loaded webview as well, in the case where it doesn't crash. (their child processes have not)

Probably this is something being initialised wrong in ChildProcessLauncher that I've disrupted by calling warmUp() early on, and we're now trying to actually use the services declared in Opera's manifest? Probably not their bug, just only happens in apps that happen to have all the Chromium manifest data for renderers?

I'll look into it, anyway.

(I pruned a couple of comments here so that I can open up the bug; the deleted comments aren't relevant to the issue at hand).

OK, I can't confirm this because I can't figure out in what case Opera actually *use* webview rather than just loading the code, but my hypothesis is that Opera was actually already broken with multiprocess webview turned on, just only in some case that we aren't hitting when we test it.

The problem is that when the multiprocess webview creates a renderer process, the service name of that renderer is "com.opera.browser/org.chromium.content.app.SandboxedProcessService0", because WebView's renderers are external services that get the package name (and uid association) of the app using WebView, not the package name of WebView itself.

However, this is *also* the service name of Opera's own first renderer process, so they can't both exist at once - service names have to be unique systemwide.

My change makes WebView create its renderer process *very* early in browser process startup, which means that our process gets created first (successfully). It looks like Opera's browser process then connects to that service and tries to use it, which results in a weird exception - probably simply because the versions don't match.

However, without my change, what I assume happens is that Opera creates its renderer process first, and Opera's own rendering is fine, but then if Opera actually create a WebView and perform a navigation inside it, we will attempt to bind to our renderer process and will crash there instead because the service name is already taken by Opera's renderer.

I don't know in what case Opera actually uses the WebView instead of just loading it, so I can't repro it crashing without my change, but this seems likely to be the cause nonetheless. I'm going to remove releaseblock-stable since it only happens with multiprocess enabled and probably doesn't affect any other application (I don't think any of the other apps that compile in the chromium code also use webview).

So, unless Opera can explain something to the contrary, this looks like an app bug: Opera assumes it's safe to have two complete copies of chromium running in the same application (webview and their own build), and it simply isn't, because while they're loaded into separate classloaders and thus the classes themselves do not clash, there are other things that are global and do clash - this was previously noted for the pak file assets in  issue 458962  and Opera had to rename their own locale paks to not conflict with webview's. If they did the same for the sandboxed process service names then this issue would be avoided.

I've added the opera developers who were cc'ed on  issue 458962  in the hope that they can get us in touch with the right person to discuss this.

Also, Robert: why did we end up going with this particular way of constructing the service name for external services, which can clash with the app?

Thanks for adding me to the cc list. I will check what we use webview for and if we can remove the use of it.

We have looked through our code and can confirm that the actual app uses webview in a few places. And we will start to remove the dependency on webview asap.

Thanks again for letting us know about this issue.

OK, you're able to remove the usage of webview? If so then that's the simplest way to resolve this issue - there won't be any conflicts if you don't load webview, because none of the conflicting things are defined in your process until the first time WebView APIs are touched.

Let me know if you find any issues with this, but I'll close this bug for now. Multiprocess is only a developer option in the N release, so this won't impact any of your users unless they have turned that developer option on. We hope to make it enabled by default in a future android release, however.

> Also, Robert: why did we end up going with this particular way of constructing the service name for external services, which can clash with the app?

We didn't foresee this case of someone using webview and including all Chromium's renderer services themselves, with the webview package prefix. Fundamentally collisions I think are always a possibility given that it's just a string.