This issue was reported in Android Version 7.0.0. Device info:
Model Number 
SM-G920V
 
Android Version 
7.0
 
Android security patch level 
August 1, 2017
 
Baseband version G920VVRS4DQH1
 
Kernel version 
3.10.61
dpi@SWDD6309 #1 Wed Aug 9 14:52:34 KST 2017
 
Build Number
NRD90M.G920VVRS4DQH1
 
SE for Android status
Enforcing
SEPF_SECMOBILE_7.0_0009
Wed Aug 09 15:25:02 2017
 
Hardware version 
G920V.07
 
Security software version
MDF v3.0 Release 1
WLAN v1.0 Release 1
VPN v1.4 Release 8.1
ASKS v1.3_161228
 
Knox version
Knox 2.7.1
Standard SDK 5.7.1
Premium SDK 2.7.1
Customization SDK 2.7.1
Container 2.7.1
CEP 2.1.1
Enterprise Billing 1.2.0
OTP 2.6.0
SE for Android 2.4.3
SSO 2.5.0
Shared Device 2.6.0
TIMA 3.3.015
VPN 2.3.0
 
Chrome Details
Chrome 60.0.3112.107
 
Operating system
Android 7.0.0; SM-G920V Build/NRD90M

Deleted: sslerror.png 154 KB

	sslerror.png 154 KB View Download

Can you provide the log from chrome://net-internals please?

This is SSL, not auth, right?  At least relevant files are in net/ssl, and doesn't look like it's David, not Asanka, who's worked on them.

Yeah, this is SSL. Also adding Needs-Feedback label.

Oh, I missed this is WebView. I suspect we don't have a way to get a NetLog out of there. :-/

There might be something useful in the adb log? Could you attach that? Though I don't know if we currently log the exception or anything. We probably should...

Hello, I will ask the enterprise customer to provide the ADB log.

 

Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Uploaded ADB logs

Deleted: Android-GM-Repro-09192017-1.txt 2.7 MB

Android-GM-Repro-09192017-1.txt 2.7 MB View Download

Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Adding two more repro

-- Repro 2 --- 
09-21 17:50:05.607 18160 18561 E cr_AndroidKeyStore: Exception while signing message with RSA private key: java.security.InvalidKeyException: Supplied key (android.security.keystore.AndroidKeyStoreRSAPrivateKey) is not a RSAPrivateKey instance
09-21 17:50:05.607 18160 18561 W chromium: [WARNING:ssl_platform_key_android.cc(161)] Could not sign message with private key!
09-21 17:50:05.607 18160 18496 E chromium: [ERROR:ssl_client_socket_impl.cc(1072)] handshake failed; returned -1, SSL error code 1, net_error -141

--- Repro 3 ----
E/cr_AndroidKeyStore(30603): Exception while signing message with RSA private key: java.lang.UnsupportedOperationException: private exponent cannot be extracted

W/chromium(30603): [WARNING:ssl_platform_key_android.cc(161)] Could not sign message with private key!

E/chromium(30603): [ERROR:ssl_client_socket_impl.cc(1072)] handshake failed; returned -1, SSL error code 1, net_error -141

Deleted: ADB-Repro2-GM.txt 1.1 MB

ADB-Repro2-GM.txt 1.1 MB View Download

Deleted: ADB-Repro3-GM.txt 59.5 KB

ADB-Repro3-GM.txt 59.5 KB View Download

+flooey, klyubin, kroot

This error seems to be coming from BouncyCastle:
https://android.googlesource.com/platform/external/bouncycastle/+/nougat-release/bcprov/src/main/java/org/bouncycastle/jcajce/provider/asymmetric/rsa/DigestSignatureSpi.java#93

It's complaining that android.security.keystore.AndroidKeyStoreRSAPrivateKey isn't an RSAPrivateKey, which it indeed isn't.
https://android.googlesource.com/platform/frameworks/base/+/nougat-release/keystore/java/android/security/keystore/AndroidKeyStoreRSAPrivateKey.java

But I don't think that's the bug because RSAPrivateKey gives you access to the key material, which these keys wouldn't. I think the real issue is that the BouncyCastle provider is being used rather than whatever the native one is. This is too far into Java crypto provider insanity for me to diagnose. Thoughts?

This is the code Chromium is using for making a signature:
https://cs.chromium.org/chromium/src/net/android/java/src/org/chromium/net/AndroidKeyStore.java?rcl=f582672eaaa4201b3bed1cc402d22000c711da45&l=52

Sounds like http://www.bouncycastle.org/jira/browse/BJA-543. However, Android M, starting from which this issue become a problem, contains a workaround. The workaround is there also in all newer versions of Android. The workaround is that "AndroidKeyStoreBCWorkaroundProvider", the JCA Provider which offers crypto algorithms which handle Android Keystore-backed keys, is installed at higher priority than the Bouncy Castle JCA Provider.

Thus, Chromium's code referenced in comment 12, should definitely result in "NONEwithRSA" Signature impl being requested from the AndroidKeyStoreBCWorkaroundProvider and then attempted to be initialized with that private key. If initialization of that Signature impl fails in Signature.initSign, then the JCA framework will see if a lower priority provider offers a NONEwithRSA impl, then create and initialize that impl and so forth. It might be that this Android Keystore key is not authorized to be used with digest "NONE" (such authorizations were introduced in Android M to Android Keystore). In that case, an attempt to have Bouncy Castle handle this key might occur.

If you have a device on which this issue reproduces then:
* Check whether there is any interesting spewing, just prior to the InvalidKeyException, to do with keystore/keymaster/QSEE saying the key isn't authorized or that an operation failed.
* Check whether the "AndroidKeyStoreBCWorkaroundProvider" provider is installed at higher priority than the "BC" provider.
* Check whether Signature.getInstance("NONEwithRSA", "AndroidKeyStoreBCWorkaroundProvider").initSign(...) works with this key -- this forces the use of just the AndroidKeyStoreBCWorkaroundProvider provider.

To confirm: does this client certificate work in Chrome? If so, that'd suggest there's something different going on in WebView, though it's the same signing code. klyubin, is it plausible app using WebView is causing Bouncy Castle to be loaded while, whatever Chrome's doing, it isn't?

Also, what ADB-Repro2-GM.txt and ADB-Repro3-GM.txt show very different failure modes. Could you say a bit about what the difference between those are? Were they captured on different devices? What kinds of devices were they?

> Check whether there is any interesting spewing, just prior to the InvalidKeyException, to do with keystore/keymaster/QSEE saying the key isn't authorized or that an operation failed.

(Comment #11 already has the full adb logs. I don't see any obvious other spewing.)

(Re NONEwithRSA, TLS 1.0 and 1.1 require using NONEwithRSA. We've always used that one, and Conscrypt does the moral equivalent internally too. In theory we could use SHA*withRSA for TLS 1.2 now, but it'd take a lot of plumbing and differ from how Conscrypt does it and not work for TLS 1.0 and 1.1 anyway.)

Sorry, I missed the system logs already attached to this ticket.
* Repro 1 -- can't seem to find any mention of the issue
* Repro 2 -- Supplied key (android.security.keystore.AndroidKeyStoreRSAPrivateKey) is not a RSAPrivateKey instance -- this is likely from Bouncy Castle
* Repro 3 -- java.lang.UnsupportedOperationException: private exponent cannot be extracted -- this is likely from Conscrypt

It looks like the key is not app-private but rather coming from Android KeyChain, meaning it's available to any app, with user's approval. This means it should be possible to reproduce the issue using a test app, if we had possession of such a device.

In Repro 2 there's also debug logging enabled for keymaster. As a result, we have stronger grounds to believe that the key was authorized for use with NONEwithRSA Signature. This is an RSA key which was imported into keymaster (as opposed to generated inside). It's authorized for use with all digests, including digest "NONE". It's authorized for all purposes, including signing. It's authorized for all padding modes, including RSA PKCS #1 v1.5.

Can it be that this is some weird Samsung FIPS 140-2 -only mode, where there's no Android Keystore provider installed in the calling app?

Hello,
Wanted to point out that this happens randomly in a given device.  Some devices is in that state for a period of time and then starts to work

Please let us know of next steps

Best
Ramiro

Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Can you please give us an update on this issue ?

+cbrubaker

Android folks: any thoughts on how to proceed here?

My suggestion would be to obtain an Android device on which this reproduces and, likely by creating a small test app where this issue still reproduces, investigate the issue.

Can one of you all own this? Everything so far points to a bug in Android somewhere.

I do not have spare bandwidth to own this, sorry. I suggest escalating this to Samsung if this occurs only on Samsung devices.

Grace: Could you help own this, per Comment #16, Comment #23 and Comment #24? We believe this is an Android/Partner issue, not a Chrome issue.

Can someone confirm this only happens on Samsung device? Please provide device model and Android OS version where the problem exists. Also provide a reference device where the problem doesn't exist. Thanks.

(Setting Needs-Feedback label.)

Additionally, there was never any answer to the questions I asked in up comment #14.

Hello, I want to provide two more updates and some answers to comment 14.

1. Our customer is reporting that this issue is also happening in LG devices, so it is not a Samsung-only issue. We see most repro in Samsung because that's their "official" phone. 
2. This issue does NOT happen in Chrome browser. It only fails in the web view.
3. I uploaded yet another repro. Again, same root exception:
10-10 14:52:16.142 22438 22624 E cr_AndroidKeyStore: Exception while signing message with RSA private key: java.security.InvalidKeyException: Supplied key (android.security.keystore.AndroidKeyStoreRSAPrivateKey) is not a RSAPrivateKey instance
10-10 14:52:16.142 22438 22624 W chromium: [WARNING:ssl_platform_key_android.cc(159)] Could not sign message with private key!
10-10 14:52:16.143 22438 22500 E chromium: [ERROR:ssl_client_socket_impl.cc(1129)] handshake failed; returned -1, SSL error code 1, net_error -141

Also, what ADB-Repro2-GM.txt and ADB-Repro3-GM.txt show very different failure modes. Could you say a bit about what the difference between those are? Were they captured on different devices? What kinds of devices were they?
Answer: yes, they were different devices, as the one I just uploaded. This is a Samsung S6

Deleted: amira2.txt 542 KB

amira2.txt 542 KB View Download

Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

davidben@, is there any action we can take from the update from #28? If not could you ask for more info?

This is an Android issue deep in Android frameworks or something else on the device. It needs to be handled by the Android team. Assigning it to me will merely blackhole it as I do not have the knowledge to debug this. See existing discussion on this bug.

Reassigning to klobag@ as from comment #25.

Is it actually confirmed whether or not this is device-specific? Ideal would be to test it on a Google device (nexus/pixel) or on the Android emulator (using the same OS version and WebView version as the devices on which the problem reproduces), which all run unmodified framework code. This would confirm whether it's a general android bug.

Without knowing whether it reproduces on other devices it's hard to know where to route this to.

torne@, assign to you so that you can follow with the feedback.

Torne, Toby please can you look at this as a matter of emergency? Thanks!

I'll try to reproduce this locally.

Confirmed from separate email thread that this doesn't appear to be device specific at all and also repros on pixel.

Assigning to rubinxu@ for now while they try to get a repro case.

My speculation here is that some piece of process-global configuration in the Java crypto stack is being changed by other code in the client application, and these changes are affecting WebView (since WebView runs in the same process). The unreliable repro may be a result of nondeterministic initialisation order I guess?

If so then testing it with a "trivial" client application that just uses WebView and doesn't also do things with Java crypto itself is unlikely to produce a repro case, and we'd need to know a lot more about the app in question here. Anyone watching the bug from the reporter side: do you know if the application makes use of Java crypto APIs itself, or changes any configuration? (possibly as OS/device compatibility workarounds, or similar?)

Er, not assigning to rubinxu@ because you have to be a chromium project member to be the assignee. I'll keep it since I'm tracking the issue and following up with people.

..also putting back the folks I removed from cc by accident.

I have a test app with WebView trying to access TLS client cert protected site and so far am unable to reproduce the issue after around 2000 tries. This is consistent with our theory that the microsoft authenticator might be doing something that's affecting the global state of JCA which is shared by webview. I'm looking into that in more details.

Meanwhile, if we can get a test account that exercises the authenticator functionality the customer is using, it would be very useful.

Bug report from someone with the problem:

Build info: heroqltevzw-user 7.0 NRD90M G930VVRS4BQI1 release-keys
Serial number: 165090e1

Deleted: bugreport-2017-11-06-21-46-57.zip 7.9 MB

bugreport-2017-11-06-21-46-57.zip 7.9 MB Download

We really need a test account/config that can repro the issue.

Bugreport in #46 contains:

11-06 21:35:34.643  1313  1313 D TimaKeyStoreProvider: TimaSignature is unavailable
11-06 21:35:34.644  1313  1313 D ActivityThread: Added TimaKeyStore provider

early in the startup of the process, which I'm slightly suspicious of. This message appears to come from samsung framework code related to knox.. those strings aren't present in the app (which only has stub versions of things with names similar to those; guessing that samsung has real implementations in the framework that take precedence?)

Logs/bugreport from a non-samsung device reproducing the same issue would be interesting to rule this out as a possible cause..

OK, we've investigated further and it does look like the most likely cause is samsung-specific knox behaviour. In an email thread it was mentioned in passing that the issue has been observed on other devices, but all the logs/bugreports are from samsung devices.

If the same problem (the key type being incorrect for the provider) is visible in logs from other devices, please can you give us logs/bugreports from those devices to demonstrate that it's a generic issue?

Also: the app uses the samsung knox SDK. Is it possible to build a version of the app which doesn't make use of this for test purposes, to establish whether the SDK code is responsible for causing the issue?

I don't think we can make further progress on our end without being provided with a working test setup that enables us to reproduce the issue, and you may need to contact Samsung to ask how their knox SDK affects java crypto configuration..

ramirocld@, can you provide the info requested in comment #49?

We were able to reproduce this bug internally and traced it to the azure authenticator app. We are liaising with Microsoft at the moment, and no further information is needed in relation to the Knox SDK.

This was not related to Knox but to a crypto library used inside the MS apps. MS have confirmed they don't need any further support from us here at this time, so closing this out.