New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 606629 link

Starred by 32 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: May 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug-Regression



Sign in to add a comment

err_ssl_bad_record_mac_alert on all sites on chrome android

Reported by 613...@gmail.com, Apr 26 2016

Issue description

Example URL:
google.com

Steps to reproduce the problem:
1. Open google chrome app on android
2. go to any website

What is the expected behavior?
Website should load

What went wrong?
Chrome says "err_ssl_bad_record_mac_alert". This happens even on websites that don't support ssl, and even on google's website.

Did this work before? Yes on previous version. Chrome updated today, so it must be related to the update.

Chrome version: 50.0.2661.89  Channel: stable
OS Version: 6
Flash Version: 

Websites will sometimes work on refresh. I don't know why it sometimes works and sometimes fails.
 
Screenshot_20160425-195921.png
54.9 KB View Download
Screenshot_20160425-195848.png
85.7 KB View Download
Components: -Internals>Network Internals>Network>SSL
Labels: Needs-Feedback
Please attach a net log per these instructions. Thanks!
https://dev.chromium.org/for-testers/providing-network-details

Comment 2 by 613...@gmail.com, Apr 27 2016

I went to Google.com which reproduced it. First it just had a blank screen, then on refresh it gave the error.
net_internals_log
567 KB View Download
Project Member

Comment 3 by sheriffbot@chromium.org, Apr 27 2016

Labels: -Needs-Feedback Needs-Review
Owner: davidben@chromium.org
Thank you for providing more feedback. Adding requester "davidben@chromium.org" for another review and adding "Needs-Review" label for tracking.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Hrm. Does this happen on just Google sites or also others? I believe you're using a cipher suite that only Google sites would use. I don't think we changed anything relating to it between 49 and 50 though. I wonder if it actually coincides with a server-side change. I'll look into that internally.

Comment 5 by 613...@gmail.com, Apr 27 2016

This happened on every site. I'll get a network dump of other sites, with and without ssl, soon.
Hrm. Without SSL is especially odd. Yeah, if you could get me dumps of those too, that would be great!

Comment 7 by 613...@gmail.com, Apr 27 2016

Not all requests are generating the error, even for the same site.

I captured it on https://news.ycombinator.com/, then after a refresh, went through a couple links, and the third one there, http://gafferongames.com/2016/04/25/never-trust-the-client/, doesn't use ssl but failed (it worked after a refresh which should also have been captured). The rust one in the middle worked and can be ignored.

The problem definitely appeared the same day as the Chrome update, but I don't know for sure that's what caused it.
net_internals_log
2.0 MB View Download
Labels: -Needs-Review Needs-Feedback
Thanks! Yeah, that rules out anything on the server. That sites without SSL are affected are I believe because of Data Reduction Proxy, so all your connections go through SSL.

We didn't change anything about ChaCha20-Poly1305 (the cipher you'd be using against Google sites, the DRP, and https://news.ycombinator.com---I forgot Cloudflare sites would also select that one) between 49 and 50 though. Between 48 and 49 there was a significant change.

Were you on an up-to-date Chrome previously?

One thing that did change is 50 assumed NEON support. But that still doesn't quite fit. Surely a device new enough to be running Android 6.0.1 would have NEON and properly report support for it...

What kind of device do you have?

Comment 9 by 613...@gmail.com, Apr 27 2016

Galaxy s3 on cyanogenmod 13. I'm testing without DRP now, all http sites seem to work. I still get errors on some https sites; I'll see if I can find any non cloudflare ones that fail.

I'm pretty sure I was up to date before, I check updates in Play regularly.

Comment 10 by 613...@gmail.com, Apr 27 2016

I downgraded to 49 and the problem went away, then I upgraded to 50 again and it showed up again. It's definitely in the 49-50 change.
Status: Assigned (was: Unconfirmed)
Oh no. It's THAT phone. Okay. :-(

This probably the same CPU with the buggy NEON unit as  issue #469511 . That one was also a Galaxy S3. And it seems I managed to regress the logic that in https://codereview.chromium.org/1730823002. I'll go sort that out. Sorry about that. :-(

To confirm you do in fact have the same CPU, are you able to run adb against your phone? If so, could you run adb shell cat /proc/cpuinfo and paste the result here? Thanks!

Coincidentally, M50 is also when we started requiring NEON and told the compiler to freely emit NEON code. I believe this is unrelated to this regression, but there's probably a similar regression on trunk that I need to go resolve.

+pasko for NEON hilarity, since the state of the world is pretty ridiculous. My guess is the hand-written stuff is better at using the pipeline than the compiler-emitted stuff. But I'm sure we'll hit compiler-emitted NEON code that tickles the bug someday...

Comment 12 by 613...@gmail.com, Apr 27 2016

Processor	: ARMv7 Processor rev 0 (v7l)
processor	: 0
BogoMIPS	: 13.50

processor	: 1
BogoMIPS	: 13.50

Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 
CPU implementer	: 0x51
CPU architecture: 7
CPU variant	: 0x1
CPU part	: 0x04d
CPU revision	: 0

Hardware	: SAMSUNG M2
Revision	: 000e

I left out the serial in the last line.
Labels: -Needs-Feedback
Yup. You've got the same CPU alright. That matches  https://crbug.com/341598#c33 . (Sorry, I linked to the wrong bug previously. The first bug is  issue #341598 .  Issue #469511  was the first time this regressed. Now it's the second.)

I'll get this fixed certainly for 51 and 52. I'll also try to merge a fix into 50, but I don't know whether there will be another release of 50. We'll see what the release folks want to do there.
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 27 2016

The following revision refers to this bug:
  https://boringssl.googlesource.com/boringssl.git/+/2b4820bd523c7ee7406537bfad1bde9bb29673bb

commit 2b4820bd523c7ee7406537bfad1bde9bb29673bb
Author: David Benjamin <davidben@google.com>
Date: Wed Apr 27 22:27:11 2016

Don't set a default armcap state in dynamic armcap modes.

The getauxval (and friends) code would be filling that in anyway. The default
only serves to enable NEON even if the OS is old enough to be missing getauxval
(and everything else).

Notably, this unbreaks the has_buggy_neon code when __ARM_NEON__ is set, as is
the case in Chrome for Android, as of M50.  Before, the default
OPENSSL_armcap_P value was getting in the way.

Arguably, this doesn't make a whole lot of sense. We're saying we'll let the
CPU run compiler-generated NEON code, but not our hand-crafted stuff. But, so
far, we only have evidence of the hand-written NEON tickling the bug and not
the compiler-generated stuff, so avoid the unintentional regression. (Naively,
I would expect the hand-crafted NEON is better at making full use of the
pipeline and is thus more likely to tickle the CPU bug.)

This is not the fix for M50, as in the associated Chromium bug, but it will fix
master and M51. M50 will instead want to revert
https://codereview.chromium.org/1730823002.

BUG= chromium:606629 

Change-Id: I394f97fea2f09891dd8fa30e0ec6fc6b1adfab7a
Reviewed-on: https://boringssl-review.googlesource.com/7794
Reviewed-by: Adam Langley <agl@google.com>

[modify] https://crrev.com/2b4820bd523c7ee7406537bfad1bde9bb29673bb/crypto/crypto.c

Sorry, new to this.
Same problem with a Sony Xperia T (LT30p, Android 4.3) after same Chrome update.
S3 owner here facing the same issue.

I also noticed the Google app where now cards and what not appears, is also having issues. It says "Google Now can't be reached at the moment". If I try and force it to connect it says "network connection error please try again later"

Screenshot attached
Screenshot_2016-04-28-10-13-42.png
73.2 KB View Download

Comment 17 by 613...@gmail.com, Apr 28 2016

I actually had the same issue with the Google app, but the latest update fixed it for me.


oscar.ter.meer: Hrm. Having it in that phone too is new to me. Do you mind running the same adb command as in comment #13?
Cc: pasko@chromium.org
(Apparently I forgot to actually CC pasko earlier.)
613ike, thetruewebster: Could you both also check which S3 you have (there's a ton of them)? I believe it's under settings -> About phone. Our previous report of this problem was on a SGH-I747M. (Unfortunately, we also know that not every SGH-I747M has the buggy CPU, but at least it gives an upper-bound.) Thanks!
If it's not there, pasko@ tells me you can also get it out of adb via:

  adb shell getprop ro.product.model

Comment 22 by 613...@gmail.com, Apr 28 2016

Mine is SPH-L710, also known as d2spr.
Sorry, kind of a noob. Don't know how to use ADB cpuinfo. I think I need the SDK for that, right?
I can provide details from apps like CPU-Z:
CPU: Qualcomm Snapdragon S4
Architecture: Krait
Revision: r1p0

System Kernel Architecture: arm7vl
System Kernel Version: 3.4.0-perf-ga3976ea (Mvv_tg)

What are you looking for?

By the way: the problem is NOT persistent. Problem occurs after fresh reboot. After entering URL's or using favourits after a while the 'address bar' can be used to search.
S3 Model: SCH-I535 
(This is the North America Verizon variant)

Comment 25 by pasko@chromium.org, Apr 28 2016

oscar.ter.meer: your device should manifest itself slightly differently. Can you share netlog and screenshots?
Project Member

Comment 26 by bugdroid1@chromium.org, Apr 28 2016

The following revision refers to this bug:
  https://boringssl.googlesource.com/boringssl.git/+/0a63b96535dff86fc226e3a13e34252e702a45d0

commit 0a63b96535dff86fc226e3a13e34252e702a45d0
Author: David Benjamin <davidben@google.com>
Date: Thu Apr 28 16:17:55 2016

Make CRYPTO_is_NEON_capable aware of the buggy CPU.

If we're to allow the buggy CPU workaround to fire when __ARM_NEON__ is set,
CRYPTO_is_NEON_capable also needs to be aware of it. Also add an API to export
this value out of BoringSSL, so we can get some metrics on how prevalent this
chip is.

BUG= chromium:606629 

Change-Id: I97d65a47a6130689098b32ce45a8c57c468aa405
Reviewed-on: https://boringssl-review.googlesource.com/7796
Reviewed-by: Adam Langley <agl@google.com>

[modify] https://crrev.com/0a63b96535dff86fc226e3a13e34252e702a45d0/crypto/cpu-arm-linux.c
[modify] https://crrev.com/0a63b96535dff86fc226e3a13e34252e702a45d0/include/openssl/cpu.h

Comment 27 by jarm...@gmail.com, Apr 28 2016

My S3 Model Number is SGH-T999. Running CyanogenMod 11-20150831. I've been experiencing the same issues starting two days ago. Some websites would work if I click reload 6-7 times, but some wouldn't. Any idea when this will be fixed? I might have to dl another browser if it's not soon. 
Labels: -Type-Bug Merge-Request-51 Merge-Request-50 M-50 Type-Bug-Regression
Hey TPMs,

This is kind of complicated. So there's a particular revision of a particular CPU (thus far confirmed to be on some, but not all Samsung Galaxy S3s) that's buggy and has a broken NEON chip. For two different reasons, our workaround for this chip has regressed in both M50 and M51.

I would like to request:

1. For M51, we merge *both* https://boringssl.googlesource.com/boringssl.git/+/2b4820bd523c7ee7406537bfad1bde9bb29673bb and https://boringssl.googlesource.com/boringssl.git/+/0a63b96535dff86fc226e3a13e34252e702a45d0. This is in BoringSSL, so it will require some fiddling with the internal DEPS. M51 has not hit stable and the changes only just rolled onto Chromium trunk (but we don't have much of an Android canary process anyway), so we can wait a few days for that without trouble.

2. For M50, I would like to *revert* https://codereview.chromium.org/1730823002.

The impact is that affected devices have trouble connecting to sites using ChaCha20-Poly1305 cipher suites. This includes Google properties and sites using Cloudflare.

It is just one chip, but I don't have a good estimate of the impact (the last couple of times this regressed, one of our users with the device noticed it while it was on beta, not stable. I guess that user finally got a new phone! :) ). I'll be adding metrics in a follow-up so we can get a sense of how prevalent this buggy chip is in the future. I'll see what kind of gymnastics I can do with our existing metrics to get a rough estimate.

Comment 29 by tin...@google.com, Apr 28 2016

Labels: -Merge-Request-50 Merge-Review-50 Hotlist-Merge-Review
[Automated comment] Request affecting a post-stable build (M50), manual review required.

Comment 30 by tin...@google.com, Apr 28 2016

Labels: -Merge-Request-51 Merge-Approved-51 Hotlist-Merge-Approved
Your change meets the bar and is auto-approved for M51 (branch: 2704)
pasko: Actually I think the problematic chipset is a Qualcomm one. So oscar.ter.meer's phone may be the same issue. Do get us a net-internals to be sure though.
Same problem for me with Chrome 50.0.2661.89 with Samsung GS3 (SAMSUNG-SGH-I747) purchased from Telus, running CM 12.1-20151117-SNAPSHOT-Y0G7DA01K6-d2att (5.1.1)
I can provide a net-internals file of the following case:
- started Chrome
- opened a tab starting net-internals (setting 'Export')
- opened a second tab and entered 'net-internals' in the adress/search box
- this generated the error
- the error page automatically reloaded after a couple of seconds
- after reload search results were shown

Again, I'm not a developer so some guidance is appreciated.
Project Member

Comment 34 by bugdroid1@chromium.org, Apr 29 2016

Labels: -merge-approved-51 merge-merged-2704
The following revision refers to this bug:
  http://goto.ext.google.com/viewvc/chrome-internal?view=rev&revision=87228

------------------------------------------------------------------
r87228 | davidben@google.com | 2016-04-29T18:35:59.314196Z

-----------------------------------------------------------------
oscar.ter.meer: Yeah, that sounds right. But if you go to chrome://net-export instead, you'll get something a bit more phone-friendly.
Visited chrome://net-export on my Samsung GS3 running CM 12.1 (more details in my last post) which generated the attached log file. Not sure if it's needed/wanted, but figured a developer may find the log useful...
572.json
89.2 KB View Download
Thanks! Yeah, if you have a Galaxy S III, it's likely this bug. And this looks similar. oscar.ter.meer's phone is the only confusing one since this is the first I've heard of a different device with this problem. But I could believe other manufacturers used that buggy CPU.
The net-internals I mentioned earlier.

Again, the problem is not persistent. I can't figure out what causes the problem. Sometimes the error message is not shown but instead an empty search result page.
Or another message: ERR_SSL_PROTOCOL_ERROR
net-internals-log[1].json
0 bytes View Download
Trying again to attach file...

About problem: reloading the page most of the time solves it.
net-internals-log.json
273 KB View Download
Is there anyway to disable support for ChaCha20-Poly1305 to workaround the problem until this is fixed?

My Samsung Galaxy S3 SAMSUNG-SGH-I747 Android 4.4.2 is affected by this.

Comment 41 by 613...@gmail.com, May 2 2016

If you really want to you can downgrade to 49 from https://www.apkmirror.com/apk/google-inc/chrome/chrome-49-0-2623-105-release/

Then change Play settings to not auto-update Chrome until the next version comes out.

Or install a chromium recent build that fixes it, go to https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Android/, scroll to the bottom, and download the latest one (if it doesn't work, try older ones). 

Comment 42 by mat...@gmail.com, May 2 2016

Another log from a Galaxy I-747 AT&T edition (Qualcomm chipset) running CM13 snapshots. Seems to be this bug and might be a useful point of reference.

chrome-net-export-log.json
183 KB View Download
Folks, if you have a Samsung Galaxy S3 and are seeing ERR_SSL_BAD_RECORD_MAC_ALERT, there is no need for logs or "me too" comments. Please hit the star in the top-left corner of the page to be notified of updates.

The only thing of some vague interest now is whether there are other phone models with the buggy chip. (Though, either way, the fix is already in for 51 and 52. The question is just whether we'd spin another release of 50.)

Comment 44 by k...@google.com, May 2 2016

Labels: -Merge-Review-50 ReleaseBlock-Stable Merge-Approved-50
Project Member

Comment 45 by bugdroid1@chromium.org, May 2 2016

Labels: -merge-approved-50 merge-merged-2661
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a4ab48f3dac253ee97295ad2fb17a9fd6cda15ea

commit a4ab48f3dac253ee97295ad2fb17a9fd6cda15ea
Author: David Benjamin <davidben@chromium.org>
Date: Mon May 02 21:25:45 2016

Revert "Disable all NEON in BoringSSL if has_broken_neon is set."

This reverts commit 396d38c7d55e1c039069017afd74488cac9060cd.

The change caused the has_broken_neon to regress because, at the corresponding
BoringSSL revision, the getauxval logic would run afterwards and clear the
setting.

For M50, revert this change to restore the old behavior. Conveniently, the
BoringSSL half of this pair (85003903fc58d8825e02162fd33a9b9c28fdec35) rolled
into Chromium after the branch point and was later subsumed by moving the check
into BoringSSL itself, so trunk and M51 have the correct broken_neon/getauxval
interaction. (Though they still regressed the check due to an interation with
Chromium setting arm_neon=1. This has since been fixed.)

BUG= 606629 
TBR=agl@chromium.org

Review URL: https://codereview.chromium.org/1943053002 .

Cr-Commit-Position: refs/branch-heads/2661@{#655}
Cr-Branched-From: ef6f6ae5e4c96622286b563658d5cd62a6cf1197-refs/heads/master@{#378081}

[modify] https://crrev.com/a4ab48f3dac253ee97295ad2fb17a9fd6cda15ea/crypto/openssl_util.cc

same problem upon upgrade to M50 in Motorola Droid Razr M system version 183.46.15.XT907.Verizon.en.US
Same problem with HTC One XL (evita) running Cyanogenmod 12.1 (Android 5.1.1) and version 50.0.2661.89 of Chrome
The fix for this is must now be in Chrome Beta v51.0.2704.36 as Google SSL sites are working again on this version.
Really? Huh. It didn't look like it was, based on https://chromium.googlesource.com/chromium/src/+/refs/tags/51.0.2704.36/DEPS

It should definitely be in there when 51 is stable though.

Comment 50 by 613...@gmail.com, May 5 2016

Can confirm it's fixed in 51.0.2704.36.
kerz@ has told me that link is wrong and we shipped a beta that did include the fixes earlier in the thread. So now I am less confused and yes, 51.0.2704.36 is supposed to contain the fix. Thanks for confirming that's resolved!
Hello, 

I'm running into this bug on an HTC One S (Ville) running CyanogenMod 12.1-20150901

Comment 53 by hap...@gmail.com, May 25 2016

I don't think this is Android specific.

I've been having this same problem for a while (I think after Chrome 49 or Chrome 50) on OS X El Capitan (10.11.4). It seems that I only get the problem when trying to upload files (for example trying to use Google Images search, or when trying to upload an image to a CMS).

Currently happening on Chrome version 50.0.2661.102 (64-bit).

Sometimes randomly upload works without problem, but usually not. I have no problems uploading the same files using Safari for example.

Comment 54 by hap...@gmail.com, May 25 2016

Just noticed that some times I get ERR_CONNECTION_RESET instead of ERR_SSL_BAD_RECORD_MAC_ALERT. And I also noticed that there is no problem if I connect using a different WiFi.

What I don't get is why other browsers like Safari work fine on both WiFis but Chrome has an issue with the other one.
Cc: davidben@chromium.org
Owner: kerz@chromium.org
haprog: If you're seeing this on Mac, it's completely different issue. Please file a *separate* bug and attach a net-internals log per these instructions. Thanks!
https://dev.chromium.org/for-testers/providing-network-details

Reassigning this to kerz@ since, codewise, this is all sorted out. Just the question of whether we wanted to respin M50 or just wait for M51. (Given how late it is in the cycle, I'm guessing we decided to just wait for M51? Shall we close this bug?)

Comment 56 by k...@google.com, May 26 2016

We're going to wait for 51 for this now, so we can close it out.
Labels: -M-50 M-51
Status: Fixed (was: Assigned)
Alright, closing this then.

Sign in to add a comment