New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 771016 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Timeout waiting for GATT connection to be created

Project Member Reported by khorimoto@chromium.org, Oct 3 2017

Issue description

We frequently see a timeout creating a GATT connection during connection attempts.

When GATT connections are created successfully, it usually takes about 2-4 seconds. Currently, our timeout is 15 seconds (see [1]), so this should be far more than enough.

When this fails, you'll see an error log:
"Timed out waiting during SubStatus [waiting for GATT connection to be created]. Destroying connection."

[1] https://cs.chromium.org/chromium/src/components/cryptauth/ble/bluetooth_low_energy_weave_client_connection.cc?q=kGattConnectionTimeoutSeconds
 
Kyle, is this a problem that is seen only after applying Joseph's 3 CL fix for register advertisement?
Yes - it seems to be unrelated to advertising.
Labels: -M-63 M-62
We should try to get this into M-62. I'm seeing it fairly often.
timeout_gatt.zip
5.4 MB Download
Owner: sonnysasaka@chromium.org
Status: Started (was: Available)
I investigated this and found out that there is some kind of race condition in bluetoothd, where if Connect() is being called but the kernel (for some mysterious reason) issues a DEVICE_DISCONNECTED event, bluetoothd would forget to reply the client and the client (Chrome) may reach a bad state so even retrying the connection would always fail again.

I have a proposed fix at http://crosreview.com/711374. This will at least make bluetoothd reply that the connection failed rather than doing nothing and timeout. The occasional connection error will still be there, but at least it won't timeout and reach a bad state so the retry will likely be successful.


Comment 6 by snanda@chromium.org, Oct 11 2017

Logs from the repro.
chrome.chronos
3.4 MB Download
chrome
11.6 KB View Download
ui.LATEST
38.7 KB Download
messages
1.5 MB View Download

Comment 7 by snanda@chromium.org, Oct 11 2017

Android bugreport & dmesg (from Chromebook) attached.
dmesg.log
244 KB View Download
bugreport-sailfish-OPM1.171009.001-2017-10-10-18-50-06.zip
9.7 MB Download

Comment 8 by snanda@chromium.org, Oct 11 2017

btsnoop log to go with c#6 & c#7.
btsnoop.zip
2.9 MB Download
Sonny wrote up a CL which he says fixes this bug:

https://chromium-review.googlesource.com/c/chromiumos/third_party/bluez/+/711374
comment 6, 7, and 8 are actually meant for issue 773062.
Attaching logs for an isolated repro (without tether) of this case.
disconnect_early.tar.gz
171 KB Download
Project Member

Comment 12 by bugdroid1@chromium.org, Oct 16 2017

Labels: merge-merged-chromeos-5.44
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/bluez/+/9fca8b3288156fb64cbcdd7a8ad56cd5c50a3926

commit 9fca8b3288156fb64cbcdd7a8ad56cd5c50a3926
Author: Sonny Sasaka <sonnysasaka@chromium.org>
Date: Mon Oct 16 21:14:02 2017

CHROMIUM: Fix race condition between device connection and disconnection

When Connect() is called and waiting for return, dev_disconnected may be
called due to MGMT_EV_DEVICE_DISCONNECTED event from kernel. In that
case reply to client that the connection failed otherwise the dbus
method will timeout because bluetoothd never replies.

BUG= chromium:771016 
TEST=Simulate a lot of Connect() to bluetooth devices and check that
error is returned from bluetoothd rather than dbus timeout when this
race condition happens.

Change-Id: Ica16f54b793986cd17cb450a9a1d15935d6f5386
Reviewed-on: https://chromium-review.googlesource.com/711374
Commit-Ready: Sonny Sasaka <sonnysasaka@chromium.org>
Tested-by: Sonny Sasaka <sonnysasaka@chromium.org>
Reviewed-by: Rahul Chaturvedi <rkc@chromium.org>
Reviewed-by: Miao-chen Chou <mcchou@chromium.org>

[modify] https://crrev.com/9fca8b3288156fb64cbcdd7a8ad56cd5c50a3926/src/device.c

Labels: Merge-Request-63 Merge-Request-62
Project Member

Comment 14 by sheriffbot@chromium.org, Oct 16 2017

Labels: -Merge-Request-62 Merge-Review-62 Hotlist-Merge-Review
This bug requires manual review: We are only 0 days from stable.
Please contact the milestone owner if you have questions.
Owners: amineer@(Android), cmasso@(iOS), bhthompson@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Review-62 Merge-Approved-62
Consider this merge approved for 62 once it has been verified on a ToT canary build and merged into 63. 
Project Member

Comment 16 by sheriffbot@chromium.org, Oct 17 2017

Labels: -Merge-Request-63 Hotlist-Merge-Approved Merge-Approved-63
Your change meets the bar and is auto-approved for M63. Please go ahead and merge the CL to branch 3239 manually. Please contact milestone owner if you have questions.
Owners: cmasso@(Android), cmasso@(iOS), gkihumba@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 17 by bugdroid1@chromium.org, Oct 19 2017

Labels: merge-merged-release-R63-10032.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/bluez/+/9fbe131ad414fb23672f29379047d59d387a9267

commit 9fbe131ad414fb23672f29379047d59d387a9267
Author: Sonny Sasaka <sonnysasaka@chromium.org>
Date: Thu Oct 19 17:38:42 2017

CHROMIUM: Fix race condition between device connection and disconnection

When Connect() is called and waiting for return, dev_disconnected may be
called due to MGMT_EV_DEVICE_DISCONNECTED event from kernel. In that
case reply to client that the connection failed otherwise the dbus
method will timeout because bluetoothd never replies.

BUG= chromium:771016 
TEST=Simulate a lot of Connect() to bluetooth devices and check that
error is returned from bluetoothd rather than dbus timeout when this
race condition happens.

Change-Id: Ica16f54b793986cd17cb450a9a1d15935d6f5386
Reviewed-on: https://chromium-review.googlesource.com/711374
Commit-Ready: Sonny Sasaka <sonnysasaka@chromium.org>
Tested-by: Sonny Sasaka <sonnysasaka@chromium.org>
Reviewed-by: Rahul Chaturvedi <rkc@chromium.org>
Reviewed-by: Miao-chen Chou <mcchou@chromium.org>
(cherry picked from commit 9fca8b3288156fb64cbcdd7a8ad56cd5c50a3926)
Reviewed-on: https://chromium-review.googlesource.com/724639
Trybot-Ready: Miao-chen Chou <mcchou@chromium.org>
Commit-Queue: Miao-chen Chou <mcchou@chromium.org>

[modify] https://crrev.com/9fbe131ad414fb23672f29379047d59d387a9267/src/device.c

Project Member

Comment 18 by sheriffbot@chromium.org, Oct 23 2017

Cc: bhthompson@google.com
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 19 by bugdroid1@chromium.org, Oct 24 2017

Labels: merge-merged-release-R62-9901.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/bluez/+/b6a3b370d7057f67677ff5d1b2876c5352d3a01a

commit b6a3b370d7057f67677ff5d1b2876c5352d3a01a
Author: Sonny Sasaka <sonnysasaka@chromium.org>
Date: Tue Oct 24 17:55:08 2017

CHROMIUM: Fix race condition between device connection and disconnection

When Connect() is called and waiting for return, dev_disconnected may be
called due to MGMT_EV_DEVICE_DISCONNECTED event from kernel. In that
case reply to client that the connection failed otherwise the dbus
method will timeout because bluetoothd never replies.

BUG= chromium:771016 
TEST=Simulate a lot of Connect() to bluetooth devices and check that
error is returned from bluetoothd rather than dbus timeout when this
race condition happens.

Change-Id: Ica16f54b793986cd17cb450a9a1d15935d6f5386
Reviewed-on: https://chromium-review.googlesource.com/711374
Commit-Ready: Sonny Sasaka <sonnysasaka@chromium.org>
Tested-by: Sonny Sasaka <sonnysasaka@chromium.org>
Reviewed-by: Rahul Chaturvedi <rkc@chromium.org>
Reviewed-by: Miao-chen Chou <mcchou@chromium.org>
(cherry picked from commit 9fca8b3288156fb64cbcdd7a8ad56cd5c50a3926)
Reviewed-on: https://chromium-review.googlesource.com/733940
Reviewed-by: Sonny Sasaka <sonnysasaka@chromium.org>
Commit-Queue: Sonny Sasaka <sonnysasaka@chromium.org>

[modify] https://crrev.com/b6a3b370d7057f67677ff5d1b2876c5352d3a01a/src/device.c

Status: Fixed (was: Started)
Project Member

Comment 21 by sheriffbot@chromium.org, Oct 27 2017

This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Verified (was: Fixed)

Sign in to add a comment