Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 1 user
Status: Fixed
Owner:
Closed: Sep 20
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment
gale paladin failed due to error 'Bluetooth interface is up but not discoverable' for jetstream_BluetoothBeaconing test
Project Member Reported by shuqianz@chromium.org, Aug 3 Back to list
Build link:
https://uberchromegw.corp.google.com/i/chromeos/builders/gale-paladin/builds/3421

error log:
()
07/29 05:58:44.467 DEBUG|             utils:0212| Running 'hciconfig'
07/29 05:58:44.499 DEBUG|              test:0389| Test failed due to Bluetooth interface is up but not discoverable. Exception log follows the after_iteration_hooks.
07/29 05:58:44.503 DEBUG|              test:0392| starting after_iteration_hooks
07/29 05:58:44.506 WARNI|             utils:0174| Failure: Bluetooth interface is up but not discoverable
07/29 05:58:44.528 INFO |             utils:0179| ---- Test finished (jetstream_BluetoothBeaconing) ----
07/29 05:58:44.534 DEBUG|             utils:0212| Running 'if test -d /var/spool/crash/; then find /var/spool/crash/ -name "*.dmp"; fi'
07/29 05:58:44.570 DEBUG|             utils:0212| Running 'mkdir -p /usr/local/autotest/results/default/jetstream_BluetoothBeaconing/sysinfo/iteration.1/var/spool'
07/29 05:58:44.604 DEBUG|             utils:0212| Running 'rsync --no-perms --chmod=ugo+r -a --safe-links --exclude=/crash/**autoserv* --exclude=*.core /var/spool/crash /usr/local/autotest/results/default/jetstream_BluetoothBeaconing/sysinfo/iteration.1/var/spool'
07/29 05:58:44.652 DEBUG|             utils:0212| Running 'rm -rf /var/spool/crash/*'
07/29 05:58:44.699 DEBUG|             utils:0212| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/jetstream_BluetoothBeaconing/sysinfo/iteration.1"'
07/29 05:58:44.733 DEBUG|              test:0395| after_iteration_hooks completed
07/29 05:58:44.736 WARNI|              test:0612| The test failed with the following exception
Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 606, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/common_lib/test.py", line 806, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 470, in execute
    dargs)
  File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/common_lib/test.py", line 380, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/tests/jetstream_BluetoothBeaconing/jetstream_BluetoothBeaconing.py", line 17, in run_once
    utils.VerifyBlutoothUpAndDiscoverable(self.router)
  File "/usr/local/autotest/common_lib/cros/jetstream/utils.py", line 333, in VerifyBlutoothUpAndDiscoverable
    raise error.TestFail('Bluetooth interface is up but not discoverable')
TestFail: Bluetooth interface is up but not discoverable
07/29 05:58:44.765 DEBUG|              test:0617| Running cleanup for test.
07/29 05:58:47.885 DEBUG|             utils:0212| Running 'cat /proc/meminfo'
07/29 05:58:47.920 INFO |       base_router:0413| Configuring AP: --factory_reset
07/29 05:58:47.925 DEBUG|             utils:0212| Running 'ap-configure --factory_reset'

it seems like a test flake to me
 
Owner: danepollock@google.com
Dane, can you take a look of this? I find that you reviewed the CL to add this test.
Cc: lgoo...@chromium.org mcchou@chromium.org kohy@chromium.org
This test is about 0.7% flaky in the jetstream lab (across all platforms), so it's highly unlikely this has flaked on both platforms by co-incidence.

It looks like ap-group-manager is having trouble talking to BlueZ over dbus:

2017-07-29T05:58:25.394177+00:00 WARNING ap-group-manager[8540]: [WARNING:object_manager.cc(369)] org.bluez /: Failed to get managed objects
2017-07-29T05:58:35.249814+00:00 ERR ap-group-manager[8540]: [ERROR:object_proxy.cc(582)] Failed to call method: org.bluez.GattManager1.RegisterApplication: object_path= /org/bluez/hci0: org.freedesktop.DBus.Error.ServiceUnknown: The name org.bluez was not provided by any .service files
2017-07-29T05:58:39.680604+00:00 INFO ap-group-manager[8540]: [INFO:ble_controller.cc(45)] Enable BLE beaconing
2017-07-29T05:58:40.171911+00:00 INFO ap-group-manager[8540]: [INFO:mesh_key_manager.cc(461)] Writing mesh ID and psk to device configuration
2017-07-29T05:58:44.418088+00:00 NOTICE root[9174]: autotest starting iteration /usr/local/autotest/results/default/jetstream_BluetoothBeaconing/sysinfo/iteration.1 on gale_0.7GHz_0GB
2017-07-29T05:58:44.460227+00:00 INFO autotest[8268]: ---- Test started (jetstream_BluetoothBeaconing) ----
2017-07-29T05:58:44.522061+00:00 NOTICE autotest[9180]: 05:58:44.506 WARNI|             utils:0174| Failure: Bluetooth interface is up but not discoverable
2017-07-29T05:58:44.527924+00:00 WARNING autotest[8268]: Failure: Bluetooth interface is up but not discoverable

This looks real, and this CL seems relevant:
https://chromium-review.googlesource.com/c/588381/5/src/advertising.c


132487373-chromeos-test%2Fchromeos6-row21-jetstream-host3%2Fjetstream_BluetoothBeaconing%2Fsysinfo%2Fvar%2Flog_diff%2Fmessages.txt
142 KB View Download
Owner: mcchou@chromium.org
Looks like bluetoothd isn't running, which would explain why BlueZ isn't doing much:
https://paste.googleplex.com/4643469918208000?raw
Agree, it is very likely that CL caused the failure. It was picked up by CQ twice. Gale paladin and the whirlwind paladin failed at this Jetstream bluethooth issue in both of the times. I will mark the CL verified -1.
I talked to the author of the crosreview.com/588381, and he just uploaded another patch set. Please keep an eye on the next the build.
This issue happened again:
https://luci-milo.appspot.com/buildbot/chromeos/whirlwind-paladin/9234

Can you take a look?
Cc: amstan@chromium.org puneetster@chromium.org
Labels: -Pri-2 Pri-1
This has caused three more whirlwind-paladin builds to fail.
https://luci-milo.appspot.com/buildbot/chromeos/whirlwind-paladin/9303
https://luci-milo.appspot.com/buildbot/chromeos/whirlwind-paladin/9305
https://luci-milo.appspot.com/buildbot/chromeos/whirlwind-paladin/9309

There was no change in common between those three builds, so it's likely an issue that has snuck into ToT.

Recommend move whirlwind-paladin to experimental via tree status and then disabling the test until the issue can be resolved.  Once the test is fixed, the build can be moved back to experimental.  There's also a common GuestInterfaces failure that potentially needs to be disabled as well.
The GuestInterfaces failure was not a test failure: it was a failure in provisioning prior to launching the test.

Project Member Comment 10 by bugdroid1@chromium.org, Sep 20
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/ap-daemons/+/ed2d0cc5e035e105123665a28b6a0556301f0cf8

commit ed2d0cc5e035e105123665a28b6a0556301f0cf8
Author: Laurence Goodby <lgoodby@google.com>
Date: Wed Sep 20 04:46:25 2017

Status: Fixed
The test has been removed for now.

Sign in to add a comment