>BlueZ kernel should not return MGMT_STATUS_FAILED to mean max adv is reached, instead there should be MGMT_STATUS_ADV_MAX or something similar returned.

I failed to see why it would make sense to turn different errors into the same, AlreadyExists is used when the client attempts to register the same instance again, while the Failed may happen for other reasons including unavailability of instances. If you are saying that BlueZ should probably have a specific error to tell clients it the is no instances available, that I would agree since the client may be able to recover by reusing an existing instance it may have.

Yes, in this case having a more granular error codes is better than not being able to differentiate different errors. Your idea would solve the problem too.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/bluez/+/06870614a8af8c2594185f42d558586f8c5e2aa0

commit 06870614a8af8c2594185f42d558586f8c5e2aa0
Author: Sonny Sasaka <sonnysasaka@chromium.org>
Date: Wed Jul 19 23:27:04 2017

CHROMIUM: Add org.bluez.LEAdvertisingManager1.GetMaximumAdvertisementInstances

This adds GetMaximumAdvertisementInstances method to the
LEAdvertisingManager1 D-Bus interface. If advertisement is supported on
the local adapter, the method returns the maximum number of
advertisement instances, and not-supported error otherwise.

BUG=chromium:736487,chromium:741056
TEST=Call the DBus method using dbus-send, and observe that return value
is correct

Change-Id: I196a829d7c697f4c2ea72caf1a5ebd3977ef856e
Reviewed-on: https://chromium-review.googlesource.com/572761
Commit-Ready: Miao-chen Chou <mcchou@chromium.org>
Tested-by: Sonny Sasaka <sonnysasaka@chromium.org>
Reviewed-by: Miao-chen Chou <mcchou@chromium.org>

[modify] https://crrev.com/06870614a8af8c2594185f42d558586f8c5e2aa0/src/advertising.c
[modify] https://crrev.com/06870614a8af8c2594185f42d558586f8c5e2aa0/doc/advertising-api.txt

https://git.kernel.org/pub/scm/bluetooth/bluez.git/commit/?id=e4dae82fb742a3e568aaed2dfb65a123441d92ca

I wouldn't create a method to query the instances separately, instead just have a property called AvailableInstances, so GetAll and GetManagedObjects can fetch it quite quickly and if any application register a new instance it signals that it was taken.

Re #6,

If we add a property to the org.bluez.LEAdvertisingManager interface, this seems to be a patch which should go upstream?

Its there already:

https://git.kernel.org/pub/scm/bluetooth/bluez.git/commit/?id=85cb255b462da360d4712f8b9ff9b5ee398f7c1c

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/bluez/+/6e9e2970017a0e8beda3a0f5a51df3dc02039ac0

commit 6e9e2970017a0e8beda3a0f5a51df3dc02039ac0
Author: Sonny Sasaka <sonnysasaka@chromium.org>
Date: Tue Aug 01 00:08:28 2017

Revert "CHROMIUM: Add org.bluez.LEAdvertisingManager1.GetMaximumAdvertisementInstances"

This reverts commit 06870614a8af8c2594185f42d558586f8c5e2aa0.

Reason for revert: This should be implemented as D-Bus property, not method.

Original change's description:
> CHROMIUM: Add org.bluez.LEAdvertisingManager1.GetMaximumAdvertisementInstances
>
> This adds GetMaximumAdvertisementInstances method to the
> LEAdvertisingManager1 D-Bus interface. If advertisement is supported on
> the local adapter, the method returns the maximum number of
> advertisement instances, and not-supported error otherwise.
>
> BUG=chromium:736487,chromium:741056
> TEST=Call the DBus method using dbus-send, and observe that return value
> is correct
>
> Change-Id: I196a829d7c697f4c2ea72caf1a5ebd3977ef856e
> Reviewed-on: https://chromium-review.googlesource.com/572761
> Commit-Ready: Miao-chen Chou <mcchou@chromium.org>
> Tested-by: Sonny Sasaka <sonnysasaka@chromium.org>
> Reviewed-by: Miao-chen Chou <mcchou@chromium.org>

Bug: chromium:736487, chromium:741056
Change-Id: I5a8ae5b30bc8b19d239fbdd7a8a664486d4caa86
Reviewed-on: https://chromium-review.googlesource.com/592427
Commit-Ready: Sonny Sasaka <sonnysasaka@chromium.org>
Tested-by: Miao-chen Chou <mcchou@chromium.org>
Reviewed-by: Miao-chen Chou <mcchou@chromium.org>

[modify] https://crrev.com/6e9e2970017a0e8beda3a0f5a51df3dc02039ac0/src/advertising.c
[modify] https://crrev.com/6e9e2970017a0e8beda3a0f5a51df3dc02039ac0/doc/advertising-api.txt

This was previously targeted to M-61, but looks like it didn't make the cut. Sonny, is M-63 a reasonable target, or should this be moved to M-64?

Since this is only error reporting fix, 64 is fine.