BLE Application Framework

[AGlass0fMilk]

Starting this discussion now as I'm beginning a number of new BLE-enabled projects at the moment.

I have worked on several Mbed BLE applications over the past few years and I found myself repeatedly starting projects the same way.

It's usually:

1. Start with some example from mbed-os-example-ble
2. Combine one or several other examples from mbed-os-example-ble
3. Many times I make a huge monolithic class based on the BLEProcess class in mbed-os-ble-utils.
4. Make a separate thread to dispatch the additional BLE event queue
5. Dispatch other application events in the main thread

What I'd like to come up with is a framework that integrates the common code from the steps above. This helps save time when starting a new BLE project, encourages the use of best practices learned by experience, and helps avoid encountering inadvertent issues like ARMmbed/mbed-os#13916 (where the solution is a non-obvious consequence of the BLE stack's implementation).

So, I'd like to get everyone's ideas for the design goals of a theoretical BLE Application Framework.

A few of mine are:

• Compatibility with Baremetal (ie: does not explicitly depend on an RTOS/Threading)
• Designed to be Thread-Safe when Threads are available (eg: Use of PlatformMutex to provide synchronization)
• Comprehensive compile-time configurations -- decrease RAM and ROM usage by removing unneeded features
• Include provisions for system state control (eg: initialization, shutting down BLE/standby, ...)
• Combine the most commonly-used features of BLE:
  • pairing w/ non-volatile storage
  • privacy
  • OOB pairing (NFC OOB Pairing Utility/Example #24)
  • pairing with MITM protection
• Built-in configurable support for the major BLE operating modes:
  • Broadcaster
  • Observer
  • Central
  • Peripheral

We should also be working towards providing modular out-of-the-box support for commonly-used services (BLE UART Service for debug UART, DFU Service, ???)

I'm going to start working with the code in this repository as the ideas develop.

[AGlass0fMilk]

One way I see us accomplishing Baremetal compatibility is by using (as most BLE examples do now) the EventQueue API from Mbed.

It appears to be compatible with Baremetal. It can also take advantage of Threads when they are available.

The only con I see to this approach is the added code size from EventQueue and its supporting dependencies. I think it may be worth it in this case though. From a recent build with mbed-os master the --stats-depth=100 results showed the following code size for the events library:

About 1.3kB flash

Another code-size-related design choice I'm debating is whether to require the use of the ChainableGapEventHandler and its associates.

EDIT: The recently-added LinkLossService does this here:

mbed-os-experimental-ble-services/services/LinkLoss/include/ble-service-link-loss/LinkLossService.h

Lines 78 to 91 in 3bd3783

	/**
	* Constructor
	*
	* Initialize the internal BLE, EventQueue and ChainableGapEventHandler objects to @p ble, @p event_queue
	* and @p chainable_gap_event_handler, respectively. Furthermore, configure the alert level characteristic
	* with the appropriate UUID and initialize it to 0x00 ("No Alert").
	*
	* @param ble BLE object to host the link loss service
	* @param event_queue EventQueue object to configure events
	* @param chainable_gap_event_handler ChainableGapEventHandler object to register multiple Gap events
	*
	* @attention The Initializer must be called after instantiating a link loss service.
	*/
	LinkLossService(BLE &ble, events::EventQueue &event_queue, ChainableGapEventHandler &chainable_gap_event_handler);

Again, I don't think the code size/RAM overhead of these classes is significant enough compared to the benefits of using them.

[pan-]

There is no problem for BLE API to run on bare metal however at a lower level, some HCI drivers uses thread!
I wouldn't be too concerned about the EventQueue size for a general, most application will probably need one anyway. If it is really a concern, then it should be possible to define interface to an EventQueue that can be overridden whether at compile time or runtime (I would prefer runtime!) but I don't think it would be reasonable to use that route until that becomes a real concern.

I have the same position for ChainableGapEventHandler, we need some mechanism to dispatch Gap events to the services.

I suspect any application too tight on memory to not use a framework, monitoring code size is very important and maybe we should set code size targets for individual components first before devising on hypothetical problems.

[AGlass0fMilk]

I have been thinking about this more and starting an implementation as a proof-of-concept for the software architecture.

I've analyzed the code in mbed-os-example-ble a bit and started to see some common pieces that always have to be duplicated. So I started scheming with how to best encapsulate those pieces in the most modular way.

Maybe I'm being a bit too creative with this, but I have been trying to come up with a framework architecture that works with all the different BLE modes of operation while trying to keep memory consumption to a minimum. I've been playing with a concept called "MixIns" that are like static decorators. Rather than composing a decorated object at runtime, it is done statically with template classes. I am also trying to avoid the "Call Super" code smell by making using of the template method pattern. I guess the use of the MixIn pattern requires the use of "Call Super", but oh well.

In all the BLE examples, regardless of the application (eg: beacon, observer, gatt server, etc), there are always the steps of initializing the BLE subsystem, possibly the Security Manager, and setting up things like Privacy and the GATT table.

See a few references here:

• https://en.wikipedia.org/wiki/Mixin
• https://stackoverflow.com/questions/18773367/what-are-mixins-as-a-concept
• https://en.wikipedia.org/wiki/Decorator_pattern#Static_Decorator_(Mixin_Inheritance)

I used the existing classes in the ARMmbed/mbed-os-ble-utils repository as a base for the framework. I think @paul-szczepanek-arm and @rajkan01 were on the right track with the BLEProcess, BLEGattServerProcess, and BLEGattClientProcess. (Side note: I have also started changing the code in ble-utils to use mbed-trace instead of simply printf)

In my existing code, the base class of every BLE application will be the BLEProcess. Depending on your requirements, you can then add in features using MixIn classes. The BLEProcess has a virtual finish_initialization method that is overridden by MixIn classes:

https://github.com/AGlass0fMilk/mbed-os-ble-utils/blob/f11a5cfe328fc63d106aa3742ec8070ed1a0ef4d/BLEProcess.h#L112-L116

The BLEProcess class encapsulates the standard BLE initialization that must take place for any BLE application. After the BLE instance is initialized, it performs standard initialization such as setting up preferred phys (which can be set using the JSON configuration system). It then calls the virtual finish_initialization which is then dispatched to the subclass implementation:

https://github.com/AGlass0fMilk/mbed-os-ble-utils/blob/f11a5cfe328fc63d106aa3742ec8070ed1a0ef4d/BLEProcess.cpp#L74-L125

MixIn classes may override the finish_initialization, but must call the base class's implementation of it afterwards.

---

Side note: one potential issue I'm seeing with this is if the order of finish_initialization execution matters. i.e.: If a bug can be introduced by changing the order of inheritance. One way we can work around this is to standardize certain orders of inheritance by defining composite types with the using keyword... but that could get messy. I don't currently see a situation where the order of finish_initialization execution matters.

---

So, for example, the PrivacyMixIn implementation of finish_initialization sets up privacy and calls the base class's finish_initialization in its onPrivacyEnabled callback. Currently, it also sets up the security manager but I am in the process of splitting this off into a separate SecurityManagerMixIn. The PrivacyMixIn will then be a subclass of the SecurityManagerMixIn because the privacy features require the SM to be initialized.

As in the above example, the MixIn classes may also subclass/implement any EventHandler methods required. As the number of features mixed into the BLEProcess grows, the MixIns will implement the default EventHandler behavior.

Ultimately, to define custom applications, the user will have to subclass a composite BLEProcess and adjust any behavior there.

I have been working on some PlantUML diagrams to show the overall architecture but they are not yet complete. I started working on some proof-of-concept code which you can check out here: https://github.com/AGlass0fMilk/mbed-os-ble-utils/tree/ble-app-framework

@pan- What do you think of this so far? Do you have any ideas for an alternate framework architecture?

[AGlass0fMilk]

Also, our goal would be to fit a minimal application (such as a beacon, perhaps) into some of the low-cost, heavily constrained BLE chips on the market (eg: nRF52805 w/ 192kB flash and 24kB RAM).

[pan-]

I progressed a bit on the mixin part, trying to come up with a structure that would reduce code size and collisions. It is very rough and require factorisation of the dispatch side but it seems to work: https://gist.github.com/pan-/e2389c3481d04f71e9c943923de7aaff

I think the mixin code is fairly straightforward, the only oddities are:

• use self() to use base member. These can also be imported with a using statement.
• EventHandler functions are not override. No need for it as the dispatch is made statically

I think it could be useful to have consistent naming for mixin functions that are calls from the class. For example, to initialise the mixin, init seems a good name. For event handlers, using the same name as the one used in the regular event handler improves readability too.

[AGlass0fMilk]

@pan- Slowly going through the gist now... where does this stuff come from? Any recommended topics so I can better understand what's going on? Any resources would be appreciated.

Will multiple MixIns be able to handle the same events?

[pan-]

I've been inspired by CRTP, the idea is to have a top level class that handles the inheritance and dispatch to the mixin:

The dispatch of an event to the mixin event handlers is made by a class named Caller which call the associated function for all Mixins that are part of the chain. If the call is not overridden then it skips it and move to the next. This is made by recursion:

The mixins are unaware of the inheritance sequence, they can just assume they have access to a BleProcessBase and dispatch of events to the right methods is made at compile time.

I really like the idea of Mixins on several level and API conventions. For example we can imagine a service manager that register services and dispatch events: BleProcess<Advertising, Services<LinkLossService, ...> > all of the mixin exposes an init function that gets called by their container when its appropriate.

I twisted a bit the idea of CRTP and CRTP to get to that result, the only hurdle I see is the generalization of the caller, I have to think about it.

From a code size and memory standpoint it should fit the bill as there is no runtime polymorphism. However I have questions about it from a usability standpoint:

• Is it easy to implement new Mixin ?
• Is it easy to compose a new BLE process ?

The runtime approach is more straightforward in that regard.

---

This might interest you: https://www.fluentcpp.com/2017/12/12/mixin-classes-yang-crtp/

[AGlass0fMilk]

Sorry, this may seem a bit rushed. As I was writing Firefox crashed and I lost my original lengthy reply...

From a code size and memory standpoint it should fit the bill as there is no runtime polymorphism. However I have questions about it from a usability standpoint:

* Is it easy to implement new Mixin ?

* Is it easy to compose a new BLE process ?

One can't have it all I guess... 😅

I think there are two ways we can address these issues:

1. Provide a variety of premade MixIns that demonstrate best practices when designing with this framework
2. Create clear documentation on why the framework was designed this way, how the framework operates, and how to go about creating your own MixIns (with a toy example possibly)

For beginners, the premade MixIns should solve the most common use cases.

For more advanced BLE applications, the user will have to spend some time learning the framework.

I think this would at least be better than the current state of things. There are many pitfalls when writing BLE applications for Mbed and the framework would at least help most users avoid them. These are the kind of problems that can only be identified through experience and knowledge of the underlying stack operation. I still make the same mistakes after starting new (but simlar to old) BLE projects.

I've actually been thinking about applying this to the USB stack as well -- it would be nice to be able to more easily combine premade USB classes to make composite devices. Something like: CompositeUSBDevice<USBSerial, USBMSD>.

[idea--list]

I think some testing/profiling tools would be really useful for such a framework.
If i get it right the LinkLoss service triggers an action if the link is lost. Maybe i would also add an RSSI threshold so that an optional alert could be triggered if RSSI falls below a user defined threshold. That way we can define some safety margin or indicate the user they should stop moving away. Would be also great to implement approx. range (±5m) calculation based on time of flight.

Beyond these i would also like to suggest a basic signal generator that could generate different type of test signals. One signal could be discrete (like values generated by the HeartRateService example). Another signal would be a continuous one, i think the sinus wave would fit best (with that we can easily notice distortions/packetlosses on the receiving side). Also the signal sampling rate should be adjustable between 1/600 - 500Hz so that we could find out safe zones for different application scenarios rapidly.

[ghost]

Is the design of the mbed-os BLE code helping or hindering efforts to make it easier to build a dev friendly interface? Seems like most of the discussion is about adding on top of what already exists. Is there any modifications we should make there that can simplify this?

[pan-]

It depends on the module we're speaking about, for example ble::Gap offers everything one can expect from a Gap module at the expense of a large number of options and complex setup. On the other hand, a module like GattServer is rather poor in term of consistency and could make things difficult. GattClient is another example where a lot of wiring is left to the user to prevent unwanted memory consumption as storing the whole Gatt server of the peer in memory is a huge no for very constrained applications. The effect is that it makes things overly complex (for example for fetching CCCDs).

My view on the problem is that the API in Mbed OS should be able to address all use cases even the more obscure one in the most efficient way from a memory standpoint. It has a cost in term of complexity and accessibility. The Application framework built on top should address efficiently the most common use cases so anyone can quickly start with a foundation that wires things like you would do in a regular mbed-os application. If a user encounters limitations with it (for memory reason or else), they can transition to the Mbed OS API for parts where it makes sense in their application.

I see work and discussion being made in this group as an incubator for an accessible Bluetooth library and ideas and improvements to upstream in the Mbed OS API. For example, the event handler chain available in Mbed OS was born from discussions here.

[ghost]

Are there any equivalents you folks like outside of the realm of MCUs? Something you wish it could be like?

I've never developed a peripheral outside of embedded uses, but I've seen folks make stuff out there like bleno2 for node, and a library for python I can't remember the name of. I have used noble and another python library I can't remember the name of to make central devices though.