This namespace contains the serial device handling functions. The goal is to have two abstract classes which define the interaction between the main device and the peripheral These two classes will be responsible for searching, connecting, exchanging data, closing connections, and device failure handling. To get going I’m going to take some advice from Mythical Man Month: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.” Well, I don’t have tables, so I guess my charts will have to do.
LumiCommunication is largely modeled after Apple’s CoreBluetooth API. It has abstractions representing both remote and local devices. These abstractions are inherited in concrete classes for different device types. Currently, the API is focusing on two device types, BluetoothLE and WiFi (ESP8266). Though, if the abstraction is effective, it shouldn’t be difficult to provide support for Bluetooth Classic and Wired Serial connections.
The CentralManager’s main responsibilities are to monitor the PC’s device status, initiate searches, connect to devices.
I’m adding a bit to the CoreBluetooth model; I’m writing this code to be hacker friendly as one of my biggest peeves with frameworks meant for interacting with embedded devices is they often don’t allow for easy modification of the device’s behavior. For example, most SoC-centered device modules (HM-10, ESP8266, etc.) which control radio hardware have firmware allowing for the modification of the module’s behavior. Often, these are modified through AT Commands. It struck me one day, why am I not writing code in such a manner? For example, instead writing concrete code inside the classes which handle searching for BluetoothLE to to automatically connect to known devices, why not create an static object which defines these behaviors, then, when I want to change the behavior of my hardware, I simply pass in a new behavioral definition object. I’ve outlined this in my BehavioralBluetooth project (abaonded for the time being).
LumiCommunication classes will have an object which defines the behavior of the hardware.
The PeripheralManager is responsible for representing the states and delivered data of the peripheral devices. There are events associated with data received from the device, confirmation of sent data, device state changes. Like the CentralManager the PeripheralManager will have a PeripheralBehavior object which will define its actions. There are received and sent buffers to monitor succesful flow of data between the local and remote device.
This is the next iteration in a three year project. Here are the current iterations:
Vorpal Hoff – an attempt at wireless uploading with a HM-11 and LPC1114 combination. Written in C / C++ (Initialized May 22nd 2014).
HM-1X Aid – this project was meant to be a GUI on top of the HM-1X modules, allowing “easy” editing of the module’s behavior. It was my first venture into C#. (It’s sooo bad;although, the serial communication was asynchronous.) (Initialized Dec. 19th 2015)
Lumi1 – this the first successful TinySafeBoot uploader. It was written in C# using the .NET WinForms. Unfortunately, it was synchronous. And I was finished with the USB-to-UART uploader before I realized there was no easy BLE support in WinForm’s .NET. (Initialized March 2nd 2016)
Lumi2 – this is where things start getting better. It is the current version of the TSB wireless bootloader. It works, is asynchronous, and has BLE support. Unfortunately, the code turned into spaghetti. This is largely due to my poor understanding of object-oriented design. It has god-modules, a horrifically implemented SerialEvent response protocol, poor encapsulation, no polymorphism. It’s just a mess. (Initialized March 21st 2016)
Lumi3 – this project was stopped early. It was an attempt to build a multiplatform uploader using Xamarin Forms. It would have allowed iOS, Android, and Windows versions of the application. Unfortunately, it is a fairly complex project. Theoretically, the uploader would work by allowing the user to select a HEX file from Dropbox, handshake with TinySafeBoot using BLE or WifI, then upload the HEX file. And though this is theoretically possible, it would take learning two new APIs: Xamarin Forms and Dropbox. And my focus is dilberate practice of language conventiosn and OOP, rather hacking through two new APIs. Most likely, I’ll come back to this project after Lumi4 (Jan 13th 2017)
Why? Seriously, dude
It is important to state the objective of the this three-year project has not been to produce a product which works, although, that’s a close second. The purpose of these repeated attempts is to improve as a developer.
Of course, I don’t believe if I try enough and eat my Wheatie I’ll grow into a great developer. But with each iteration I’m focusing on developing a handful of new skills. This learning strategy is from the book Peak by Anders Ericsson. The continued and intentional practice is labeled by Ericsson as “deliberate practice.” It’s with this mindset I’ve approached this iterative coding project, hoping with each iteration the code improves, but more importantly, my skill as a developer improves.
Deliberate practice involves selecting particular deficits to focus on. This is more effective, as the improvement is in a few intentional areas, rather than trying to practice every important nuance of a skill at once. This seems easy to get behind; especially, when it comes to developer skills. There are just too many to try and refine all at once.
Sadly, focused practice isn’t something I was intentional about for the first few code bases in this series. It wasn’t until Lumi3 and Lumi4 did it bubble up as crucial in the process of developing my skills.
Testing fascinates me. When I studied psychometrics there were a battery of tests as to whether and instrument worked as intended. In the psychology world these were the fundemental building blocks of effective research and practice. Why did we use CPT? Because it passes a variety of tests to demonstrate efficacy. In the developer world tests still hold my fascination. They consistently demonstrate a product is capable of completing the task for which it was designed.
Of course, I’ve struggled with adopting testing in projects. A lot of this has to do with poor understanding of how to design a test to meet a purpose. It wasn’t until I was listening to a Coding Blocks (#54 – Writing Amazing Unit Tests) episode on writing unit tests did I get more comfortable with testing. Specifically, when they discussed the differences between unit and integration testing.
When I first attempted to write tests for a project it was unit tests. Unfortunately, this project was the second iteration of my Lumi uploader and the tests resulted in a hot-mess. This is because I was trying to test functions which relied on inputs from other systems.
This is as far as I made it writing a unit test on a method which was meant to test whether the StartBLEWatcher() method was working. In unit testing there should be an assert on the output of the method, but StartBLEWatcher() returned discovered BLE devices (yes, I realize the method could be re-written better, thus the reason for this article). This is where I got frustrated. ”How the hell am I suppossed to write unit tests on code which interacts with other hardware!?” I mean, I get it, unit testing is the bread-and-butter of professional programmers. It helps building big projects which would otherwise collapse under size. But how the hell do I write unit tests for code which relies on outside systems!? Saldy, I found the answer too late: You don’t.
In the Coding Blocks episodes there is a discussion on the difference between unit tests and integration tests. A unit test is meant to test a small piece of code and it should rely on no other code. An integration tests checks whether a piece of code works as intended. However, unlike unit tests, integration tests do rely on outside systems by their very definition.
When I heard this discussion I went to the first StackOverflow answer on the subject:
Question: “What is the difference between integration and unit tests?”
A unit test is a test written by the programmer to verify that a relatively small piece of code is doing what it is intended to do. They are narrow in scope, they should be easy to write and execute, and their effectiveness depends on what the programmer considers to be useful. The tests are intended for the use of the programmer, they are not directly useful to anybody else, though, if they do their job, testers and users downstream should benefit from seeing fewer bugs.
Part of being a unit test is the implication that things outside the code under test are mocked or stubbed out. Unit tests shouldn’t have dependencies on outside systems. They test internal consistency as opposed to proving that they play nicely with some outside system.
An integration test is done to demonstrate that different pieces of the system work together. Integration tests cover whole applications, and they require much more effort to put together. They usually require resources like database instances and hardware to be allocated for them. The integration tests do a more convincing job of demonstrating the system works (especially to non-programmers) than a set of unit tests can, at least to the extent the integration test environment resembles production.
Actually “integration test” gets used for a wide variety of things, from full-on system tests against an environment made to resemble production to any test that uses a resource (like a database or queue) that isn’t mocked out.
I’ve been working on writing my own flash UART uploader since May 2014. Originally, I was trying to write an uploader in C using the GCC compiler. The idea was to upload a Intel HEX file compiled for the LPC1114 to the uC remotely, using a Bluetooth LE connection. Here’s a description of the custom circuit board designed for the project:
Unfortunately, the project was out of my league. After spending months writing C code there it was not usable. Of course, learned a lot about C in the process.
Well, after a couple of years I started on code to upload compiled Atmel ATMega and ATtiny programs using the same method outlined in the Valdez Mutant article. But this time, the uploader was written in C# on Windows. And it interfaced with the TinySafeBootloader on the Atmel uCs.
Strangely, I actually finished the project. The first code-base was written as a C# Forms application. This worked out great! I was actually able to use the System.Devices.Ports to access a CH340G or FTDI chip. The USB-to-UART then shook hands with the bootloader on either an ATMega328P, ATtiny84, or ATtiny85 (others should be supported, but these were the only tested due to the simplicity of the Arduino HAL).
It was really the lack of BluetoothLE support which forced a change in directions. However, the elusive wireless upload to an AVR was just too close to abandon. Reluctantly, I created yet another code base. This time, it was derived from the Windows Universal App platform.
After a few months I had a working version. It was able to upload to ATtiny chips and ATMega chips over Bluetooth LE.
However, when I started trying to add ESP8266 support–well, things went to the Pooh. It seemed of all the problems listed above the only one resolved was the adding of Bluetooth LE support. My skill was not increasing.
Also, there were two additional issues which arose:
Handling advertisement and connection for Bluetooth.
There was a rather nasty bug around writing to a connected device.
I’ll not dig into the details, but with this sample in mind here is the outline of how I achieved BluetoothLE in-app scan and connect.
When OnAdvertisementReceived fires you get the discovered devices ID from the EventArgs
After the user discovers the device sought, then a user input would start a the asynchronous creation of a BluetoothLEDevice using the ID found from the AdvertisementWatcher.
Here’s where it gets hackish: If the device is successful in connecting, then there is no event–rather, a callback timer should be started with enough time for the BluetoothLEDevice to connect and enumerate.
When the timer callback fires then, using the new var device = await BluetoothLEDevice.FromBluetoothAddressAsync(ID).
After the wait, the services variable should have all of the services found on the BluetoothLEDevice. At this point, all the services on the remote device should be enumerated–and var services = device.GattServices, which includes enumerating services and characteristics.
What the API actually expects is the user will connect to the device using Windows built-in Bluetooth support. This API seems poorly thought out and unfortunate. Even Apple, with all of their “developer guidance”, doesn’t tie the developers’ hands when searching and connecting to BluetoothLE devices. Of course, CoreBluetooth was developed early in BluetoothLE’s lifecycle, so maybe that’s before API developers knew better than turn too much power over to code-consumers? Who knows! But I’ve strong feelings on the matter, given it took me so much time to figure out Microsoft’s intentions.
And with that–I’m closing down the Lumi3 project and starting on Lumi4.