ETO REST API in Swift

Logging in to ETO

This code base has become a bit of a pet project.  It uses Alamofire to create GET and POST requests against Efforts to Outcomes REST API.  This enables native applications to interact with our Homeless Management Information System (HMIS) database.  I’ve been coding it to practice the craft.  Also, there are a few operations in our continuum of care which could be improved by creating native applications, as opposed to using webapplications.  For example, street outreach teams are often without their computer, given the bulk.  But rarely is a street outreach specialist without a mobile device.  This would allow the outreach specialist to access all data regardless of physical settings.

Identified benefits from native applications:

  • Self check in at shelters
  • Using TouchPoint responses instead of Points-of-Service for data activities, such as shelter check-in.
  • Street outreach
  • Leveraging existing hardware (most staff have smartphones)
  • User friendly UIs
  • Data aggregation through automated queries
  • Point-in-Time Count mobile-device survey
  • Collecting GPS coordinates of street outreach and point-and-time activities

This will be a documentation journal, given the project will most likely evolve to be bigger than my organic memory could manage.

Login()

Setting up a session which is authorized to request information from an ETO databse has several steps.  The first, being the user has an actual account with the database 

  1. Take username, password, and webservice URL and make POST request.
  2. Async-wait for response
  3. Unwrap response data.
  4. If POST request returned 200-299, request was success. If failed, handle.
  5. When successful, convert the return JSON data.  If failed, handle.
  6. Pull SSOAuthToken from JSON data.
  7. If it was found in JSON data, call GetListOfEntperises() passing SSOAuthToken
  8. Async-wait returned list of Enterprises.
  9. Call completion callback meothd with parameter “Success”
//
//  ETOWebServiceLogin.swift
//  ETO-TouchPoint-Test
//
//  Created by Thomas Ladvien on 10/13/16.
//  Copyright © 2016 Honeysuckle Hardware. All rights reserved.
//

import Foundation
import Alamofire
import SwiftyJSON

public func login(currentSessionInfo: SessionInfo, username: String, password: String, completion: @escaping (_ response: ReturnInfoWithString) -> Void){

        // ETO Authentication Web Service.
        let SSOAuthenticateService = "https://services.etosoftware.com/API/Security.svc/SSOAuthenticate/"

        let parameters: Parameters = [
            "security": [
                "Email": username,
                "Password": password,
            ]
        ];

        var returnInfo = ReturnInfoWithString()

        // TODO: Check for HTTP response code.
        let request = Alamofire.request(SSOAuthenticateService, method: .post, parameters: parameters, encoding: JSONEncoding.default).validate().responseData { response in
            if let data = response.data {

                switch response.result {
                case .success:
                    let json = JSON(data: data)
                    returnInfo.value = json["SSOAuthenticateResult"]["SSOAuthToken"].stringValue
                    if ("00000000-0000-0000-0000-000000000000" != returnInfo.value) {
                        returnInfo.callback = self.prepareResponse(targetResponse: .Success)
                        currentSessionInfo.SSOAuthToken = returnInfo.value
                        completion(returnInfo)
                    } else {
                        returnInfo.callback = self.prepareResponse(targetResponse: .NoSSOAuthToken)
                        completion(returnInfo)
                    }
                case .failure:
                    returnInfo.callback = self.prepareResponse(targetResponse: .HTTPRequestFail)
                    completion(returnInfo)
                }

            } else {
                returnInfo.callback = self.prepareResponse(targetResponse: .UnknownFailure)
                completion(returnInfo)
            }
        }
        if(DebugMode){
            debugPrint(request)
        }
    }

Ok, let me break down steps in this function.

  1. Take username, password, and webservice URL and make POST request.
  2. Async-wait for response
  3. Unwrap response data.
  4. If POST request returned 200-299, request was success. If failed, handle.
  5. When successful, convert the return JSON data.  If failed, handle.
  6. Pull SSOAuthToken from JSON data.
  7. If it was found in JSON data, call GetListOfEntperises() passing SSOAuthToken
  8. Async-wait returned list of Enterprises.
  9. Call completion callback meothd with parameter “Success”
How to Build a Robotics Digital-Hackerspace

Some HTML

Why the Hell?

Wow, it’s been awhile since I’ve actually written anything in Drupal based text editor.  Feels like home.

Honestly, I’m not going to go into the “Why” of building this site.  Maybe I’ll do that later.  Let’s just say, there was an impetus to recreate the home we were forced to leave.

How the Blink?

Spinning Up a Server:

The first thing was deciding on a server.  I didn’t have much (um, any) experiencing in spinning up a server.  But after a bit of reading, checking the bank account, and finding what would work for Drupal site, I chose Linode

Definitely happy so far.  Their business model seems to be no frills Linux servers which are fast to spin up–if you know what you are doing.  Of course, I didn’t.  Still, they had a lot of fairly up to date walkthroughs.

Here are the walkthroughs I used (in order) to spin up this server:

  1. Getting Started
  2. Securing your Server
  3. Hosting a Website
  4. Installing Drupal 7

This almost all the information needed to create this site’s based.  A couple of caveats:

sudo apt-get install php5 php-pear sudo apt-get install php5-mysql

Needs to be replaced with:

sudo apt-get install php php-pear sudo apt-get install php-mysql

This will install the latest version of PHP, for me, that was PHP7.

Also, the install directory is different for PHP7.  This is important, since the php.ini file needs to be edited.  In 7 it may be found:

/etc/php/7.0/cli

The last thing to do was relocate the Drupal 7 files to the directory where Apache can serve them up.

cd /var/www/html sudo wget https://ftp.drupal.org/files/projects/drupal-7.50.zip

If all the script-blogs are followed correctly, then when entering the web IP of the site in the browser you should see the following:

Design Goals:

Goals.  It’s good to have them.  Thinking through what I’d like out of a new robot-home here are some of the concepts which seemed critical and important:

  1. An open place for anyone to express ideas, with minimum viable censorship.
  2. Responsive.  Not just mobile friendly, but optimized for quick viewing of information.
  3. Shoutbox.  Gots to have one.
  4. WYSIWYG and rich content editors:
    • Iframes
    • Image boxes
    • Image storage system
    • Code-highlighting including our favorite languages (and Ada for Mr. Bdk6)
  5. The primary features of our old home:
    • Collect
    • Chill-out Zone
    • Forums
    • Shoutbox, oh wait, already stated that.
  6. New features requested by members:
    • Content rating
    • Member points
    • Wiki
    • Anything we could dream of

Theme:

There are three layers to the theme: 

  1. AdaptiveThemes was selected as the main
  2. Corolla over it
  3. FooTheme went over it.

In the /var/www/html/sites/all/themes/corolla/ directory there is a file called corolla.info inside a referenced was added for css/corolla_overrides.css.

Then, in the /var/www/html/sites/all/themes/corolla/css directory the corolla_override.css file was made which includes several hackish css snippets:

    /* The title was originally set to -5px (weird) */
    #site-name a {
      letter-spacing: 0px;
    }

    .wrap-code {
      overflow-x: scroll;
    }

    /* These hide some uncessary shoutbox labels. */
    .shoutbox-interval-msg {
      visibility: hidden;
      display: none;
    }
    #edit-nick{
      visibility: hidden;
      display:none;
    }

Regarding the color scheme.  The palettee can be edited in the Footheme “Color” section.  This requires th Color (Drupal Core) module be enabled.  The palettee was selected by using Pictaculous taking an image of Russian revolutionary art.  This was meant to capture the feeling palette selected by an artist with a better understanding of color-emotion connections.

Fonts
Logo selection

Modules:

Admin Tools:

“Admin Tools is an addon module for the Admin module, which provides a sidebar navigation for selected roles. The Admin Tools module adds functionality and quick access for clearing caches, running cron and updates much like Admin Menu.”

hate cache errors.  Mother-blinking cache!

Administration Menu:

Admin tools sped up development a lot.  It basically simplifies the admin menu so 4th level items are exposed to one click.

Blog (core)

Allows the Drupal site act like a good old blog–for us, this allows multi-authoring content and management.

Block (core)

Allows a block design of the UI.

Chaos Tools

This is a dependency for other modules (a lot of others).

CKEditor

The CKEditor is the core of the Drupal blogging package.  It is the editor used to create this post.  However, it put up the most fight when trying to install.  Actually, that’s not fair.  It wasn’t the CKEditor it was the code highlighting which was such a pain.  The code highlighting allows this:

    for(int i = 0; i < marioTouchesRobots; i++){
        aHackerSpiritDies();
    }

I’m going to list out the steps used to setup the CKEditor used for this article, but then discuss some of the pitfalls, which ended up costing a lot of development time.

Steps to Setup CKEditor with CodeSnippets and HighlightingJS:

  1. Download the CKEditor - WYSIWYG HTML editor module.
  2. Enable the CKEditor module.
  3. Go to Configuration–>CKEditor–>Edit CKEditor Global Profile
  4. Set the “Path to CKEditor” to //cdn.ckeditor.com/4.5.4/full-all. This will use the content delivery network to serve up the CKEditor JavaScript.  It also lets you access a lot of plugins without having to manage them.  The other option is to pull a copy on the local server–it’s a lot more hassle managing.
  5. Go to Configuration–>CKEditor–>Profile X–>Edit (note, X = Text Editing profiles users will be able to select when blogging.  These can be managed under content Content Authoring –> Text Formats).
  6. Go to Basic Setup.  Here, add all the Text Formats this particular CKEditor profile should affect.

  1. Under Security make sure “Always run security filters for CKEditor” is Enabled (should default).
  2. Under Editor Appearance go straight to the check-box title “Plugin for inserting Code Snippets” and enable it.
  3. Also, enable what other CKEditor Plugins needed.  Note, there are more plugins then this, but these are the ones provided through the Content Delivery Network.
  4. Scroll to the end and hit Save.  Now, go back to Configuration–>CKEditor–>Profile X–>Edit.  Go straight to Editor Appearance.  There should be a new button available
  5. Add the Code Snippet button to the “Current Toolbar”
  6. This should enable the CKEditor and provide the Code Snippet button.
  7. Download the highlight js Drupal module.  This should be installed in the modules directory
  8. Navigate to var/www/html/sites/all/libraries folder and make a directory called ‘highlightjs’, switch to it.
  9. The highlight js module is dependent on the actual highlightjs css libaries though.  Download a package in the /var/www/html/sites/all/libraries/highlightjs folder.
  10. Unzip highlightjs here.
  11. Issue the command ‘sudo mv higlight.pack.js highlightjs’.  This is required or the highlight module can’t find the libraries.
  12. And the command ‘sudo chmod 666 highlightjs’.
  13. Go to the modules dashboard and enable Highlight JS Syntax.  DO NOT enable Highlight JS Filtetr.
  14. Open the modules /var/www/html/sites/all/modules/ckeditor$
  15. Type sudo nano ckeditor.config.js
  16. Add each HighlightJS language you would like to show in the dropdown box in the CKEditor.  The part on the left of the colon is should match the HighlighJS language code.  The part between the ‘ ‘s will be what is displayed in the CKEditor dropdown.  When adding supported languages, here’s a good reference – Supported HighlightJS languages (but it doesn’t include custom languages, like Arduino).  Don’t forget to save when done.
        config.codeSnippet_languages = {
            php: 'PHP',
            python: 'Python',
            arduino: 'Arduino',
            c: 'C'
        };

There is an issue with the HighlightJS module where the text escapes the divs.  It took a long time to find the culprit.  Apparently, the HighlightJS modules causes this whenever it renders HTML produced by CKEditor.  

  1. Go to /var/www/html/sites/all/modules/highlightjs
  2. Type sudo nano highlight_js.css
  3. Enter the following style and save:
    .field-items {
        width: 100%;
    }

And that should be it.  A couple words of warning.  Make sure you don’t enable the HighlightJS Filter.  This will essentially double encode the HTML entities inside the block.  This causes >, <, and & to show as ">, &lt, &" respectively.  This simple little issue took a lot of development time to solve--given the manual was lacking.

Color Comment Contextual Links Dashboard Database Logging Entity API Field Field SQL Storage Field UI File Filter Five Star Forum Help Highlighter JS Filter Highlighter JS Syntax Highlighter Image IMCE Libraries List Menu Module Filter Node (core) Number Options Overlay Path Poll RDF Search Shoutbox Shoutbox Patch Statistics Statistics Counter SysLog System (core) Taxnomy Text (core) Update Manager User Userpoints Userpoints Service Views Views Content Panes Views UI Voting API

Lumi

Update 3/29/2016

I’ve begun re-writing the entire program as a Windows App. Not really my choice. It seems the BluetoothLE API Microsoft has thrown together (for-real-doh, it’s super bad) is most easily adopted under the Windows Universal App toolchain.

Anyway, here’s the resting space of the new projects

Lumi

Lumi for Desktop

Where I come from they having a saying: “Video or it didn’t happen.” I’ve always liked that saying. It means we have to put our work where our typing is. Here is a video of the uploader I wrote to interface with the TinySafeBoot AVR two-wire (UART) bootloader.

It was written with the idea a serial connection could be directly opened from the computer using Bluetooth 4.0. Or, if we get super fancy, using WiFi. On the receiving end, connected to the AVR, would be an HM-10 or an ESP8266 respectively. This would allow direct wireless uploading of Arduino sketches (or Atmel studio hex files) to the targeted Arduino (or AVR, if you will).

It has a couple of kinks, but I’m fairly pleased with it.

For a little more information on the projects:

http://ladvien.github.io/robots/tsb/

The whole projects is named after my friend Lumi who passed away last year. Amazing man with good taste in photography subjects. :)

Source Code:

https://github.com/Ladvien/Lumi_TinySafeBoot_Uploader

TinySafeBoot, Arduino, and Wireless Upload to ATtiny85

The Dream

I’ve dreamed of a PCB with everything: uC, H-bridges, PSU, Lipo charger, inductive power collector, and a wirelesss device which would allow wireless uploading to the uC. A platform I could use for robotics and wearable projectss alike. Most of these features I have solved, yet one which I believed to be ever elusive to a hacker is wireless uploading. I’ve had lot of failed attempts, some as painful as writing my own uploader for the LPC1114, others as “simple” as hacking away at the Arduino IDE source. However, I’ve found a combination which brings me a bit of hope: TinySafeBoot, Arduino IDE, and most Atmega and ATtiny Atmel chips.

TinySafeBoot

TinySafeBoot is a pretty neat GPL projects which is around 500 bytes. It is a bootloader which allows selection of any two pins as RX / TX for two-wire communication (it also has a one wire protocol). And it has a serial communication protocol.

TinySafeBoot

The feature which makes TSB such a great choice for wireless bootloading is its serial protocol. Unlike the hardware protocol Arduino uses, serial communication allows the remote device to control the bootloaders workflow without flipping any hardware pins. Contrast with the Arduino’s bootloader (I think it’s optiboot now?) which focuses on resetting the device repeatedly to activate the bootloader.

The hardware approach doesn’t work for quick, remote hardware resets. With the serial lag, the Arduino bootloaders are often comfortably booting an existing program by the time the first byte of a new program is reaching the RX pin. Also, there is no quick (sub 400ms) way to flip pins on my focused wireless devices, the HM-1X and ESP8266. In short, either the Arduino bootloader or software would need to be hacked; neither, which I’ve had the time to do.

There are other chip possibilities. I spent a lot of time trying to write a wireless uploader for the LPC1114. However, I do not yet have the skill level to finish it (I’ve not given up on it). There is also a psychological hangup working with the LPC1114. Its documentation. It is not that the docs for the 1114 are so bad, it is the docs for Atmel (makers of the Arduino heart) are so much better. This makes solving the wireless upload dilemma much more beneficial if it were solved for Atmel chips. Well, at least for a hacker like me who does not yet have the skill to move quickly through rougher documented chips. Honestly, it’s about time–I’ve only so much and sometimes it is about getting a projects finished rather than what I might learn (blasphemy, right?).

Of course, TSB has other features such as auto-bauding and password protection, etc. It’s not perfect, but I like it for its simplicity.

Getting TSB Setup

I have tested TSB on three devices, the ATtiny85, ATmega328P, and ATtiny1634. However, the ATtiny1634 didn’t work.

There are two parts to TSB,

  1. Firmware
  2. Software

We are only going to download the software which allows you to generate “custom” firmware for whatever chip you are targeting.

From here on I will be demonstrating getting TSB going on an ATtiny85.

ISP

First, we will need an AVR ISP. Long ago I bought an AVR-ISP MKII which I used in combination with Atmel Studio 7. This will allow us to burn the bootloader and set the fuses. If you do not have an official AVR-ISP you can, theoritically, use AVRdude through the Arduino IDE in combination with an Arduino as an ISP programmer.

So, either,

  1. AVR ISP MKII
  2. Atmel Studio 7

Or

  1. Arduino IDE
  2. Arduino Uno ISP

Either way, once we have our ISP setup we will connect it to the ATtiny85 like,

One note about the above breadboard. There is a momentary switch connected in series with the RESET pin,

PB5<--->Switch<--->GND

This will allow us to easily reset the ATtiny85, which is needed to put the TSB into bootloader mode.

Notice the RX / TX will. They are not part of the ISP. They will be our bootloader’s TX / RX pins when the bootloader has been burnt.

Bootloader Creation

Now, to generate the bootloader.

If you haven’t, download the software, extract it to a workspace, and open that workspace in the command-line. The TSB software is CLI only.

Now enter the following,

C:\TSB_workspace>tsb tn85 b3b4

TSB then should generate a hex file titled something like,

tsb_tn85_b3b4_20150826.hex

This hex file will be our bootloader customized for our particular AVR, the ATtiny85.

To breakdown the TSB command,

  • tn85 tells the TSB SW we are looking to install this firmware on an ATtiny85. If you need to look up a chip code you can find it in a file in the TSB workspace titled “devices.txt”
  • b3b4 tells the TSB SW you want to use PB3 as RX and PB4 as TX. I chose these as pins since they are not being used by the ISP. Note, having the ISP and UART connected to the same pins will cause problems.

Bootloader Burning

To burn the newly generated bootloader you will need to have your ISP connected to the ATtiny85. First, check the fuses of your chip. According to the TSB website we need to have the following fuses set,

ATtinys:

  • SELFPRGEN must be set to enable flash writes from firmware, e.g. TSB
  • BODLEVEL should be set to avoid flash corruption during unsafe device power-up.
  • LOCKBITS may be set to MODE 3 for enhanced security (i.e. serious password protection intended).

ATmegas:

  • BOOTRST activated lets the MCU jump into the Bootloader Section (rather than $0000) with every hardware Reset.
  • BODLEVEL should be set to avoid flash corruption during unsafe device power-up.
  • BOOTSZ=10 or BOOTSZ=11 to reserve 512 bytes for a Bootloader Section.
  • BLB to MODE 2 or 3 protects Bootloader Section from undesirable write access by firmware.
  • LOCKBITS may be set to Mode 3 (LB-Mode 3) in a security environment.

Using Atmel studio, go to Tools-->Device Programming select your programmer (AVR ISP MKII for me), select the ATtiny85, and hit apply. Now, go to Fuses and make sure they are set to the following for the ATtiny85.

If you need to set the fuses using an Arduino-as-ISP, well, I’m not sure I’ve ever done that, but here’s an article that seems legit,

Once the fuses have been set it’s time to burn the bootloader. Using the Atmel approach, go to Tools--Device Programming and click on Device Signature, this will allow the ISP to see if the chip is ready for uploading. Once the chip is verified, go to Memories and click on the ellipses and select your hex file,

Then hit Program. With luck, your bootloader will burn and there will be much rejoicing.

To make sure the bootloader was burned correctly connect your ATtiny85 directly to a USB-to-UART. Open a serial terminal and connect at any baud rate 9600. Hit reset on the ATtiny85 and then quickly (within 2 seconds) send the @ in the serial-terminal. If the bootloader has been burned the ATtiny85 should reply with TSB followed by device data.

Wirelessly uploading

Here’s the fun part. I hooked up my ATtiny85, 2 x HM-10 modules, and an FTDI chip to a PC–like below,

To see if you everything is working, open a serial terminal program. I’m a little partial to this one: HM-1X Aid. Connect the your USB-to-UART using the serial terminal at 9600bps. Once you are connected, press the reset button on your ATtiny85 and send the character @ from your terminal several times. If everything is setup correctly the chip should reply with, TSB followed by device information. If it doesn’t reply, a few things to check.

  1. Test your wireless connection. I had to make sure the Bluetooth devices had paired and the Bluetooth device connected to the ATtiny was baud setting was less than 115200.
  2. Make sure your fuses are set correctly. Especially, SELFPRGEN on the ATtiny, since it doesn’t come from the factory set.

Uploading an Arduino Sketch Wirelessly

Ok, if you were able to get the bootloader to reply wirelessly it’s time to test the whole setup. Open your Arduino and make sure you have install support for the ATtiny85,

Then, in your Arduino IDE select,

  • Board: ATtiny
  • Processor: ATtiny85
  • Clock: 8mhz (internal)

Add this test sketch,

void setup() {
  // put your setup code here, to run once:
  for(int i = 0; i < 9; i++){
    pinMode(i, OUTPUT);
  }
}

void loop() {
  // put your main code here, to run repeatedly:
  int pin = 0;
  for(int i = 0; i < 9; i++){
    digitalWrite(i, HIGH);
  }
  delay(500);
  for(int i = 0; i < 9; i++){
    digitalWrite(i, LOW);
  }
  delay(500);
}

Still in the Arduino IDE go to Sketch--->Export compiled Binary. Save the hex file to your TSB Workspace. Open your TSB Workspace in the command prompt and type the following, replacing the COM port # with COM port you have the USB-to-UART on and replace the file name with the name of your newly compiled hex file.

C:\TSB_Workspace\tsb com4:9600 fw name_of_sketch.hex

Hit the RESET button on your ATtiny85 and then hit enter on your PC. If all goes as planned the command line will change to “Reading” and “Writing” appropriately. When the file has been uploaded, reset your ATtiny85. After a few seconds each pin on the ATtiny85 should go HIGH for half a second and then go LOW for half a second. If you don’t have an LED or MM, open your serial terminal again, connect to the COM port with your wireless setup, your serial terminal should receive the character, “0x00” every half a second.

Other baud rates work. But I found anything over 38400 to be unstable.

That’s it. Let me know what questions you have.

The future

Now there is a way to upload hex files wirelessly to a large array of chips, I plan to try a few things,

  1. Write an uploader in C# which could handle controlling a PIN on the HM-1X remotely which would pull the ATtiny85’s (or other chip) reset line low. This would allow me to wirelessly set the chip into bootloader mode.