Ladvien's Lab

Latest Posts

A LEGO Classifier -- CNN and Elbow Grease

I've a robot friend. To be clear, the friend is not a robot, rather, we build robots together. One of the projects we tossed about is building a LEGO sorting machine. Rockets is the friends name--again, not a robot--teaches robotics to kids. For their designs, LEGOs are the primary component. Unfortunately, this results in much time spent to preparing for an event.

He mentioned to me, "What I really need is a sorting machine." And proceeded to explain his plain for building one.

I was skeptical for some time, but finally, I got drawn in he talked about incorporating a ...

Setup a Local MySQL Database

The last two articles have been getting oriented to SQL, however, the information in them will disappear quickly if we don't give you a way to practice on data meaningful to you. Let's face it, as much fun as it is to find out random employees salaries, those don't mean anything to you .

This article will show you how to setup a copy of MySQL Server on your PC, connect to it, load data from a CSV, and query those data. There's a lot to get done, so let's get started.

Local MySQL Server Setup

Each of the three operating ...

Understanding the MySQL Query

Welcome back! Alright, now we know how to connect to a remote server from within MySQL Workbench, let's start writing some queries.

Here's a common SQL query:

    SELECT e.emp_no,
           e.last_name,
           t.title
      FROM employees AS e
 LEFT JOIN titles    AS t
        ON e.emp_no = t.emp_no
     WHERE e.hire_date > '1999-12-31'
  ORDER BY e.last_name DESC;

This query produces the following table when run on our employees database.

row_num emp_no last_name title
0 47291 Flexer Staff
1 60134 Rathonyi Staff
2 72329 Luit Staff
3 108201 Boreale Senior Engineer
4 205048 Alblas Senior Staff
5 222965 Perko Senior Staff ...
Beginning MySQL for Data Analysts

I'm usually writing about hacking, robotics, or machine learning, but I thought I'd start journaling thoughts on data analytics, which is how I pay the bills these days. I wanted to begin with a series on MySQL, as I've some friends I feel it'd help enter the field. But, I'll eventually expand the series to include visualizations, analysis, and maybe machine learning. And I hope these articles help someone move from manually generating reports in Excel to writing scripts that'll automate the boring stuff. As I like to say, "knowing to code gives you data superpowers!"

I'm a professional data ...

Creating a Neural Network Webservice

We're almost done. In the previous articles we've used a local machine to train a CNN to detect toxic sentiment in text. Also, we prepared a small (1GB RAM) server to use this pre-trained network to make predictions. Now, let's finish it and create a webservice where anyone can access our awesome magical algorithm.

Prediction Service

On your remote server, navigate to your flask_app folder and create a file called nn_service.py . The following code creates an HTTP request endpoint /detect-toxic and it exposes to other programs running on the server. A bit more explanation after the code.

cd /home ...
Preparing a Small Server for a Neural Network Webservice

Previously, I wrote about training a CNN to detect toxic comments from text alone. But, I realized, even if one has a nice little NN to solve all the world's problems it doesn't help unless it is in production.

This article is going to cover how to prepare a server and needed word embeddings to mechanize the NN in a Flask webservice.

Server Setup: Preamble

For this project I'm using a small server from Linode--called a "Nanode." At the time of writing these servers are only $5 a month. The catch? They only have 1GB of RAM. It's definitely going ...

Training a Toxic Comment Detector

I'm writing learning-notes from implementing a "toxic comment" detector using a convolutional neural network (CNN). This is a common project across the interwebs, however, the articles I've seen on the matter leave a few bits out. So, I'm attempting to augment public knowledge--not write a comprehensive tutorial.

A common omission is what the data look like as they travel through pre-processing. I'll try to show how the data look before falling into the neural-net black-hole. However, I'll stop short before reviewing the CNN setup, as this is explained much better elsewhere. Though, I've put all the original code, relevant project ...

Image source: Louis Reed
Distributing Machine Learning Jobs

Boss

A human sends machine learning job to the Boss. A Job is JSON object containing the the desired machine learning script and the parameters needed for successful execution. The Boss stores the Job and Creates an Order. The Order is another JSON object representing the state of a requested Job.

         Job #4
 0                        Boss
/|\ +----------------->   ____
/ \                       +""+
                          +__+
                         [ ==.]`)
                   +----+====== 0 +--+
                   +                 |
                Order #3           Job #3
                   |                 |
                Order #2           Job #2
                   |                 |
                Order #1           Job #1

Worker

The Worker uses node-scheduler to fire an HTTP request to the Boss letting it know the Worker is "bored." The Boss will then search through the Orders for the ...

Image source: Louis Reed
Using Python, NodeJS, Angular, and MongoDB to Create a Machine Learning System

I've started designing a system to manage data analysis tools I build.

  1. An illegitimate REST interface
  2. Interface for existing Python scripts
  3. Process for creating micro-services from Python scripts
  4. Interface for creating machine learning jobs to be picked up my free machines.
  5. Manage a job queue for work machines to systematically tackle machine learning jobs
  6. Data storage and access
  7. Results access and job meta data
  8. A way to visualize results

I've landed on a fairly complicated process of handling the above. I've tried cutting frameworks, as I know it'll be a nightmare to maintain, but I'm not seeing it.

  • Node for ...
Image source: Darius Bashar
Recording Brain Waves -- Mongo Database with a NodeJS API

Saving Brain Waves to Remote MongoDB by way of Node REST API

In this section I'm going to focus getting a remote Linux server setup with MongoDB and NodeJS. This will allow us to make POST requests to our Linux server, saving the EEG data.

I'm going to assume you are able to SSH into your Ubuntu 16 LTS server for this guide. You don't have a server? No sweat. I wrote a guide on setting up a blog post which explains how to get a cheap Linux server setup.

1. Install MongoDB

SSH into ...