OpenCV on a Raspberry Pi

Originally posted on


No longer afeared of frying my Pi, I’ve moved on to trying to implement some of my bot goals. Like many, I want my bot to be able to interact with people, but I didn’t realize that I’d stumble on this ability.

I’ve looked at many visual processing boards like the CMUcam v4, but I’m not paying $100 for any board. I looked into making one, it looks possible, but not much cheaper. So, I got curious as to what alternatives there are. I stumbled on Hack-a-Day’s recommended article: OpenCV on Raspberry Pi.

Anyway, he provided instructions on setting up OpenCV (open source computer vision) on Raspberry Pi. Of course, it was about 20 minutes later I had the code working on my Pi.

I had been skeptical of the Pi’s ability to run any computer vision software, and morever, it’s usefulness given the Pi’s processing constraints. But once I had it up and running, I noticed it actually ran smoother than I had hoped. Don’t get me wrong, I think it is less than 10FPS, but I could tell it would work for many robot applications More than that, if the Raspberry Pi was used only for the computer vision, then it would still be cheaper than many other hardware driven CV boards.

Basic Raspberry Pi and WiFi Dongle

  • WiFi Dongle: $6.17
  • Raspberry Pi: $35.00
  • SD Card (4g): $2.50
  • Web cam: $8.00
  • Total for Basic RPi: $51.67

Therefore, I went to work on hacking his code.

Many hours later, I ended up with a _very crude _ Raspberry Pi, Ardy, Camera, and Servo orchestration to track my face. Mind you, this is a proof of concept, nothing more at this point. But I hope to eventually have my bot wandering around looking for faces.

Image of Pi VNC. The box outline is being written through i2c.

Pulling apart a little $8 eBay camera.

To Setup the Raspberry Pi:

If you’re setting it up from sratch, start with these instructions.

But if you’re already setup, I think all you need is OpenCV.

$ sudo apt-get install python-opencv

The Code:

The Arduino code reads bytes from the i2c, converts them to characters, then places the characters into an integer array. The Pi is sending 4 numbers, 2 coordinates, x1, y1, x2, y2.

The Python code is “” by Roman Stanchak and James Bowman, I’ve merely added lines 101-105, which load the coordinates of the box around your face into a a string, converts that to a string array. I also added function txrx_i2c(). This function converts the string array into bytes and sends it to the i2c bus.

To change this setup from i2c to UART, focus on the txrx_i2c() in the Python code and the onRead() in the Arduino code. I assure you, UART would be much easier.

If anyone has any questions hollar at me. Oh! And if someone can tell me ways I could optimize this code, I’m all ears

#include <Wire.h>
#define SLAVE_ADDRESS 0x2A
#include <Servo.h>

Servo CamServoX; //Attach the pan servo.
Servo CamServoY; //Attach the tilt servo.

int ServoTimer = 250; // Change to adjust how quickly the servos respond.

int SmallXJump = 3; //Sets the movement amount for small pan jumps
int LargeXJump = 7; //Sets the movement amount for large pan jumps

int SmallYJump = 1; //Sets the movement amount for small pan jumps
int LargeYJump = 2; //Sets the movement amount for large pan jumps

//How close your face is to the edge to trigger a jump.
int SmallYLimit = 40;
int LargeYLimit = 20;

int SmallXLimit = 40;
int LargeXLimit = 20;

//Set servos to initial position.
int posX = 90; //Servo position.
int posY = 90; //Servo position.

int x1; int y1;int x2; int y2; //Holders for frame dimesions.

// Indexes for getting i2c bytes, then, converting them to integers.
int i = 0;
int varI = 0;

//Sets flag to trigger ServoWrite() from the main loop.
//I tried to put this under 'onRequest' call, but the Raspberry Pi kept giving me errors.
//This flagging was a work around.
int NoServoData = 0;

int dim[12]; //Char array for char[] ---> int conversion.
char d[8]; // Char holder array for byte-->char conversion.

void setup() {
    // initialize i2c as slave

    //Attach servos
    CamServoX.attach(10); //Tilt (Y)
    CamServoY.attach(9); //Pan (X)

    //Write initial servo position.

void loop() {

//Again, this is the work around.  The flag "NoServoData" is set under the i2c onReceive.
if (NoServoData==1){


//This is just to show the RPi can be written to.  
//Replace with stuff you want to write to the Pi.
char data[] = "Pasta";  
int index = 0;

// callback for sending data
void sendData() {
    if (index >= 5) {
         index = 0;

// callback for receiving data.
void readData(int numbytes) {

//Holds the chars
int c;

if (Wire.available() > 0){
  while(Wire.available())    // slave may send less than requested
    c =;
  //Add each integer to a char array.
  //Skip commas ',' and keep adding the integers until char '\0' is received.
  //Then print out the complete string.

  if (c != ','){
    if(c != '\0'){
      d[i] = d[i] + c;  //Appends the characters to an array.
    i=0; //Reset the d char array index.
    if(varI < 7){  //We only want to get integers until we get all four numbers (x1, y1, x2, y2) plus
      dim[varI]=atoi(d); //Convert the d int into ASCII and store it in the dim array.
      d[0]=0;d[1]=0;d[2]=0;d[3]=0;d[4]=0;d[5]=0; //Clear the d array (i2c doesn't like for loops in this function
      varI++; //Increase dim index.
      //We now have all four numbers, load them into the variables.

      NoServoData = 1;  //Set the WriteServo() call flag.
      varI=0; //Reset the dim index to prepare for next set of numbers.
   i=0; //Reset some

void ServoWrite(){
  int x3 = 160 - x2; // Calculate the distance from the right edge of the screen
  int y3 = 120 - y2; // Calcualte the distance

  //For X Axis
  if(x1 < SmallXLimit ){  //Only do small jumps, since not too far away from the edge.
        if(posX>1){ //If the pan servo is at its edge, do nothing.
          for (int i = 0; i < LargeXJump; i++){
            posX++;  // Set the new position
            CamServoX.write(posX); //Make the adjustment.
            delay(ServoTimer); //Delay between servo increments.

  if(x3 < SmallXLimit){
          for (int i = 0; i < LargeXJump; i++){

  if(x1 < LargeXLimit){
          for (int i = 0; i < SmallXJump; i++){

  if(x3 < LargeXLimit){
        for (int i = 0; i < SmallXJump; i++){

  //For Y Axis
  if(y1 < SmallYLimit ){
          for (int i = 0; i < SmallYJump; i++){

  if(y3 < SmallYLimit){
        for (int i = 0; i < SmallYJump; i++){

  if(y1 < LargeYLimit){
          for (int i = 0; i < LargeYJump; i++){

  if(y3 < LargeYLimit){
        for (int i = 0; i < LargeYJump; i++){

//Reset servo write flag.

Now for the Python Code:

Have to execute using "sudo python --cascade=face.xml 0"
(Normal build sudo python "%f")
This program is demonstration for face and object detection using haar-like features.
The program finds faces in a camera image or video stream and displays a red box around them.

Original C implementation by:  ?
Python implementation by: Roman Stanchak, James Bowman
import sys
import as cv
from optparse import OptionParser
import time
import threading
import readline
import pygame
from pygame.locals import *
import sys
import smbus

# Parameters for haar detection
# From the API:
# The default parameters (scale_factor=2, min_neighbors=3, flags=0) are tuned
# for accurate yet slow object detection. For a faster operation on real video
# images the settings are:
# scale_factor=1.2, min_neighbors=2, flags=CV_HAAR_DO_CANNY_PRUNING,
# min_size=<minimum possible face size

min_size = (20, 20)
image_scale = 2
haar_scale = 1.2
min_neighbors = 2
haar_flags = 0

"""i2c Code"""
bus = smbus.SMBus(1) # Open up a i@C bus.
address = 0x2a # Setup Arduino address

sendstring = "" # This will be my send variable (RPI-to-Arduino)
bytearraytowrite = [] #Actual array for holding bytes after conversion from string.

#This function actually does the writing to the I2C bus.
def toWrite(a):
	global sendstring
	global bytearraytowrite
	bytearraytowrite = map(ord, sendstring) #This rewrites the string as bytes.
	for i in a:
		bus.write_byte(address, i)

def txrx_i2c():
	global sendstring
	#while True:
	sdata = ""
	rdata = ""
	for i in range(0, 5):
			rdata += chr(bus.read_byte(address));
	#print rdata
	#print bytearraytowrite
	#print "".join(map(chr, bytearraytowrite)) #Will convert bytearray to string.

	#Writes the key commands to the i2c bus.


def detect_and_draw(img, cascade):
    global sendstring

    # allocate temporary images
    gray = cv.CreateImage((img.width,img.height), 8, 1)
    small_img = cv.CreateImage((cv.Round(img.width / image_scale),
			       cv.Round (img.height / image_scale)), 8, 1)

    # convert color input image to grayscale
    cv.CvtColor(img, gray, cv.CV_BGR2GRAY)

    # scale input image for faster processing
    cv.Resize(gray, small_img, cv.CV_INTER_LINEAR)

    cv.EqualizeHist(small_img, small_img)

        t = cv.GetTickCount()
        faces = cv.HaarDetectObjects(small_img, cascade, cv.CreateMemStorage(0),
                                     haar_scale, min_neighbors, haar_flags, min_size)
        t = cv.GetTickCount() - t
        print "detection time = %gms" % (t/(cv.GetTickFrequency()*1000.))
        if faces:
            for ((x, y, w, h), n) in faces:
                # the input to cv.HaarDetectObjects was resized, so scale the
                # bounding box of each face and convert it to two CvPoints
                pt1 = (int(x * image_scale), int(y * image_scale))
                pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
                cv.Rectangle(img, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)
                x1 = int(x * image_scale)
                y1 = int(y * image_scale)
                x2 = int((x + w) * image_scale)
                y2 = int((y + h) * image_scale)
                sendstring = str(x1) + "," + str(y1) + "," + str(x2) + "," + str(y2) + ","
                sendstring = sendstring.translate(None, '() ')
                print sendstring
                sendstring = ""
    cv.ShowImage("result", img)

if __name__ == '__main__':

    parser = OptionParser(usage = "usage: %prog [options] [filename|camera_index]")
    parser.add_option("-c", "--cascade", action="store", dest="cascade", type="str", help="Haar cascade file, default %default", default = "../data/haarcascades/haarcascade_frontalface_alt.xml")
    (options, args) = parser.parse_args()

    cascade = cv.Load(options.cascade)

    if len(args) != 1:

    input_name = args[0]
    if input_name.isdigit():
        #Where the image is actually captured from camera. "capture" is the variable holding image.
        capture = cv.CreateCameraCapture(int(input_name))
        capture = None

    cv.NamedWindow("result", 1)

    width = 160 #leave None for auto-detection
    height = 120 #leave None for auto-detection

    if width is None:
    	width = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH)) #Gets the width of the image.
    	cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_WIDTH,width) #Gets the width of the image.

    if height is None:
	height = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT))

    if capture: #If "capture" actually got an image.
        frame_copy = None
        while True:

            frame = cv.QueryFrame(capture)
            if not frame:
            if not frame_copy:
                frame_copy = cv.CreateImage((frame.width,frame.height),
                                            cv.IPL_DEPTH_8U, frame.nChannels)

#                frame_copy = cv.CreateImage((frame.width,frame.height),
#                                            cv.IPL_DEPTH_8U, frame.nChannels)

            if frame.origin == cv.IPL_ORIGIN_TL:
                cv.Copy(frame, frame_copy)
                cv.Flip(frame, frame_copy, 0)

            detect_and_draw(frame_copy, cascade)

            if cv.WaitKey(10) >= 0:
        image = cv.LoadImage(input_name, 1)
        detect_and_draw(image, cascade)

Pi Power -- How I Made a Battery Powered USB Hub

Originally posted on

As I prepare to start adding peripherals to my Pi Bot, I wanted to be sure to get around the 700ma power budget the Pi has. After searching for a cheap battery powered USB hub and finding little, I decided to hack up a few cheap(ish) parts and make my own.

  1. USB Hub: $1.39

  2. 5000mAh Battery: $17.93

  3. DC-DC Converter: $2.76

Total: $22.08

The Battery Hack:

1. Crack it open.

2. Find POWER and GND.

3. Wire it up.

4. Make a small hole for wires and bring wires out.

5. Solder the respective leads to the DC-DC converter.

6. Smile, then sit through my way too long of a video to make it into the HUB.

Hope all are well. :)

NOTE: Regarding the error at the end of the video. Don’t panic (that’s what I did). I actually found out this had nothing to do with my hub, it had to do with plugging an iPhone into a Raspberry Pi.

NOTE2: I realize I used the wrong “hearty,” my brain has problems typing homonyms and parahomonyms. :P

Blueberry Pi -- How I Setup My Raspberry Pi as a Robot Base

Originally posted on

This article is specific:How I personally would setup my Raspberry Pi to act as robot base. But, I’ll be clear, this is one of nth possible setups. A chessboard has 64 squares but those working the board allow for innumerable possibilities.

That aside, here we go:

1. Get Berryboot. Berryboot will allow you to download several Raspberry Pi images.

Now extract the zip files to a blank SD card.

Put the BerryBoot SD card in your Pi and boot it up.

2. Setup RPi with Raspbian Wheezy (first option).

3. Setup your WiFi dongle. I believe BerryBoot will now setup your WiFi dongle on initial boot, which it did for me (even gave me the option to download the image via WiFi). But, I had trouble getting my WiFi dongle pulled up after booting Raspbian Wheezy.

If you have difficulty with manual WiFi dongle setup, you might try this video.

Lastly, if you are looking for a WiFi dongle for cheap, with good range, and uses very little mAhs (the Pi can only feed about 700mAhs through the USB port). You might try this one, $6.17.

4. Setup PuTTY on your Desktop Computer. Follow this video.This will allow you to begin SSHing into the Pi. That way you don’t have to look at a little RCA screen like me. For those who aren’t familiar with SSH (like I was before this video), the video will explain it. At risk of oversimplification,it allows you to access your Raspberry Pi command line through your desktop.

You have to plug in your Pi’s network number.You can find this by pulling up your wireless hub’s configuration page. You should see what address your Pi is listed at. For some strange reason, if it doesn’t list the device name, just view the page while the Pi is up, then unplug your Pi and refresh the wireless hub configuration page. The device that disappeared is your Pi. I’ve never had to change the port number, but beware you might need to depending on your setup.**

If you want to know whether your have the correct information, try login’ in and if you get a screen like this, your good.

Your username and password are by default:pi, raspberry

Remember! In the case of a Raspberry Pi, always share your password, ‘cause everyone has it anyway :)

Once you have PuTTY setup, you should be able to bring up your Pi command line, something like this:

5. Setup VNCServer on your Raspberry Pi. Follow this video. (Or this walkthrough). Putty will let you access your Pi’s command line, but setting up a VNC will actually allow you to access your Pi’s Desktop GUI from your PC, in the same manner as Putty.

**6. Setup a VNC Client on your Desktop Computer. Real VNC. **There are many different programs, I happened to end up using Real VNC.

Once you have VNC setup on both machines, PuTTY into your Pi and start the VNC server.

$sudo vncserver

Two notes here, if you did better with the video instructions than I did, your vncserver will start automatically on boot. Unfortunately, I have to type it each time (I’m too lazy to figure out the boot part of it). As a result, you’ll have problems running certain Python scripts through VNC if you don’t use $sudo vncserver

You’ll enter your Pi address, but port should be 1 (if I remember the video instructions correctly).

You should end up with at a windowed version of your Raspberry Pi desktop. One more note, somewhere in the video it gets you to setup the “geometry” of the VNC desktop. The limitations you put there will be reflected in the quality of the desktop you see in the window. In essence, if you put in 640x480, that’s the resolution this desktop will end up. So, please, take advantage of the Pi’s GPU :)

Use something like this, “-geometry 1024x728 -depth 24”

7. Resize your SD card to use all its space. (Note, this should already be done by BerryBoot. But other diskimages will limit your SD card to 2GB, regardless of its actual size).

8. Git manager will allow you to pull code from git hubs (again, this should already be installed, but just in case).

I**nstall the git manager: **

At Raspberry Pi prompt: **$sudo apt-get install git**

The way to use it is like so,

At Raspberry Pi prompt: **$sudo git clone**

9. **Install SMBus. **This is specifically for my setup, since I’ll be using the I2C bus to communicate between the Pi and the Arduino.

At Raspberry Pi prompt: **$sudo apt-get install python-smbus**

10. Any other Python modules you might fancy.

Useful for keystroke, GUI, and other interfacing needs:

Pygame (should come with Raspbian). (sudo apt-get install pygame)

Lady Ada’s Python codes for an array of I2C sensors:

Adafruit I2C library (git)

Access your Raspberry Pi from iDevice web based GUI:

PiUi (git)

Control serial devices:

pySerial (sudo apt-get install python3-pyserial)

(I’ll add other resources as fellow LMRs leave them in the comments).

11. (optional) Install Arduino IDE on Raspberry Pi. This will allow you to program the Arduino directly from your Pi–and if you follow my design, you’ll be able to do so without ever leaving your desktop computer. You can do this by opening the VNC Server, opening the Arduino IDE on the remote desktop, selecting the sketch you want to upload, and as long as your Arduino is connecting by way of USB, you can then upload your sketch from where you sit. This allows for quick changes to Arduino code without switching wires around. Also, I think Kariloy is looking for a way to upload sketches by way of GPIO pins. This would make a cleaner design.

**12. Install WinSCP. This will allow you to transfer files between your desktop and the Pi. **I find this helps with programming management. I’m a messy filer. If I file at all.

13. Take a deep breath.

14. Follow these instructions for making my I2C optoisolator board.

Again, there are many commercial boards that will serve the same function. Also, you can do the same with a USB cable, serial pins to GPIO, or RF connection–basically any way that lets the Arduino and Pi talk at a reasonable speed. The speed restraint will of course depend on your need. I doubt many methods will be apt for running a responsive quadrocopter. But in my case, my Pi is the central nervous system and the Arduino is the autonomous nervous system. The Pi will send directives, but it’s up to the Arduino to manifest them through responsive actuators. And I chose this optoisolator because I didn’t want an voltage restraint on my actuators or fear of frying my Pi.

Once you have the board setup, you can run:

$sudo i2cdetect -y -a 1

This should bring up a list of active I2C registers. You should find your Arduino at whatever address you set in your Arduino code.

Now, I’ve read this fellow’s article on how Raspberry Pi I2C pins are actually 5v tolerant. (Note, this is only for I2C pins, due to their pull-up resistors.)

So in theory, you can skip the optoisolator all together. But that’s you, I’ll stick with my optoisolation.

15. Download my code–or someone cooler’s.

Note, my code is really just the base for a robot. Right now, my it is nothing more than a very, very complex radio controller for a RC car. But someday, I’ll make a real robot :)

**16. Tweak and gut the code as you see fit. **

17. Ask questions: Pretty much everyone on this site is smarter than me, they’ll know the answer.

To other LMRians. Please feel free to tell me how to change, add, or retract from this article. As tired as I am right now, I plan to revise when I’m less muddled.

Arduino to RPi -- Galvanically Isolated I2C

Originally posted on

Breakout PCB Arduino Code

I’ve waited to finish incorporating my Raspberry Pi into my bot for an ample bit. But since I know so little about electricity, I swore to myself I wouldn’t add my Pi to my bot until I was absolutely sure I wouldn’t fry it.

Well, I’m still not “absolutely” sure, but I feel this little optoisolator has brought me a lot closer. This builds on my post a week or so ago about making Eagle parts.

I plan to actually list out what tweaks a Wheezy image needs to get this optoisolator build to work. It’s actually pretty easy–but whatever you, don’t be lured in by quick2wire. Those buggers wasted most of my day :(

If anyone has questions let me know.

Oh, one note. When I populated the board I used 4.7k resistors on the Arduino side, but I pulled off everything on the Raspberry Pi side. It seems the Pi has built in pull-ups that do the job rather well.

ADUM1250ARZ Datasheet

Hope everyone is well :)

Populating and Programming and APM

Originally posted on

I decided to try making an Arduino Pro Mini at home. Being done, it’s not worth it. You can buy one for a dollar more than you can make them, and it took awhile to populate. Although, it’s “fun.”

This projects was also a chance for me to test the Spying-Stalactite I built.

I’ve enjoyed it. It allows me to reflect on my strategy while populating boards. It’s simply a drop down with some high-powered LEDs (~2500 lumen), heatsink, and coolant fan. It has a hole for my iphone to do the recording. Cheap and simple. Although, I need to diffuse the light, as you might see by the video that it washes out the details of the projects. Also, I’ll add a few more lights and do away with the tungsten lamp, since the iphone is constantly in a white-balance battle as I move infront of the mixed lightsources.

I populated this board; everything came out fine (although, it was much more difficult trying not to block the camera with my head). I popped it into Atmel studio and it read out the device voltage and signature. Of course, I bricked it, as I seem to do a lot.

My next projects is a Fuse Doctor. :)

I had ordered the boards from OSHPark and had planned on making three. So, I populated another and took some time programming it. I’ve outlined my steps below:

1. Hook up the AVRISP MKII

2. Open Atmel Studio. Go to Tools – Device Programming.

3. Setup:

  • Tool: AVRISP mkII
  • Device: ATmega328P
  • Interface: ISP

Click apply

4. Read Target voltage (it should be ~5V). Read Device Signature.

  1. Open boards.txt that comes with Arduino (\Desktop\arduino-1.0.3\hardware\arduino\boards.txt).

  2. Scroll down to the area marked:

8. Pull the programming information for the board from this area. Now, I’ve bricked a few boards, but I think I’ve figured this one out. When programming this board with the MKII and Atmel Studio, you should follow this order.

1. Set the fuses:

  • Extended: 0xFD
  • High: 0xDA
  • Low: 0xFF
  • (Double check the board file to make sure I didn’t make typos)
  • Hit “Program”

2. Upload Bootloader.

“The bootloader for the 5v, 16mhz Arduino Pro Mini (which is what I built) is “ATmegaBOOT_168_atmega328.hex (Desktop\arduino-1.0.3\hardware\arduino\bootloaders\atmega\ATmegaBOOT_168_atmega328.hex).
It’s important to note that the 3.3v and 5v versions use different bootloaders.

  • Go to the Memories tab
  • Hit the browse ellipsis.
  • Select the “ATmegaBOOT_168_atmega328.hex”
  • (Double check the boards file to make sure I’m not screwing you up).
  • Hit program.

3 Set Lock Bits.

  • Go to the “Lock bits” tab.
  • Check the boards.txt file for Lockbit number
  • Lockbit: 0xCF
  • (Double check the boards.txt. I don’t take blame for bricked boards :P).
  • Hit “Program”

9 Upload the Blink Sketch; the LED by the reset button should blink.

10 Let me know how it went. If you bricked a chip using these instructions, let me know so I can modify them quick.

Now that I’m used to the camera and stalactite, I plan to annotate my next board for tips on working with 0402s.

Hope all are well.

ps. Birdmun et al., sorry bout the copyright issues. Not a professional at anything, especially video editing :)