Pyro Vision Functions for the Pioneer

Using a Pioneer with a Camera

The camera should be plugged into the Pioneer, and the green light on the front of the camera should be on. If the light isn't on, check to make sure the power switch on the back of the camera is on.

Because of the way the connection to the camera is currently implemented, if there is a bug in your program that prevents it from loading, you must exit Pyro, correct the bug, and then re-invoke Pyro. Reloading your edited Brain program will not work. Also in these cases, Pyro may not shut down cleanly leaving processes running. You can suspend these processes with Control-Z and then do a killall -9 pyro to kill these hung processes.

Pyro Vision Functions

The following functions can be used in Pyro with the Video for Linux (V4L) frame grabber on the Pioneers. They are defined in /usr/local/pyrobot/camera/v4l/init.py.

• colorFilterHiLow

• colorFilterOneTol

• colorFilterThreeTol

• superColor

• trainColor

• gaussianBlur

• meanBlur

• medianBlur

• maxBlob

• saveImage

To start the V4L service:

        self.getRobot().startService("V4LCamera")
        self.camera = self.getRobot().getService("V4LCamera")
        self.camera.makeWindow()

The image that the camera captures is dumped to a shared memory buffer. This buffer can be accessed by both Pyro and C. For speed reasons, the vision processing is done in C. All the changes to the image are done by manipulating this shared memory buffer, which is then read by pyro to display the newly modified image.

The frame grabber grabs images in a different thread then what the program is running in, so it updates itself reguardless of where the program is executing. This can cause wierd things to happen. Lets say you run a color filter to keep only red items in the image, and halfway through the filter process, the framegrabber updates the image. This would cause the filtering that has just been done to be over written, and the filter will continue to filter the bottom half of the image. Therefore, you would be left with an image with the top half not filtered and the bottom half being filtered for red. To stop this, the camera has a flag called lockCamera. If this flag is set, it will stop the framegrabber from being able to update the shared memory so that all the filtering, blobbing, etc can be done on the same image. Here is an example:

        self.camera.lockCamera = 1 #lock the buffer
        self.camera.superColor("red",0)
        self.camera.maxBlob(2, 1, 255, "mass")
        self.camera.lockCamera = 0 #unlock the buffer

Once you are done manipulating the image and have the information you need, setting lockCamera to 0 will allow the framegrabber to update the images again.

Another flag that the camera has is the sleepFlag. If set, the sleepFlag will cause the program to sleep, for what sleepTime is set to, after filtering processes have been completed. This allows the effects to be seen in the video window. This is mostly for testing purposes to make sure that the filter does what you want it to do. Here is an example:

        self.camera.sleepTime=2.0  #sleep for 2 seconds after filters
        self.camera.sleepFlag = 1  #turn sleep flag on
        self.camera.lockCamera = 1 #lock the buffer
        self.camera.superColor("red",0)
        self.camera.maxBlob(2, 1, 255, "mass")
        self.camera.lockCamera = 0 #unlock the buffer

General explanation of use of channel for processing.

Talk about problems with color filtering, and vision in general.

camera.colorFilterHiLow

This function filters for color values within specified ranges. Points that have values between the low and high ranges specified for red, green and blue will be marked in the buffer as containing the color. After using this function, you would probably call maxBlob.

Parameters:

• low red value (0-255)

• low green value (0-255)

• low blue value (0-255)

• high red value (0-255)

• high green value (0-255)

• high blue value (0-255)

• channel (optional parameter, defaults to 1 for green channel, pass 0 for red, 1 for green or 2 for blue)

Return Values:

• none

Example calls:

  self.camera.colorFilterHiLow(70,30,30,255,60,60)

  self.camera.colorFilterHiLow(0,30,30,255,60,60)

camera.colorFilterOneTol

This function filters the image for a specific color with one tolerance being used for each of the RGB values.

Parameters:

• red value (0-255)

• green value (0-255)

• blue value (0-255)

• tolerance value (0-255) (optional parameter, defaults to 30)

• channel (optional parameter, defaults to 1 for green channel, pass 0 for red, 1 for green or 2 for blue)

Return Values:

• none

Example calls:

  self.camera.colorFilterOneTol(215,215,70,40,2)

camera.colorFilterThreeTol

This function also filters the image for a specific color, but the user has the option of setting a specific individual tolerance for R,G and B.

Parameters:

• red value (0-255)

• green value (0-255)

• blue value (0-255)

• red tolerance value (0-255) (optional parameter, defaults to 30)

• green tolerance value (0-255) (optional parameter, defaults to 30)

• blue tolerance value (0-255) (optional parameter, defaults to 30)

• channel (optional parameter, defaults to 1 for green channel, pass 0 for red, 1 for green or 2 for blue)

Return Values:

• none

Example calls:

  self.camera.colorFilterThreeTol(255,255,0,30,30,200,0)

camera.superColor

Lets say you wanted to filter for red. If you just look at the red component and filter and color that has a red value over 180, for instance, then any white pixel will be filtered as a red pixel, because white has a red value of 255. What super color does, is subtracts out the secondary colors from the main color. So if you want to filter on red, the green and blue values get subtracted from the red value resulting in an overall value from 0-255. Super Color is only for Red, Green and Blue (the three primary colors). The return value is between 0-255 so you have the ability to choose how red, green, or blue you want to filter for.

Parameters:

• color: "red", "green", or "blue"

• channel: (0-2) channel to put the resulting values into. puts zeros in the other two channels.

• lighten: (0-1) (optional parameter, defaults to 0.) Often the resulting value is low and not visable against the black background. The lighten parameter will add 60 to final result to make the pixel brighter and easier to see for testing.

Return Values:

• none

Example calls:

Before:

  self.camera.superColor("red",0)

  self.camera.superColor("red",0,1)

camera.trainColor

The train color function takes a histogram of the colors in a 50x50 pixel box in the center of the image. It takes the average color of the peak and that is the color that it returns. This function is useful for determining what overall RGB value an object has. It is also good for passing into a filter function to beable to filter for the color that was just trained on.

Parameters:

• none

Return Values:

• histRed: Red Value of Trained Color

• histGreen: Green Value of Trained Color

• histBlue: Blue Value of Trained Color

Example call(s).

  self.camera.trainColor()
  self.camera.colorFilterOneTol(self.camera.histRed,self.camera.histGreen, self.camera.histBlue, 30)

camera.gaussianBlur

The gaussianBlur function blurs the image by changing the current pixel's values by weighting the values of the pixels around it. The kernel size is 3, and cannot be changed.

Parameters:

• none

Return Values:

• none

Example call:

  self.camera.gaussianBlur()

camera.meanBlur

This function blurs the image by changing the color of the pixel being looked at to the mean value of the pixels surrounding it. The number of surrounding pixels being looked at is defined by the kernel parameter. If kernel is 3, then the pixel being looked at is the center of a 3x3 box, shown in the diagram.

Parameter:

• kernel (optional parameter, defaults to 3. Only enter in odd numbers greater then 0)

Return Values:

• none

Example calls:

Before:

  self.camera.meanBlur()

  self.camera.meanBlur(9)

camera.medianBlur

This function blurs the image by changing the color of the pixel being looked at to the median value of the pixels surrounding it. The number of surrounding pixels being looked at is defined by the kernel parameter. If kernel is 3, then the pixel being looked at is the center of a 3x3 box, shown in the diagram.

Parameter:

• kernel (optional parameter, defaults to 3. Only enter in odd numbers greater then 0 and less then 20)

Return Values:

• none

Currently, MedianBlur runs very slowly and should only be used for learning about the differnt types of blurs available. Also, the larger the kernel size, the slower it will run.

Example calls:

Before:

  self.camera.medianBlur(5)

  self.camera.medianBlur(9)

  self.camera.medianBlur(11)

camera.maxBlob

This function finds the maximum blob of an image by either mass or area. The color channel and the high and low values are passed in. MaxBlob searches the image and keeps track of the all the blobs in the image. It then returns the upper left and lower right coordinates, as well as the mass of the maximum blob. A white box is then drawn around this max blob for identification purposes during testing.

Parameter:

• channel: color channel to look for blobs on

• low value(0-255): minimum value to look for as part of a blob

• high value(0-255): maximus value to look for as part of a blob

• drawBox (0-1): (optional parameter, defaults to 1), This option allows the user to stop drawing the box around the max blob, to save processesing time.

Return Values:

• none

Example calls:

  self.camera.maxBlob(1,1,255)

  self.camera.maxBlob(2,1,255)

Edge Detection

This function uses the sobel method to detect the edges of an image.

  self.camera.edgeDetection()

camera.saveImage

This function will save the image that is in the processing buffer to a ppm file. Seeing as how ppm files are quite large, you might want to convert the ppm files to a different image format. The best way to do this is to use an image editor (such as xv) and use Save As to save the image in a different format. Another way is to use xview with the following command:

xview -dump jpeg yourfile.jpeg yourfile.ppm

The unix command ppmtogif can also be used to convert your ppm file to the gif format, however, if you try to use ppmtogif to convert your file you may get an error that there are too many colors. You may try the following, but the colors in the resulting image may not match the colors of the original image in some spots, which is why we recommend using an image editing program.

ppmquant 256 yourfile.ppm | ppmtogif > yourfile.gif

Parameters:

• filename: a string with the name of the file to save the image to, can include a path, name defaults to image.ppm in the current directory

Return Values:

• 1 if image was saved properly

• 0 if there was an error during the save

Example call:

  self.camera.saveImage("myPicture.ppm")

Examples

1. This is a demonstration of the robot blurring an image, training on the color in the center of the image, filtering for that color, and then finding the max blob.

from pyrobot.brain import Brain
from pyrobot.brain.behaviors.core import *  # Stop

import time

class CameraTest(Brain):
    def __init__(self, name, robot):
        Brain.__init__(self, name, robot)
        print "initializing camera ..."
        self.getRobot().startService("V4LCamera")
        self.camera = self.getRobot().getService("V4LCamera")
        self.camera.makeWindow()

        print "done initializing camera"

    def step(self):
        self.camera.lockCamera = 1
        self.camera.sleepFlag = 1
        self.camera.sleepTime = 1.0

        print "Blurring Image"
        self.camera.gaussianBlur()

        print "Training on a color"
        self.camera.trainColor()
        print "Trained on color r:", self.camera.histRed,
        print " g:", self.camera.histGreen,
        print " b:", self.camera.histBlue

        print "Filtering Image"
        self.camera.colorFilterOneTol(self.camera.histRed,self.camera.histGreen,self.camera.histBlue )

        print "Blobbing Image"
        self.camera.maxBlob(1, 1, 255)

        self.camera.flag = 0
        sleep(0.5)

        print "\n"

def INIT(robot):
    return CameraTest('CameraTest', robot)

2. In this example, the robot uses superColor to find red, blobs the image, then tracks the largest blob by moving the camera using the PTZ service.

from pyrobot.brain import Brain
from pyrobot.brain.behaviors.core import *  # Stop

import time

class CameraTest(Brain):
    def __init__(self, name, robot):
        Brain.__init__(self, name, robot)
        print "initializing camera ..."
        self.getRobot().startService("V4LCamera")
        self.camera = self.getRobot().getService("V4LCamera")
        self.camera.makeWindow()

        self.getRobot().startService("ptz")
        self.ptz = self.getRobot().getService("ptz")

        self.ptz.pan(0)
        self.ptz.tilt(0)
        self.ptz.zoom(0)

        self.pan = 0
        self.tilt = 0


        print "done initializing camera"

    def step(self):
        self.camera.lockCamera = 1

        self.camera.superColor("red",2)

        print "Blobbing Image"
        self.camera.maxBlob(2, 1, 255, "mass", 0)

        self.camera.lockCamera = 0

        if( self.camera.mass > 10 ):
            self.pan = (( self.camera.min_x + self.camera.max_x ) / 2 -
                        (self.camera.width/2)) / 10
            self.tilt = -1*(( self.camera.min_y + self.camera.max_y ) / 2 -
                            (self.camera.height/2)) / 10

            self.ptz.panTiltRel(self.pan, self.tilt)

            print "PAN IS ", self.pan
            print "TILT IS ", self.tilt

        print "\n"

def INIT(robot):
    return CameraTest('CameraTest', robot)

Exercises

1. Follow that Ball.

Write code for your robot that will search for a ball of a specified color. Once the robot finds the ball, it should move towards it. If you move the ball, the robot should relocate the ball and move towards it again.

Extension to the exercise: Robot Soccer. Have the robot push the ball towards a goal to play soccer.

2. Pick Up the Trash.

Develop a system that performs the task from the AAAI 1994 Pick Up the Trash Competition (find reference). Your robot will need to find trash of a particular color (you can choose a can of soda, a cup, or some other object that your gripper can pick up). Once it finds the trash, the robot should move towards it and pick it up with the gripper. Then the robot should find the trash can and deliver the object.

Start with one color trash and one trash can. As an extension, create an environmentally conscious robot that will deliver trash to the trash can and recyclables to the recycling bin. (It will be easiest to complete this task using four distinct colors.)

[PyroModulesContents]

Immutable page (last edited 2007-03-27 00:17:38 by LucasJustason)

Or try one of these actions: LikePages, LocalSiteMap, SpellCheck