UserPreferences

PyroVisionSystemFunctions


Pyro Vision System Functions

If you wish to practice with these functions, use the TutorialCamera. Either load the camera via the Device... option under Load main Pyro menu, or from the command line:

pyrobot -r Test -d TutorialCamera

You can use the Pause/Play options under the View menu of the camera0 window. Simply wait for the image that you wish to practice on, and select Pause.

To do these commands interactively:

camera = self.robot.camera[0]

Then you will have a camera object to work with interactively.

Methods

Pyro's camera inherits from pyrobot.camera and pyrobot.vision objects. pyrobot.vision has the following methods:

   def loadFromFile(self, filename):
   def saveToFile(self, filename):
   def getScaledImage(self, xscale=0.5, yscale='unset', mode='sample'):
   def getGrayScale(self):
   def getColorFilter(self, r, g, b):
   def display(self):
   def get(self, x, y, offset = 0):
   def getRow(self, y):
   def getDim(self):
   def getCol(self, x):
   def getVal(self, x, y):
   def setVal(self, x, y, val):
   def reset(self, vector):
   def resetToColor(self,r,g,b):
   def incr(self, x, y, offset = 0):
   def cropPixels(self, l, t='unset', r='unset', b='unset'):
   def getBitmap(self, cutoff, cutoff2='unset', mode='brightness'):
   def histogram(self, cols = 20, rows = 20, initvals = 0):
   def convolve(self, convmask, bit = 0, threshold = 0):
   def getPlane(self, colorPlane): # 0, 1, 2
   def swapPlanes(self, plane1, plane2):

pyrobot.camera has the following methods:

   def setFilterList(self, filterList):
   def popFilterList(self):
   def getFilterList(self):
   def loadFilters(self):
   def saveFilters(self):
   def getData(self):
   def stopMovie(self):
   def continueMovie(self):
   def startMovie(self, filename = None):
   def saveImage(self, filename = None):
   def saveAsTGA(self, path = "~/V4LGrab.tga"):
   def update(self):
   def updateOnce(self):
   def makeFilterMenu(self, data):
   def setTitle(self, title):
   def printit(self, event):
   def makeWindow(self):
   def togglePausePlay(self, event = None):
   def pauseButton(self):
   def playButton(self):
   def updateButton(self):
   def setUpdateInterval(self, val):
   def apply(self, command, *args):
   def addFilter(self, func, *args):
   def makeMenu(self, bar, name, commands, accel = None):
   def listCallbackList(self):
   def togglePlay(self, event):
   def toggleFilterMode(self):
   def getCanvasWidth(self):
   def getCanvasHeight(self):
   def processLeftClickDown(self, event):
   def processLeftClickUp(self, event):
   def processMiddleClick(self, event):
   def processRightClick(self, event):
   def hideWindow(self):
   def getImage(self):
   def updateWindow(self):
   def startDevice(self):
   def stopDevice(self):
   def getDeviceData(self):
   def getDeviceState(self):
   def updateDevice(self):
   def delFilter(self, pos):
   def popCallbackList(self):
   def clearCallbackList(self):
   def clearFilters(self):
   def processAll(self):

You can look at the code for those two classes: [WWW]pyrobot/camera/__init__.py and [WWW]pyrobot/vision/__init__.py.

Filters

The following are avaliable to the camera.apply() and camera.addFilter() methods. Use the addFilter() method for a one-time effect. Use the apply() method inside of another function that itself is added to the filter list via the addFilter() method.

camera.apply("match" red, green, blue, tolerance, outChannel) 
camera.apply("matchRange", red1, green1, blue1, red2, green2, blue2, outChannel) 
camera.apply("matchList", [(red1, green1, blue1, red2, green2, blue2, outChannel), ...]) 
camera.apply("superColor", redWeight, greenWeight, blueWeight, outChannel) 
camera.apply("threshold", channel, thresholdValue) 

camera.apply("setPlane", colorChannel, value)
camera.apply("set", x, y, r, g, b)
 
camera.apply("blobify", inChannel, lowValue, highValue, sortmethod (0 = square area, 1 = mass), blobCountToReturn, drawBox) 
camera.apply("histogram", x1, y1, x2, y2, bins) 
 
camera.apply("meanBlur", kernelEdgeSize) 
camera.apply("gaussianBlur") 
camera.apply("medianBlur", kernel) 
camera.apply("sobel", kernel) 
 
camera.apply("motion") 
camera.apply("backup") 
camera.apply("restore") 
camera.apply("inverse") 
 
camera.apply("grayScale") 
camera.apply("scale", redScale, greenScale, blueScale) 
camera.apply("drawRect", x1, y1, x2, y2, fill, channel) 

All of the filters store their return values in self.robot.camera[0].filterResults which is a list.

"match"

This function filters the image for a specific color with one tolerance being used for all of the RGB values.

Parameters:

Example calls:

  camera.addFilter("match", 215,215,70, 40,2)

http://www.cs.uml.edu/robots/PyroImages/OneTolBefore70.gif http://www.cs.uml.edu/robots/PyroImages/OneTolAfter70.gif

"matchRange"

This function filters for color values within specified ranges. Points that have values between the low and high ranges specified for red, green and blue will be marked in the buffer as containing the color.

Parameters:

Example calls:

  camera.addFilter("matchRange", 70,30,30, 117,60,60, 1)   # this you do just once
  camera.apply("matchRange", 70,30,30, 117,60,60, 1)       # this you do inside a loop

http://www.cs.uml.edu/robots/PyroImages/foo3.gif http://www.cs.uml.edu/robots/PyroImages/foo.gif

  camera.addFilter("matchRange", 0,30,30, 255,60,60, 1)

http://www.cs.uml.edu/robots/PyroImages/HiLowBefore0255.gif http://www.cs.uml.edu/robots/PyroImages/HiLowAfter0255.gif

  camera.addFilter("matchRange", 225,225,0, 255,255,200, 0)

http://www.cs.uml.edu/robots/PyroImages/ThreeTolBefore200.gif http://www.cs.uml.edu/robots/PyroImages/ThreeTolAfter200.gif

"matchList"

This function filters for a list of color values within specified ranges, like matchRange.

Parameters:

Example calls:

  camera.addFilter("matchList", [(70,30,30, 117,60,60, 1), (125,125,125, 135,135,135, 0)])   # this you do just once
  camera.apply("matchList", [(70,30,30, 117,60,60, 1), (125,125,125, 135,135,135, 0)])       # this you do inside a loop

"superColor"

Lets say you wanted to filter for red. If you just look at the red component and filter any pixel that has a red value over 180, for instance, then any white pixel will be filtered as a red pixel, because white has a red value of 255. What superColor does, is subtracts out the secondary colors from the main color. So if you want to filter on red, the green and blue values get subtracted from the red value resulting in an overall value from 0-255.

Parameters:

Example calls:

  camera.addFilter("superColor", 1, -1, -1)

http://www.cs.uml.edu/robots/PyroImages/superRedBefore.jpg http://www.cs.uml.edu/robots/PyroImages/superRedNoLightAfter.jpg

  camera.addFilter("superColor", 1, -1, -1, 1)

http://www.cs.uml.edu/robots/PyroImages/superRedBefore.jpg http://www.cs.uml.edu/robots/PyroImages/superRedLightAfter.jpg

camera.meanBlur

This function blurs the image by changing the color of the pixel being looked at to the mean value of the pixels surrounding it. The number of surrounding pixels being looked at is defined by the kernel parameter. If kernel is 3, then the pixel being looked at is the center of a 3x3 box, shown in the diagram.

http://www.cs.uml.edu/robots/PyroImages/meandiagram.jpg

Parameter:

Example calls:

  camera.addFilter("meanBlur")

http://www.cs.uml.edu/robots/PyroImages/MeanBefore3.gif http://www.cs.uml.edu/robots/PyroImages/MeanAfter3.gif

  camera.addFilter("meanBlur", 9)

http://www.cs.uml.edu/robots/PyroImages/MeanBefore3.gif http://www.cs.uml.edu/robots/PyroImages/MeanAfter9.gif

camera.gaussianBlur

The gaussianBlur function blurs the image by changing the current pixel's values by weighting the values of the pixels around it. The kernel size is 3, and cannot be changed.

Parameters:

Example call:

  camera.addFilter("gaussianBlur")

http://www.cs.uml.edu/robots/PyroImages/GaussBefore.gif http://www.cs.uml.edu/robots/PyroImages/GaussAfter.gif

camera.medianBlur

This function blurs the image by changing the color of the pixel being looked at to the median value of the pixels surrounding it. The number of surrounding pixels being looked at is defined by the kernel parameter. If kernel is 3, then the pixel being looked at is the center of a 3x3 box, shown in the diagram.

http://www.cs.uml.edu/robots/PyroImages/mediandiagram.jpg

Parameter:

/!\ Currently, MedianBlur runs very slowly and should only be used for learning about the differnt types of blurs available. Also, the larger the kernel size, the slower it will run.

Example calls:

  camera.addFilter("medianBlur", 5)

http://www.cs.uml.edu/robots/PyroImages/MedianBlurBefore.jpg http://www.cs.uml.edu/robots/PyroImages/MedianBlurAfter5.jpg

  camera.addFilter("medianBlur", 9)                                    

http://www.cs.uml.edu/robots/PyroImages/MedianBlurBefore.jpg http://www.cs.uml.edu/robots/PyroImages/MedianBlurAfter9.jpg

  camera.addFilter("medianBlur", 11)

http://www.cs.uml.edu/robots/PyroImages/MedianBlurBefore.jpg http://www.cs.uml.edu/robots/PyroImages/MedianBlurAfter11.jpg

blobify

This function finds up to the five largest blobs of an image by either mass or area. The color channel and the high and low values are passed in. "blobify" searches the image and keeps track of the all the blobs in the image. It then sets the upper left and lower right coordinates, as well as the mass of these one to five blobs into a list named blob. Optionally, white boxes are drawn around these blobs for identification.

Parameter:

Example calls:

  camera.addFilter("blobify", 1,1,255,0)

http://www.cs.uml.edu/robots/PyroImages/BeforeBlob1.gif http://www.cs.uml.edu/robots/PyroImages/AfterBlob1.gif

  camera.addFilter("blobify",2,1,255,0)

http://www.cs.uml.edu/robots/PyroImages/BeforeBlob2.gif http://www.cs.uml.edu/robots/PyroImages/AfterBlob2.gif

Edge Detection

This function uses the sobel method to detect the edges of an image.

  camera.addFilter("sobel")

http://www.cs.uml.edu/robots/PyroImages/MtDew_blur.jpeg http://www.cs.uml.edu/robots/PyroImages/MtDew_edges.jpeg

Examples

1. This is a demonstration of the robot blurring an image, training on the color in the center of the image, filtering for that color, and then finding the max blob. This examples expects for you to have a robot with a camera and ptz already loaded.

from pyrobot.brain import Brain  

def process(camera):
        camera.apply("gaussianBlur")
        camera.apply("match", 187, 50, 50)
        camera.apply("blobify", 1, 1, 255, "mass", 1)

class CameraTest(Brain): 
    def setup(self): 
        print "initializing camera ..." 
        self.camera[0].addFilter( process )
        self.ptz[0].panTilt(0, -10)  #look down slightly
   
    def step(self): 
        print "Robot stepping..."

def INIT(engine): 
    return CameraTest('CameraTest', engine) 

Exercises

1. Comparing filters.

2. Follow that Ball.

4. Pick Up the Trash.

Next: Simulated Vision using FakeCamera Up: PyroModuleComputerVision