UserPreferences

Pyro Vision Functions using Phission

Phission Control Functions

Phission is a new high-performance image processing system for Pyro. It is written in C++, but has methods in Python to access the underlying code. This system is very fast, but has a bit of a learning curve due to its sophisticated functions.

Control Functions are in charge of Phission utilities, such as starting and stopping Phission. They are usually used in class constructors and destructors, as well as from the command line. 

setup(capturechannel, capturewidth = 320, captureheight = 240)

destroy()

pause()

play()

Filter Functions

Filter Functions do the actual image manipulations. These functions can be called to apply edge detection, blurring, histograming, and motion filtering to the video stream itself. Calling any of these functions will remove all previous filters that have been applied. Any of these filters will continue to run until a new filter (or no_Filter) is inserted. 

no_Filter()

canny_Filter()

sobel_Filter()

gaussianBlur_Filter()

medianBlur_Filter()

histogramRGB_Filter(x, y, size)

histogramHSV_Filter(x, y, size)

motion_Filter()

ColorTrackRGB_Filter(color, tol)

ColorTrackHSV_Filter(color, tol)

SpotColorTrackRGB_Filter(x, y, size, tol)

SpotColorTrackHSV_Filter(x, y, size, tol)

Data Collecting Functions

Data Collecting Functions are used to get information back from your video stream. This information can be used in programming to allow your robot (or simulated robot) to act differently depending on what it "sees." 

getHistData()

getMaxBlob()

getMaxBlob_cx()

getMaxBlob_cy()

getMaxBlob_mass()

Examples

Apply Random Filter

The following example will continuously loop, changing the current filter every 50 iterations. The new filter will be randomly chosen from seven of the currently supported filters. Each time the filter is changed, the name of the new filter (as well as its parameters) will be displayed. 

  1 
  2 
  3 
  4 
  5 
  6 
  7 
  8 
  9 
 10 
 11 
 12 
 13 
 14 
 15 
 16 
 17 
 18 
 19 
 20 
 21 
 22 
 23 
 24 
 25 
 26 
 27 
 28 
 29 
 30 
 31 
 32 
 33 
 34 
 35 
 36 
 37 
 38 
 39 
 40 
 41 
 42 
 43 
 44 


from pyrobot.brain import Brain
from time import *
from random import *
from phissioncamera import *

class SimpleBrain(Brain):
    def setup(self):
        self.counter = 0            # initialize counter to 0
    def step(self):
        self.counter += 1           # increment counter
        if ((self.counter % 50) == 0):   # run randomFilter() every 50 iterations
            self.randomFilter()

    def randomFilter(self):
        x = randrange(0,7,1)        # get a random number
        if(x == 0):
            self.camera.canny_Filter()
            print "canny_Filter()"
        if(x == 1):
            self.camera.sobel_Filter()
            print "sobel_Filter()"
        if(x == 2):
            self.camera.gaussianBlur_Filter()
            print "gaussianBlur_Filter()"
        if(x == 3):
            self.camera.medianBlur_Filter()
            print "medianBlur_Filter()"
        if(x == 4):
            self.camera.motion_Filter()
            print "motion_Filter()"
        if(x == 5):
            self.camera.SpotColorTrackHSV_Filter(160,120,12, (12,60,120) )
            print "SpotColorTrackHSV_Filter(160,120,12, (12,60,120) )"
        if(x == 6):
            self.camera.SpotColorTrackRGB_Filter(160,120,12, (30,30,30)  )
            print "SpotColorTrackRGB_Filter(160,120,12, (30,30,30)  )"

def INIT(engine):
    camera = phissioncamera()     # create a new phission camera
    camera.setup(1,320,240)       # set up the phission camera -- change first param to 0 if not on robot
    brain = SimpleBrain('SimpleBrain', engine)  # create the brain
    brain.camera = camera         # set the brain's camera variable
    return brain
[PhissionRandomFilterProgram]Edit

Intelligent Motion Track

The following example will begin by finding the largest moving object in the video stream, and capturing the color of that object. The state will then be changed to ColorTrack, where the largest object of the captured color will be tracked. If at any point there is no object at or above a reasonable tolerance (1000 pixles) the state will return to MotionCatch and the process will be repeated. 

  1 
  2 
  3 
  4 
  5 
  6 
  7 
  8 
  9 
 10 
 11 
 12 
 13 
 14 
 15 
 16 
 17 
 18 
 19 
 20 
 21 
 22 
 23 
 24 
 25 
 26 
 27 
 28 
 29 
 30 
 31 
 32 
 33 
 34 
 35 
 36 
 37 
 38 
 39 
 40 
 41 
 42 
 43 
 44 
 45 
 46 
 47 
 48 
 49 
 50 
 51 
 52 
 53 


from pyrobot.brain.behaviors import *
from pyrobot.brain.behaviors.core import *
from time import *

from phissioncamera import *

camera      = phissioncamera()

class MotionCatch(State):
    def onActivate(self):
        print "Activating MotionCatch State"

        self.motion_thresh   = 1000
        camera.motion_Filter()
        sleep(1)

    def update(self):
        max_blob = camera.getMaxBlob()
        if(max_blob.mass > self.motion_thresh):
            camera.SpotColorTrackHSV_Filter(max_blob.cx, max_blob.cy, 15, (12, 60, 200) )
            self.goto('ColorTrack')


class ColorTrack(State):
    def onActivate(self):
        print "Activating ColorTrack State"
        sleep(1)
        self.track_thresh   = 1000

    def update(self):
        max_blob = camera.getMaxBlob()

        print "-------------------"
        print "mass =", max_blob.mass
        print "(x,y) = ", max_blob.cx, max_blob.cy

        if(max_blob.mass < self.track_thresh):
            self.goto('MotionCatch')

class state1(State):
    def update(self):
        self.goto('MotionCatch')

def INIT(engine):
    brain = FSMBrain(engine)

    brain.add(state1(1))
    brain.add(MotionCatch())
    brain.add(ColorTrack())
    brain.camera = camera
    camera.setup(1,320,240)
    return brain
[PhissionMotionTrackProgram]Edit

Fuzzy Camera Example

The following example combines the use of Fuzzy logic and the Phission system to track an object using a PanTiltZoom (PTZ) camera. It does this by taking a color histogram of the center of the initial image. This histogram will be used for the duration of the run. The largest object matching the given color (within the provided tolerance) will be tracked by the camera.

/!\ This example requires the use of a PTZ camera 

  1 
  2 
  3 
  4 
  5 
  6 
  7 
  8 
  9 
 10 
 11 
 12 
 13 
 14 
 15 
 16 
 17 
 18 
 19 
 20 
 21 
 22 
 23 
 24 
 25 
 26 
 27 
 28 
 29 
 30 
 31 
 32 
 33 
 34 
 35 
 36 
 37 
 38 
 39 


from pyrobot.brain.fuzzy import *
from pyrobot.brain.behaviors import *
from phissioncamera import *
from time import sleep

class BBB(BehaviorBasedBrain):
    def destroy(self):
        self.removeDevice("ptz0")

class Avoid (Behavior):
    """Avoid Class"""
    def update(self):
        if( camera.getMaxBlob_mass() > 200 ):
            self.IF(Fuzzy(0, 160 ) << camera.getMaxBlob_cx(), 'pan',  5.0, "pan left")
            self.IF(Fuzzy(0, 160 ) >> camera.getMaxBlob_cx(), 'pan', -5.0, "pan right")
            self.IF(Fuzzy(0, 120 ) << camera.getMaxBlob_cy(), 'tilt',-5.0, "tilt down")
            self.IF(Fuzzy(0, 120 ) >> camera.getMaxBlob_cy(), 'tilt', 5.0, "tilt up")

class state1 (State):
    """ sample state """
    def setup(self):
        print "about to train color"
        sleep(10)
        camera.SpotColorTrackHSV_Filter(80,60,8, (20, 80,110))
        print "color trained!"
        self.add(Avoid(1, {'pan': 1, 'tilt': 1}))

def INIT(engine):
    engine.robot.startDevice("ptz")
    ptz = engine.robot.ptz[0]
    brain = BBB({'pan' : ptz.panRel,
                 'tilt' : ptz.tiltRel,
                 'update' : engine.robot.update }, engine)
    brain.camera = phissioncamera()
    brain.camera.setup(0,160,120);
    brain.add(state1())
    brain.activate('state1')
    return brain
[PhissionFuzzyCameraProgram]Edit

Pyro Modules Table of Contents

Modules

  1. PyroModuleIntroduction

  2. PyroModuleObjectOverview

  3. PyroModulePythonIntro

  4. PyroModuleDirectControl

  5. PyroModuleSequencingControl

  6. PyroModuleBehaviorBasedControl

  7. PyroModuleReinforcementLearning

  8. PyroModuleNeuralNetworks

  9. PyroModuleEvolutionaryAlgorithms

  10. PyroModuleComputerVision

  11. PyroModuleMapping

  12. PyroModuleMultirobot

  13. FurtherReading

Additional Resources

  1. PyroIndex

  2. PyroAdvancedTopics

  3. PyroUserManual

  4. [WWW]Pyro Tutorial Movies

Reference: PyroSiteNotes

[PyroModulesContents]Edit