Advanced Topic: Writing your own Vision Code

$/!\$ This is currently only be to used by UMASS Lowell, for testing. This page refers to the vision system prior to Pyro 2.2.0. For the newer version please see PyroModuleVisionSystem.

Setting up the System

The files in the /usr/local/pyrobot/camera/v4l directory should be copied to /usr/local/pyrobot/camera/v4l/shadow and /usr/local/pyrobot/camera/v4l/.backup. The files in shadow should be given 755 permission, with the root as the owner and the group. The files in .backup should be given 700 permissions, because they are only there incase the files in shadow some how get over written. The students will copy all the files from the shadow directory and place them anywhere within their directory structure. The files to be copied are: Makefile, grabImage.c, v4lcap.c and v4lcap.h and init.py. These files should be given 700 permissions so that there is no way of accidental copying or modifying of their code by someone else.

$/!\$ Currently,The Makefile must be editted as following:

should be changed to read

#include /usr/local/pyrobot/Makefile.cfg (provided /usr/local/pyro is the path to the pyro directory)

The /v4l directory will also need to be given 777 permissions so that the user can change the symlinks to point to their own code. However, a better idea would be to make a vision group and add any user to the group that will be making changes to the vision. Then make the /v4l directory have 775 permissions and change its group to vision via the 'chgrp vision' command.

Editting the files

You should copy the files in /usr/local/pyrobot/camera/v4l/shadow to a local directory for editing.

grabImage.c is where all the vision processing function definitions, such as maxBlob, mean_filter, etc... are held. The init.py file is where the function definitions for the pyro calls are held. If changes are made to grabImage.c, or v4lcap.* the Makefile must be run to generate the grabImage.so file. This is the compiled file that is read by python. If changes are made to init.py, no recompliation is needed, but pyro must be restarted in order for either change to take affect.

$/!\$ Technically, you only need to reload the robot with in pyro for the changes in init.py to take affect, however there seems to be a bug which causes pyro to crash if a robot is loaded when a robot is already loaded. Also, if User A starts up pyro, while User B already has their vision code loaded up, the links in /pyrobot/camera/v4l/ will be over written to point to User A's vision files, but User B is still reading from his/her own files. User A will not beable to connect to the robot is User B is running pyro anyway.

For speed reasons, the vision processing is done in C. Inorder for Python to read the C files, the C code uses various structs that can be seen by looking through the code already provided in grabImage.c, many Books have on Python have sections about Embedding C, or you can visit the Python documentation at http://www.python.org/doc/current/ext/ext.html.

Things to know

The framegrabber is constantly saving images from the camera in a shared memory buffer, which can be thought of a 3-Dimensional Array of width, height and depth. Depth is 3, because of the three color values, RGB. This buffer can be seen both by C and Python. The pointer to the beginning of this shared memory is saved in a global variable map,which is defined in v4lcap.h. Most commonly, when u are creating a function to parse the image, you will want to create a variable and set it equal to map.

   unsigned char *image
   image = map

To parse through the image, a nested for loop is the easiest way.

   for(h = 0;h<height;h++)
      for(w=0;w<width;w+=3)

or

   while(image < map*height*width*depth)
   {
      ... do stuff here
      image=image+3
   }

The +3 increments are used to jump from Blue value to Blue value throughout the buffer. If you wanted to get the color values, you can then do

   unsigned char blue,green,red;

   blue = *image
   green = *(image+1)
   red = *(image+2)

They are unsigned characters because a char is a byte, so when it it unsigned it allows for values from 0-255

$/!\$ Remember: The frame grabber saves the values into the buffer in BGR format, not RGB

After your function is written, for reasons to do with python, you must add the function to dictionary at the end of the grabImage.c file.(the format is easy to follow, and does not need explanation)

Running Pyro

To run pyro using your newly created and compiled code, at the command line simply type:

    pyro -m (path)

ie: pyro -m /usr/local/username/vision/  
    pyro -m /home/username/visiondir/
    pyro -m ~username/myVision

This will then tell pyro to load up your vision files instead of the default ones. It does this by creating a symlink to your grabImage.so and init.py from /usr/local/pyrobot/camera/v4l. Therefore, the filenames must cannot be changed from grabImage.so and init.py.

$/!\$ Known bug: If a brain is loaded that makes use of the V4LCamera service, and that brain file is opened to be editted through pyro, The editor window must be closed before pyro is exited or else the camera device will not be released properly, and the camera device will stay busy until the robot is rebooted

Pyro Modules Table of Contents

Pyro - Back to Pyro main page
Beyond Legos - NSF grant that pays for Pyro

Modules

Additional Resources

Reference: PyroSiteNotes

[PyroModulesContents]