Adventures of NeeC++

Weka is a powerful data mining tool that develops predictive modeling and data analysis based on information it receives. A more accurate way to predict if a ball is within an image view or not is to perform machine learning. I used Weka  to produce a decision tree based upon data that will be collected. I took multiple pictures with the Nao’s camera, some  with the orange ball in view, and others without. By using these images, it is possible to determine attributes that are similar in images with the ball in the frame. The problem is, Weka does not accept image files, so I had to transform these images into a data file somehow. Since images are just made up of hundreds of pixels with certain RGB values, I created a  custom program that displays all of the RGB values of a given image. (This research spawned an image to pixel value convertor code that will be given to the OpenCV community) Once all of the RGB values of an image was produced, they were than set as data inside a Weka file (Figure 3.1), with the RGB values each being a separate attribute. Another attribute was added, called the ballFound method. This was just a Boolean method that outputted either a YES, or NO. After each pixel data line, the ballFound attribute would determine if the those RGB values produced an orange color. If the values returned an orange color, the ballFound method would be set to YES, if not it would return NO. So if an image file returned a certain number of YES’s, this characteristic would determine if an orange ball is within a given image frame or not.

Figure 3.1 Image converted to a data file for Weka

Ultimately the goal is for Weka to produce the most accurate and efficient decision tree  based on the ballFound method. The more data that Weka evaluates, the predictive modeling it develops will be more precise. Therefore 30 images were taken, with 20 of them having the ball in view and 10 without. These pictures were then loaded into the OpenCV program that was custom created to convert images to RGB pixel values. Next each image had it’s own separate data file. Once done these data files were given to Weka for analysis. The decision tree the Weka produces is shown in figure 3.2.

Figure 3.2 Weka produced decision tree

In order to utilize this decision tree within the Nao’s vision code, this tree must be converted into multiple conditional statements in  C++. Here is the code snippet that was added:

if (blue <= 95 && red > 84)

orangePixelCount++;

This fundamental is the decision tree. If the blue value has a pixel value of 59 or less and if the red value has a pixel value greater than 44, than an orange pixel value is found. In the vision code, if there is more than 30 found orange pixel values, than a ball is determined to be inside the image frame. Now the question is, does this decision tree improves the object recognition accuracy. This was tested by comparing the two different methods. One using the Weka decision tree code and the other code without. In the vision codes, there were drawing methods implemented in order to visually see how many orange pixel values were found in an image and where. The method using the Weka decision tree has blue dots drawn and the program without the decision tree has red dots drawn over the orange pixels found (Figure 3.3)

Figure 3.3 Comparison of Code with decision tree and without

Clearly the vision code using the decision tree produced by Weka (on the right) identifies more of the orange pixels on the ball. The precision on both codes are excellent however the recall on the code using the decision tree is significantly greater. Thusly Weka helps tremendously in color detection. A more in depth graph compares the accuracy and pixel values found. (Figure 3.4)

Figure 3.4 Weka Comparison Graph

Not only can Weka be utilized for robotic vision, it can also be used to figure out the best or most efficient tasks the Nao should perform. Decision trees are highly useful when paired with Artificial Intelligence.

After almost 2 weeks, Apple finally approved my update that fixes backwards compatibility issues. So people with software 2.2.1 and up can download and use iDex.

Please let me know if you are still having problems downloading iDex. Again make sure your device has at least the 2.2.1 software.

Now that I have calibration up and running in Xcode, I now want to merge my existing vision code with my calibration code. For now I simply changed the input values from my vision code so that it reads the image from the undistorted view instead of a raw feed.

I’ll describe some background information of my existing vision code.

A couple weeks ago, I made program that was able to successfully perform image segmentation on a live video feed. Image segmentation is important with robotic vision because it allows for easier identification of color-coded objects. Here is a screenshot of what my program did:

The left window shows the raw image and the right image shows the transformed segmented image. What image segmentation does is sort of “blob” similar pixel values into one. Lighting conditions of a room greatly affects the value of a pixel. Sometimes the same orange ball from the image above can have lower pixel values or higher ones depending on how much light is shining on it. By blobbing similar pixels together, all of the unnecessary color detail is removed and colors are easily identifiable. You could see how this can be extremely helpful in robotics, it allows for them to identify an object faster and more efficiently.

I also implemented a color detection program within my segmentation code as well. The color detection code was produced by my robotics Professor, Dr. Andrew Williams (http://www.andrewbwilliams.com/Home.html), my research partner, Jazmine Miller, and I. To sum it up, our code searches for certain RGB values, in this case an orange color. Right now when I put the ball in front of my camera while the program is running, our code simply prints out “Ball Found” and the location of the ball within the frame. Eventually we will build upon this program and produce an actual ball tracking function

So I put all of these programs together and I now have a color detecting program that gets it’s image feed from an Undistorted and Segmented Image View. Here is a short video of my final project at work (make sure you notice the output window. It displays if ball is found in the current frame):

So in the Calibration code, we returned some values from the cvCalibrateCamera() function. The question is, what are these values and what purpose do they serve?

The results that the function cvCalibrateCamera2() produces are the camera’s intrinsics matrix, the distortion coefficients, the rotation vectors, and the translation vectors. The intrinsic parameters of the camera are the intrinsics matrix and the distortion matrix. In the program both of these files are stored in .xml files by using the cvSave() function. The rotation vectors and the translation vectors are extrinsic measurements which show where our object being used, in this case a chessboard, is located. Here is a screenshot of our saved .xml files:

Under the data brackets of each .xml files is our data coefficients found from our cvCalibrationCamera2() function. This data is important for use in future vision code.

Now onto explaining the distortion.xml file and the intrinsics.xml file.

• The distortion.xml file stores the distortion coefficients. These values help correct distortion and create an undistorted view.
• The intrinsics.xml file stores the parametes that define the camera being calibrated into a single 3×3 matrix. This makes it possible to transform 3D coordinates to the images 2D coordinates.

So whats next? Using this information to successfully track objects and possibly have some localization working as well