Donnerstag, 7. Juni 2012

3nd OpenCV PSMove Example (Region Of Interest)

This is the third blogpost about the implementation of a tracker for the colored sphere of the PSMove controller. This time it is about the optimizing the trackers computation time, and some other stuff.

1. Increasing calculation speed

Thanks to Budaházi Viktor from [moveframework] i learned [mailinglist], that 70fps is probably not fast enough, as feature applications using the psmoveapi may already cause a heavy load to the system.

Therefore i introduced a technique called ROI (region of interest) [opencv roi example] in order to reduce calcualtion time. The main idea is, that instead of evaluating the whole picture, just evaluating a region of the picture in which it is very likely to find what we are searching for. This can speed up calculations tremendiously, however the framerate is not constant anymore, as one is downsizeing or extending the ROI during runtime.

So here is what i did: I defined in the aplication to have a arbitrary number of levels of ROIs which will reduce their size by 40% on each level downwards. In the demo i choose to have 5 leves of ROI, so that:
Level 1.                     640x480px (full camera image)
Level 2. 60% of (1) --> 383x383px
Level 3. 60% of (2) --> 229x229px
Level 4. 60% of (3) --> 137x13  px
Level 5. 60% of (4) --> 82x82    px

In the main loop of the tracker i calculated the bounding-box of the sphere found in the current image. Then for the next iteration of the loop, the ROI level is set to one that can hold that bounding box (multiplied by 2). If the sphere was not found at all, i'll go upwards in the hierarchy of ROI levels until the sphere is found again. The center of ROI is always set to the last location where the sphere was found. Future implementations my use movement prediction and additional sensor-data to put the center more into the direction the user is likely to move the controller. This would reduce switchin between the ROI leves for fast movements.

In the top of the video you may now see the framerate, expected sphere color, average luminace, camera exposure and the ROI size. The white square in the image denotes size and location of the current ROI. Note how the framerate increases when i go further away from the camera. With the help of ROI i now get framerates up to 1500fps :P.

2. Handling distortions from (flourescent) light sources

I also figured out, that having fluorescent lightsource in a room may cause the calibration to fail and highly influence the accuracy of the tracking. The lamps cause due to there operational mode to have, just like the camera, something similar to a refresh rate. As the camera and the lightsources are not synchronized and don't have the same refresh rate, the video feed seems to flicker, that means there are travelling darker/brigher horizontal/diagonal regions within the camera image. [light flickering]

In one of the previous posts, i explained, that i perform the color calibration with the help of a sequence of difference images. The light flickering causes small but recognizeable differences in these difference images wich are then in turn regarded as the sphere beeing lit/unlit.

Increasing the number of image pairs taken by one already decreased false-detections reasonably. However it is still not enough as there still remain false-detections in all image pairs, and further increasing the number of pairs taken may neither be bearable by the user nor is it clear if it would be beneficial.

I learnd from a colleague about morpholgical operations like [dilation] and [erosion] which were quit helpful to cleanse the image from smaller false-detections.

1) orignial image
2) difference image
3) thresholded image
4) eroded/dilated image

Notice how in the lower left corner the lamp causes a greater white area in the thresholded image (3) and how it is removed by a subsequent erosion and dilation in image (4).

3. Choosing the right camera exposure time

Depending on the current average luminance in the camera image, the camera exposure is choosen appropriately. I found out, that a exposure value smaller than 0x10 causes colors to be very grey-ish and going higher than 0x40 increases motion unsharpness (fast moving controller) and the sphere looks very white-ish due to the long exposure.

In the very beginning of the calibration i therefore starte with exposure 0x10 and go step by step up to exposure 0x40 until i get a average luminace of 25.

The average luminance is defined in my case as:

IplImage* cameraImage;
CvScalar avgColor = cvAvgS(cameraImage,0x0);
float averageLuminance = (avgColor[0] + avgColor[1] + avgColor[2]) / 3;

If the resulting average luminance is about 0x20, i reduce the spheres brightness to 70% and if it is above 0x30 i decrease it to 50%. This assures, that the shperes color does not look too white-ish for longer exposure times.

Well thats it for this time. ...

system requirements:
Eingestellt von um

Keine Kommentare:

Kommentar veröffentlichen

Abonnieren Kommentare zum Post (Atom)