OPEN FINGER COUNTING USING SIMULINK

Before we focused on embedded system, we tried to simulate open finger counting on matlab-simulink to find out an algorithm.
We recorded a video which includes a hand in front of a black background and fingers are opening one by one on that video. We use this video as an input to the simulink blocks diagram that we built up to count open fingers. Firtsly we convert the RGB(red-green-blue) video into the gray scale using Color Space Conversion block. Autotreshold is convert the intensity image to binary image. We need to do that because to do other implementation we have to work on binary image, so we can find the hand (see figures’ part3). To eleminate the noise on binary image we use a median filter. The Median Filter block replaces the central value of an M-by-N neighborhood with its median value. If the neighborhood has a center element, the block places the median value there. The filtered binary image go into open, bottom-hat and close blocks. These blocks are used for morphological operations on binary or intensity image. The Opening block performs an erosion operation followed by a dilation operation using a predefined neighborhood or structuring element. We used the Bottom-hat block to perform bottom-hat filtering. Bottom-hat filtering is the equivalent of subtracting the input image from the result of performing a morphological closing operation on the input image. The Closing block performs a dilation operation followed by an erosion operation using a predefined neighborhood or structuring element(see figure1 for output of closing). After these morphological operations we again apply median filter to eleminate the morphological operations’ noise. The Label block labels the objects in a binary image, BW, where the background is represented by pixels equal to 0 (black) and objects are represented by pixels equal to 1 (white). At the Label port, the block outputs a label matrix that is the same size as the input matrix. At the Count port, the block outputs a scalar value that represents the number of labeled objects. If we do not use filtering again before the Label block, it will count the number of noises also as you will see in the figures below.

Figure 1: Labeled image without filtering
The following autotreshold block makes the labels visible(see figures’ part2). We use the Blob Analysis block to calculate statistics for labeled regions in a binary image. The Bbox output is 4-by-N matrix of bounding box coordinates, where N is the number of blobs. We use that coordinates on Draw Shape block. Draw Shape block draws rectangles in our application. It has two input which are Image and Pts. Image input gets the image that we draw rectangles on and Pts input gets the coordinates of the rectangles. The Insert Text block writes the number of finger that is being counted on the display(see figures’ part1).
RESULTS









SIMULINK SCHEMATIC

INTRODUCTION

We will use this blog to explain the development on our graduation project.

Our project is about image processing that based on detecting the number of open fingers of a hand. Image processing is a common topic, this concept is taking more attentions day by day. As the applications areas of image processing are getting wider, a lot of tools to make this process easier are being developed by companies. Matlab, Simulink and Open CV can be given as examples to the software that simplifies image processing. By these programs development of image processing projects on computer is very simple nowadays. But the necessity of this concept on independent systems like robots, exposed different developments on hardware systems. For this purpose big hardware companies started to spend a lot of money on research and development of different boards which have special attributes about image and video processing. But working with these systems are not as easy as using Matlab and Simulink for such kind of projects.

As this software is easy to use, we first tried to make our project work on the computer. We worked with Simulink and achieved to count the open fingers from a video that we recorded. In the next post we will try to explain the Simulink blocks that we use for this process. Also a few figures of the results of testing can be seen.

After that we will mention about the choice of the embedded system. We work on DaVinci that manufactured by Texas Instruments. Some specialties of the board can be read. After that we explained the steps that we followed to use that board, difficulties that we encountered, algorithms, written programs and test results.