Hand segmentation from an RGB and depth image obtained from kinect device

Hi all. I am trying to use the dataset from this paper and segment hand from the image. I attach a sample RGB and depth image. I have written this code.

    close all; 
depth_map=load('3.txt');
imagesc(depth_map);
image=imread('4.jpg');
figur;imshow(image);
foo=depth_map;
min_v=min(foo(foo>0));
threshold=min_v+100;%threshold - varying this is not helping much but ofcourse will deteriorate 
foo(foo>threshold)=0;
BW=imbinarize(foo);
figure; imshow(BW);%shown in result3.jpg file.

The result is shown in the result3,jpg file which is attached. As you can see even elbow comes. How do I segment just the hand? I am relatively new to RGB and depth images and kinect. Any help is greatly appreciated.

[Selva Karna]

you can calculate boundary from BW image , after you can apply this boundary point to Original Image, then u can get hand

[Image Analyst]

I think the cartoon captures my thoughts:

So, what is your step two? How are you determining the boundary between the forearm and the hand?

[kowshik Thopalli]

The person will always wear a black belt at the end of his hand i.e at the wrist. The paper says it uses ransac to identify the belt and thus separate it from the rest of the image. Figure 3 in the paper very well illustrates this. Here is the link to the paper again - https://www.microsoft.com/en-us/research/publication/robust-part-based-hand-gesture-recognition-using-kinect-sensor-2/

[kowshik Thopalli]

Btw it is a beautiful cartoon. I apologise for being vague.

[Shyam verma]

Hey! can you give the dataset as the link that you mentioned above for dataset is not working/showing error.

with regards

[e-mail]

Answer 1

[Image Analyst]

If the person always has a black wrist band, then you should have 2 blobs, not 1, and then you can use bwareafilt() to extract the largest one, which will be the forearm, and then subtract that from the original mask to get only the hand.

forearmMask = bwareafilt(binaryImage, 1);
handMask = binaryImage & ~forearmMask;

[kowshik Thopalli]

Two issues. One I am not getting two blobs by using the code that I am doing. And it is very heuristic to select the threshold. Here in this example I have 100. But I have 1000 images and this number keeps varying.how to over come these? I have a feeling that what I am doing is not correct. Can you please show me the right path.