Gaze Interest Heat Map

Hello! I am attempting to take x and y coordinates from an eye tracking study and translate them into "attention heat maps." My desired heat map should look similar to:

http://fruition.net/denver-website-design-firm/website-design/heat-maps-website-design-conversion-analysis/

or:

http://www.jasonbabcock.com/eyetracking.html

I know the location of each gaze point, the frame rate, the time that the gaze point is created, however I cannot figure out how to turn this information into a usable format. This page is close to what I need:

http://www.mathworks.com/matlabcentral/answers/1729-adding-z-values-for-surf-contour-plots

except that I need to amend the code so that overlapping z values are added, instead of blended together. I.E. if z1=10 and z2=15, the resulting z value will be 25. Any help that can be given would be greatly appreciated.

Answer 1

[Jarrod Rivituso]

It sounds like you are looking for a 2-dimensional histogram. There is none in default MATLAB, as far as I know. However, several users on the file exchange have made submissions to this point:

http://www.mathworks.com/matlabcentral/fileexchange/1487-2d-histogram-matrix

http://www.mathworks.com/matlabcentral/fileexchange/8422-2-dimensional-histogram

I believe the main idea for each is that you

1. define x,y vectors that represent the grid of bins for the histogram
2. then you count each entry of your data adding values to the bins,
3. and finally you use pcolor or imagesc or something like that to make the plot.

[Walter Roberson]

hist3() and rotate to view from above?

[Chris Berry]

Thank you Jarrod for your quick response. I apologize, I think my earlier post was a bit confusing. What I am attempting to do is overlay gaussian curves at specific X and Y coordinates, and combine all Z values to create a "Heat Map."

[Jarrod Rivituso]

So are you trying to fit a 2-D Gaussian distribution, and then specifically plot that distribution?

The histogram approach could end up looking like a Gaussian distribution, such as this:

x = randn(1000,1) + 7;

y = randn(1000,1) + 10;

[n,c] = hist3([x y],[50,50]);

imagesc(c{1},c{2},n)

Or you could replace imagesc with surf and then rotate

surf(c{1},c{2},n,'linestyle','none')

Depending on the size of the bins you choose, the data will look finer or more coarse. In the example, I've chose a 50-by-50 grid of bins

If you are trying to truly fit a Gaussian curve to the data, then I think you would need to first determine the necessary coefficients of the distribution and then plot the fitted equation, probably using something like meshgrid as shown in Matt Tearle's post (which you referenced)

Answer 2

[Image Analyst]

I think the accepted answer is way off base. I think that the way to go is to use scatteredInterpolant function of MATLAB, or krigging.

[S]

Could you please elaborate a bit on this? Taking into account that we have as input an array of coordinates and a timestamp for each of these, how could we use scatteredInterpolant since for that we need to define an interpolant function?

[Image Analyst]

See attached scatteredInterpolant demo.

[dim-ask]

I agree that the accepted answer is off-base, just to mention that what you propose is a good solution for a heatmap when you have a "value" for each point and the density of the points is not relevant, for example when you are looking at pupil size for looking at different parts of the screen, or temperatures in different regions.

For a fixation heatmap, where the density of the points usually matters (in contrast to, say, how many temperature stations each place has), this will not produce the intended result because it will not give a "hotter" colour for regions with more points (ie more fixations) vs less. Imo just "painting" the points in a blank image and passing them through a gaussian filter after gives the best and simplest result. A common alternative is using sth like ksdensity but that would just be a more computationally heavy way to basically do the same thing.

Answer 3

[Deepa AS]

How to generate the heat map with the help of eye gaze data that consists of x and y co ordinates ?

Answer 4

[dim-ask]

There is a relatively simple solution that imo addresses best what OP was asking, and hopefully it can help other people arriving here. Eg

backimag = uint8(zeros(600,600,3)); % backimag is the background image we have; let's put black
X = randi(100, [100 1]) + 200;      % some random x-coordinates
Y = randi(100, [100 1]) + 100;      % some random y-coordinates
W = ones(size(X));                  % we could have weights on each gaze (eg to normalise for different number of trials per subject)
heatmap = mat2cell(hot,256,[1 1 1]);% we choose a heatmap colouring, eg "hot", and convert it to "cell"
sigma  = 16;                        % the variance parameter for the gaussian kernel

[dimY, dimX, ~] = size(backimag);

% Create "mask"
mask = zeros(dimY, dimX);
for k = 1:size(X,1)
    mask(Y(k), X(k)) = mask(Y(k), X(k)) + W(k);
end

% Filter using a gaussian kernel
mask = imgaussfilt(mask, sigma);

% Normalise total mass of heatmap
% Here we equal the total mass with the one of one rectangle out of a 8x8 grid
% Decrease (increase) the constant to make the heatmap hotter (colder)
normConstant = 8;
normMask = dimX * dimY / normConstant^2 / sum(mask , [1 2]);
mask = mask * normMask;

% Colour the background image with the heatmap
newImage = backimag;
for rgbInd = 1:3
    thisHeat = heatmap{rgbInd}( floor(mask*255) + 1 );
    newImage(:,:,rgbInd) = (newImage(:,:,rgbInd) + uint8(thisHeat*255));
end
figure; imshow(newImage)

So, the idea is that we have a background image as an array Y x X x 3. We then create a "mask" with the gaze data ie a Y x X array where we add 1 at the positions that our gaze data gives, and we apply a gaussian filter on it (eg imgaussfilt from image processing toolbox). We then overlay this to the background, for each rgb-coordinate separately.

I used to use ksdensity to do basically the same estimation but it was much slower, while this one seems quite optimised, plus it avoids time-consuming plotting functions as it only processses arrays. The way the images are overlayed is a bit naive but works great with darker backgrounds (usually I grayscale them and make them darker before, so it looks more clear). One can do it in other ways.

Here I implemented it in a function:

https://raw.githubusercontent.com/earlychildcog/eyeVis/main/src/plotHeatmap.m

and the github repository includes scripts to create heatmap videos:

https://github.com/earlychildcog/eyeVis

[Filip Mihai Toma]

Hello,

Can you please elaborate on:

heatmap = mat2cell(hot,256,[1 1 1]); ?

From what I knew, mat2cell had a different funcitonality and "hot" is not one if its arguments

Also, regarding the for loop where you calculate mask values, in this current code example, I only obtain zero.

My question is:

how do you weight for duration fixation for each X-Y coordinate of the fixation.

For example

Position Duration fixation Position X Position Y

1 200 ms 500 700

2 250 ms 400 300

3 580 ms 800 900

So in the above 3 rows, I would expect the heatmap to be "the reddest" around the third point which is located at coordiantes 800-900 (X-Y axes) because a participant looked at that point for 580ms, as compared to the other two positions, where they only fixated for 200 and 250 ms, respectively.

Thanks

[Walter Roberson]

@Filip Mihai Toma

hot in this context is not a named option. Instead, hot is a function that is being called with no parameters.

There are a number of built-in colormaps, such as hot and prisim and hsv and copper . Those colormaps can be invoked with a parameter that is the number of entries on the colormap that are requested. There is code in the functions to interpolate the underlying colormaps into the requested number of values. In the case that the functions are invoked with no parameter, they look to see if there is an existing figure and if so then create a colormap the same size as the colormap for the existing figure. In the case where there is no existing figure, they create a colormap the same size as the default colormap.

The code could have been written

colormap_to_use = hot();
heatmap = mat2cell(colormap_to_use, 256, [1 1 1]);

[Walter Roberson]

how do you weight for duration fixation for each X-Y coordinate of the fixation.

Are you asking how to make a colormap with different sizes for the different colors? For example a colormap that that had the first 1/3 be one color, the 1/2 after that be a second color, and the remaining 1/6 be a third color?

If so then I recommend that you look at https://www.mathworks.com/matlabcentral/fileexchange/69470-custom-colormap which allows you to give a list of positions and associated colors and the code will build the internal details of the colormap for you.

[andrey oppenheimer]

we are using Tobii pro spectrum.

we would like to integrate this heat map to my eye tracking code but also add diameter from the data.

and to change the tracker size on the video with the diameter.

['dat']['data']['gaze']['left']['pupil']['diameter']

[dim-ask]

@Filip Mihai Toma

> how do you weight for duration fixation for each X-Y coordinate of the fixation.

In line 4, I would change the default weight array W = ones(size(X)); to W = array_of_fixation_periods/max(array_of_fixation_periods) so in your case W = [200 250 580]/580 .

[dim-ask]

@andrey oppenheimer If I understand well, you want to show a video for each subject with a "marker" that changes size related to the pupil size rather than a heatmap? Here, this is a density plot essentially, so it is completely different.

If you just want to have a marker whose coordinates track the gaze and the size the pupil diameter, I would just use something like plot(y_coord, x_coord,'Marker','.','MarkerSize',10*pupil_size) or sth similar, overlayed over a background image, though ime changing the size of the marker is not really so salient, and it would go better with changing the colour.

If you wanted a heatmap that shows the average size of the pupil size when gazing at parts of the screen instead of the density of fixations, you can try the scatteredInterpolant function as in @Image Analyst's comment above.

[andrey oppenheimer]

thanks ill try it