How to write code for entropy between two vectors?

Hi everyone.

I have two numeric vectors X and Y with one column and 500 rows and I want to get entropy between two vectors. Also I want to get entropy X. How to do that in matlab?Please help me. Thanks alot.

Answer 1

[Abhaya]

Hi Nazila,

To calculate the joint entropy between vectors ‘x’ and ‘y’, you can refer to the steps given below:

• Compute the joint probability matrix ‘p’ for vector ‘x’ and vector ‘y’. If you don't have the probability matrix, You can use the MATLAB ‘histcounts2’ function, to generate the probability matrix.

[p, ~, ~] = histcounts2(X, Y, 'Normalization', 'probability'); 

• Flatten the probability matrix.

p= p(:); 

• Remove the zero probabilities.

p(p==0)=[]; 

• Calculate the Entropy Using Shannon's Entropy Formula.

H = -sum(p .* log2(p)); 

To calculate the entropy of variable ‘x’ you can follow the approach described above.

For more information on entropy calculation, please follow the MATLAB community discussions.

• https://www.mathworks.com/matlabcentral/answers/436929-how-to-find-entropy-of-x
• https://www.mathworks.com/matlabcentral/answers/42142-entropy-of-a-signal

For more information on MATLAB ‘histcounts2’ function, please refer to the MATLAB documentation linked below.

https://www.mathworks.com/help/matlab/ref/histcounts2.html

Hope this helps.