Finding Optimal Number Of Clusters for Kmeans

I want to find the number of clusters for my data for which the correlation is above .9. I know you can use a sum of squared error (SSE) scree plot but I am not sure how you create one in Matlab. Also, are there any other methods?

Answer 1

[Taro Ichimura]

Hello,

you have 2 way to do this in MatLab, use the evalclusters() and silhouette() to find an optimal k, you can also use the elbow method (i think you can find code in matlab community) check matlab documentation for examples, and below

% example
load fisheriris
clust = zeros(size(meas,1),6);
for i=1:6
clust(:,i) = kmeans(meas,i,'emptyaction','singleton',...
        'replicate',5);
end

va = evalclusters(meas,clust,'CalinskiHarabasz')

[KAE]

https://www.mathworks.com/matlabcentral/fileexchange/65823-kmeans-opt--optimal-k-means-with-elbow-method?focused=8772533&tab=function

Answer 2

[Pamudu Ranasinghe]

Refer "evalclusters" function

https://in.mathworks.com/help/stats/evalclusters.html#d123e303261

eva = evalclusters(X,'kmeans','CalinskiHarabasz','KList',1:6);
Optimal_K = eva.OptimalK;

[Walter Roberson]

And see https://www.mathworks.com/matlabcentral/answers/52322-how-to-determine-number-of-clusters-automatically-for-each-image-to-be-used-in-k-means-algorithm#comment_2222525 for why evalclusters is mostly arbitrary with not so much real use.

Real mathematics says that every unique point should be its own cluster.