How can I train ANFIS using Particle Swarm Optimization algorithm ?

27 Sep 2023
3 Answers
Answer Accepted
Updated 14 Oct 2023
23 Views (30 days)

0 votes

I have my ANFIS developed from a dataset, I want to optimize it using PSO algorithm, please I need someone that will help me through. Please I need guidance, codes or proposed tutorials that I will easily learn how to optimize the ANFIS with PSO.

2 Comments
Show None Hide None

Sam Chak
Sam Chak on 27 Sep 2023
Hi @Ahmad
If I remember correctly, the Adaptive Network component of ANFIS has the capability to minimize the prescribed error measure using the hybrid learning method: Gradient Descent + Least Square Estimation. What role does the PSO play when you mention 'optimizing the ANFIS with PSO'? Are you suggesting replacing the optimizing 'AN' part with 'PSO', effectively creating 'PSOFIS'?
Ahmad
Ahmad on 27 Sep 2023
Thanks for your response, after training the dataset with ANFIS, to achieve a lesser error measure in this case RMSE, thats why am using PSO. I have already trained my dataset using ANFIS, having an RMSE of 0.013, so I want to achieve a lower value of RMSE using PSO. But may be I misunderstood the issue, because am new in this field. I will be happy if u can guide me more on it. Thank you.

Sign in to comment.

Sign in to answer this question.

 Accepted Answer

Sam Chak
Sam Chak on 11 Oct 2023
Ran in:

3 votes

Ran in:
Hi @Ahmad
The output (target) of the patient data constitutes a Boolean function, yielding values from a two-element set {0, 1}. It generates discrete outputs contingent on various input value combinations. Consequently, without postprocessing, I have reservations about ANFIS's ability to provide predictions based on input data, as fuzzy systems yield intermediate truth-values between true (1) and false (0). One viable approach is to round the predicted floating-point output values of the FIS.
For your information, I refrained from splitting the observational data into training and cross-validation sets. This decision was influenced by the belief that a continuous function might not aptly capture the Boolean function, unless one considers the Heaviside step function. Consequently, there exists a certain percentage of predicted targets that significantly deviate from the observational targets, which I have denoted as "outliers" (not to be confused with the statistical > 3σ outliers). One plausible explanation for these outliers may be attributed to unobserved parameters, such as Age, Sex, Family history, Smoking, Obesity, Sedentary lifestyle, and Stress.
And nope, the Particle Swarm Optimization (PSO) algorithm won't help for the same reasons explained above.
% Observational Data
data = readtable('hearts.csv');
X = table2array(data(:, 1:6)); % 6 Inputs of patient data
Y = table2array(data(:, 7)); % 1 Output (target)
data_Train = [X Y];
% Setting up the initial FIS using Subtractive Clustering method
genOpt = genfisOptions('SubtractiveClustering');
inFIS = genfis(X, Y, genOpt);
anOpt = anfisOptions('InitialFIS', inFIS, 'EpochNumber', 500);
% Training data with ANFIS
outFIS = anfis(data_Train, anOpt)
ANFIS info: Number of nodes: 107 Number of linear parameters: 49 Number of nonlinear parameters: 84 Total number of parameters: 133 Number of training data pairs: 303 Number of checking data pairs: 0 Number of fuzzy rules: 7 Start training ANFIS ... 1 0.320519 2 0.319414 3 0.318188 4 0.317348 Step size increases to 0.011000 after epoch 5. 5 0.316539 6 0.315504 7 0.314095 8 0.312564 Step size increases to 0.012100 after epoch 9. 9 0.311205 10 0.310197 11 0.309449 12 0.308929 Step size increases to 0.013310 after epoch 13. 13 0.308523 14 0.308169 15 0.30781 16 0.307463 Step size increases to 0.014641 after epoch 17. 17 0.307117 18 0.306767 19 0.306374 20 0.305968 Step size increases to 0.016105 after epoch 21. 21 0.30555 22 0.305126 23 0.30466 24 0.304207 Step size increases to 0.017716 after epoch 25. 25 0.303778 26 0.303382 27 0.302983 28 0.302613 Step size increases to 0.019487 after epoch 29. 29 0.302262 30 0.301924 31 0.301564 32 0.301215 Step size increases to 0.021436 after epoch 33. 33 0.300877 34 0.300549 35 0.300198 36 0.299856 Step size increases to 0.023579 after epoch 37. 37 0.299521 38 0.29919 39 0.298826 40 0.298458 Step size increases to 0.025937 after epoch 41. 41 0.298084 42 0.29771 43 0.297305 44 0.296906 Step size increases to 0.028531 after epoch 45. 45 0.296507 46 0.296099 47 0.29564 48 0.295206 Step size increases to 0.031384 after epoch 49. 49 0.294876 50 0.294629 51 0.294434 52 0.29416 Step size increases to 0.034523 after epoch 53. 53 0.29399 54 0.293718 55 0.293624 56 0.293304 Step size increases to 0.037975 after epoch 57. 57 0.293205 58 0.292934 59 0.292971 60 0.292658 61 0.292707 Step size decreases to 0.034177 after epoch 62. 62 0.292445 63 0.29249 64 0.292159 65 0.292257 Step size decreases to 0.030760 after epoch 66. 66 0.292002 67 0.292062 68 0.291772 69 0.291858 Step size decreases to 0.027684 after epoch 70. 70 0.291636 71 0.29169 72 0.291438 73 0.291511 Step size decreases to 0.024915 after epoch 74. 74 0.291316 75 0.291362 76 0.291142 77 0.2912 Step size decreases to 0.022424 after epoch 78. 78 0.29103 79 0.291065 80 0.290875 81 0.290915 Step size decreases to 0.020181 after epoch 82. 82 0.290768 83 0.29079 84 0.290626 85 0.290648 Step size decreases to 0.018163 after epoch 86. 86 0.290523 87 0.290529 88 0.290391 89 0.290394 Step size decreases to 0.016347 after epoch 90. 90 0.290289 91 0.290279 92 0.290162 Step size increases to 0.017982 after epoch 93. 93 0.29015 94 0.29006 95 0.290061 96 0.289957 97 0.289947 98 0.289853 Step size increases to 0.019780 after epoch 99. 99 0.289837 100 0.289748 101 0.289762 102 0.289645 103 0.289651 Step size decreases to 0.017802 after epoch 104. 104 0.289543 105 0.289545 106 0.289409 107 0.289423 Step size decreases to 0.016022 after epoch 108. 108 0.289315 109 0.289317 110 0.289196 111 0.2892 Step size decreases to 0.014420 after epoch 112. 112 0.289109 113 0.289102 114 0.289 Step size increases to 0.015861 after epoch 115. 115 0.288994 116 0.288918 117 0.288926 118 0.288838 119 0.288835 120 0.288757 Step size increases to 0.017448 after epoch 121. 121 0.28875 122 0.288677 123 0.288699 124 0.2886 125 0.288614 Step size decreases to 0.015703 after epoch 126. 126 0.288523 127 0.288531 128 0.288411 129 0.288434 Step size decreases to 0.014133 after epoch 130. 130 0.288337 131 0.288346 132 0.288235 133 0.288246 Step size decreases to 0.012719 after epoch 134. 134 0.288162 135 0.28816 136 0.288065 Step size increases to 0.013991 after epoch 137. 137 0.288062 138 0.287991 139 0.288 140 0.287917 141 0.287913 142 0.287842 Step size increases to 0.015390 after epoch 143. 143 0.28783 144 0.287764 145 0.287781 146 0.287689 147 0.287695 Step size decreases to 0.013851 after epoch 148. 148 0.287612 149 0.287612 150 0.287499 151 0.287515 152 0.287426 153 0.287431 Step size decreases to 0.012466 after epoch 154. 154 0.287355 155 0.287353 156 0.287255 157 0.287267 158 0.287192 159 0.287194 Step size decreases to 0.011220 after epoch 160. 160 0.287131 161 0.287127 162 0.287044 163 0.287053 164 0.28699 165 0.28699 166 0.286937 Step size increases to 0.012342 after epoch 167. 167 0.286931 168 0.286885 169 0.286906 170 0.286836 171 0.286849 Step size decreases to 0.011107 after epoch 172. 172 0.286787 173 0.286794 174 0.286704 175 0.28673 Step size decreases to 0.009997 after epoch 176. 176 0.286658 177 0.286672 178 0.286584 179 0.286605 Step size decreases to 0.008997 after epoch 180. 180 0.286538 181 0.286546 182 0.28647 183 0.286479 Step size decreases to 0.008097 after epoch 184. 184 0.286424 185 0.286422 186 0.286359 Step size increases to 0.008907 after epoch 187. 187 0.286358 188 0.286314 189 0.286321 190 0.286268 191 0.286265 192 0.286222 Step size increases to 0.009798 after epoch 193. 193 0.286212 194 0.286176 195 0.286187 196 0.286131 197 0.286134 Step size decreases to 0.008818 after epoch 198. 198 0.286086 199 0.286083 200 0.286013 201 0.286024 202 0.285969 203 0.28597 Step size decreases to 0.007936 after epoch 204. 204 0.285926 205 0.28592 206 0.285858 207 0.285862 208 0.285816 209 0.28581 210 0.285773 Step size increases to 0.008730 after epoch 211. 211 0.285761 212 0.285731 213 0.285738 214 0.285689 215 0.285689 216 0.285648 Step size increases to 0.009603 after epoch 217. 217 0.285641 218 0.285606 219 0.285627 220 0.285566 221 0.285578 Step size decreases to 0.008642 after epoch 222. 222 0.285525 223 0.285531 224 0.285451 225 0.285475 Step size decreases to 0.007778 after epoch 226. 226 0.285412 227 0.285424 228 0.285346 229 0.285365 Step size decreases to 0.007000 after epoch 230. 230 0.285307 231 0.285313 232 0.285246 233 0.285254 Step size decreases to 0.006300 after epoch 234. 234 0.285208 235 0.285203 236 0.28515 Step size increases to 0.006930 after epoch 237. 237 0.285146 238 0.285111 239 0.285114 240 0.285073 241 0.285065 242 0.285033 Step size increases to 0.007623 after epoch 243. 243 0.285018 244 0.284993 245 0.284995 246 0.284953 247 0.284947 248 0.284914 Step size increases to 0.008386 after epoch 249. 249 0.284901 250 0.284874 251 0.284885 252 0.284835 253 0.284837 Step size decreases to 0.007547 after epoch 254. 254 0.284796 255 0.284791 256 0.284725 257 0.284738 258 0.284687 259 0.284688 Step size decreases to 0.006792 after epoch 260. 260 0.284649 261 0.284642 262 0.284585 263 0.284589 264 0.284548 265 0.284541 266 0.28451 Step size increases to 0.007472 after epoch 267. 267 0.284495 268 0.284472 269 0.284477 270 0.284436 271 0.284431 272 0.284399 Step size increases to 0.008219 after epoch 273. 273 0.284388 274 0.284363 275 0.284378 276 0.284328 277 0.284334 Step size decreases to 0.007397 after epoch 278. 278 0.284293 279 0.284292 280 0.284224 281 0.284243 282 0.284191 283 0.284199 Step size decreases to 0.006657 after epoch 284. 284 0.284157 285 0.284157 286 0.284096 287 0.284109 288 0.284065 289 0.284066 Step size decreases to 0.005992 after epoch 290. 290 0.284033 291 0.284025 292 0.283978 293 0.28398 294 0.283948 295 0.28394 296 0.283917 Step size increases to 0.006591 after epoch 297. 297 0.283902 298 0.283886 299 0.283888 300 0.283857 301 0.283851 302 0.283827 Step size increases to 0.007250 after epoch 303. 303 0.283816 304 0.283798 305 0.283811 306 0.28377 307 0.283775 Step size decreases to 0.006525 after epoch 308. 308 0.283742 309 0.283741 310 0.283683 311 0.283701 312 0.283657 313 0.283665 Step size decreases to 0.005872 after epoch 314. 314 0.28363 315 0.283631 316 0.283578 317 0.283592 Step size decreases to 0.005285 after epoch 318. 318 0.283553 319 0.283557 320 0.283507 321 0.283517 Step size decreases to 0.004757 after epoch 322. 322 0.283483 323 0.283482 324 0.283441 325 0.283444 326 0.283417 327 0.283411 328 0.283392 Step size increases to 0.005232 after epoch 329. 329 0.28338 330 0.283367 331 0.283368 332 0.283343 333 0.283339 334 0.283318 Step size increases to 0.005755 after epoch 335. 335 0.28331 336 0.283294 337 0.283305 338 0.283271 339 0.283277 Step size decreases to 0.005180 after epoch 340. 340 0.283248 341 0.28325 342 0.283202 343 0.283219 Step size decreases to 0.004662 after epoch 344. 344 0.283181 345 0.28319 346 0.283141 347 0.283157 Step size decreases to 0.004196 after epoch 348. 348 0.28312 349 0.283129 350 0.283085 351 0.283096 Step size decreases to 0.003776 after epoch 352. 352 0.283064 353 0.283068 354 0.283032 355 0.283038 Step size decreases to 0.003399 after epoch 356. 356 0.283012 357 0.283011 358 0.282982 359 0.282983 360 0.282962 361 0.282957 362 0.282942 Step size increases to 0.003738 after epoch 363. 363 0.282934 364 0.282922 365 0.282922 366 0.282903 Step size increases to 0.004112 after epoch 367. 367 0.282899 368 0.282884 369 0.28289 370 0.282866 371 0.282867 Step size decreases to 0.003701 after epoch 372. 372 0.282847 373 0.282845 374 0.282815 375 0.282821 376 0.282798 377 0.282798 Step size decreases to 0.003331 after epoch 378. 378 0.28278 379 0.282777 380 0.282752 381 0.282753 382 0.282735 383 0.282731 384 0.282717 Step size increases to 0.003664 after epoch 385. 385 0.282711 386 0.2827 387 0.282702 388 0.282683 389 0.282681 390 0.282666 Step size increases to 0.004030 after epoch 391. 391 0.282661 392 0.282649 393 0.282656 394 0.282633 395 0.282635 Step size decreases to 0.003627 after epoch 396. 396 0.282616 397 0.282616 398 0.282586 399 0.282593 400 0.282571 401 0.282573 Step size decreases to 0.003265 after epoch 402. 402 0.282555 403 0.282553 404 0.282528 405 0.282531 406 0.282512 407 0.282511 408 0.282497 Step size increases to 0.003591 after epoch 409. 409 0.282492 410 0.282481 411 0.282485 412 0.282465 413 0.282465 414 0.28245 Step size increases to 0.003950 after epoch 415. 415 0.282447 416 0.282434 417 0.282443 418 0.282419 419 0.282424 Step size decreases to 0.003555 after epoch 420. 420 0.282404 421 0.282405 422 0.282375 423 0.282384 Step size decreases to 0.003200 after epoch 424. 424 0.282361 425 0.282364 426 0.282335 427 0.282342 Step size decreases to 0.002880 after epoch 428. 428 0.282321 429 0.282322 430 0.282298 431 0.282301 Step size decreases to 0.002592 after epoch 432. 432 0.282284 433 0.282281 434 0.282262 Step size increases to 0.002851 after epoch 435. 435 0.282261 436 0.282248 437 0.282249 438 0.282234 439 0.28223 440 0.282219 Step size increases to 0.003136 after epoch 441. 441 0.282213 442 0.282204 443 0.282204 444 0.28219 Step size increases to 0.003450 after epoch 445. 445 0.282186 446 0.282175 447 0.28218 448 0.282161 449 0.282161 Step size decreases to 0.003105 after epoch 450. 450 0.282146 451 0.282144 452 0.282121 453 0.282123 454 0.282107 455 0.282105 456 0.282092 Step size increases to 0.003415 after epoch 457. 457 0.282087 458 0.282077 459 0.28208 460 0.282062 461 0.282061 462 0.282047 Step size increases to 0.003757 after epoch 463. 463 0.282043 464 0.282031 465 0.282038 466 0.282016 467 0.282017 Step size decreases to 0.003381 after epoch 468. 468 0.282 469 0.281998 470 0.28197 471 0.281974 472 0.281954 473 0.281952 474 0.281936 Step size increases to 0.003719 after epoch 475. 475 0.28193 476 0.281918 477 0.281921 478 0.281899 479 0.281897 480 0.281879 Step size increases to 0.004091 after epoch 481. 481 0.281873 482 0.281858 483 0.281865 484 0.281838 485 0.281838 Step size decreases to 0.003682 after epoch 486. 486 0.281816 487 0.281811 488 0.281777 489 0.281781 490 0.281755 491 0.281752 492 0.281731 Step size increases to 0.004050 after epoch 493. 493 0.281723 494 0.281707 495 0.28171 496 0.281683 497 0.28168 498 0.281658 Step size increases to 0.004455 after epoch 499. 499 0.28165 500 0.281632 Designated epoch number reached. ANFIS training completed at epoch 500. Minimal training RMSE = 0.281632
outFIS =
sugfis with properties: Name: "sug61" AndMethod: "prod" OrMethod: "probor" ImplicationMethod: "prod" AggregationMethod: "sum" DefuzzificationMethod: "wtaver" DisableStructuralChecks: 0 Inputs: [1×6 fisvar] Outputs: [1×1 fisvar] Rules: [1×7 fisrule] See 'getTunableSettings' method for parameter optimization.
% Predict using trained FIS & display RMSE
Ypred1 = round(evalfis(outFIS, X));
rmse1 = sqrt(mean((Y - Ypred1).^2));
txt1 = sprintf('RMSE of the prediction is %.4f.', rmse1);
disp(txt1)
RMSE of the prediction is 0.3350.
% Identify outliers by finding indices
Outliers = abs(Y - Ypred1); % Forecasts that differ significantly from observations
idx = find(Outliers == 1);
numOutliers = length(idx);
numSamples = length(Y);
Percentage = 100*numOutliers/numSamples;
txt2 = sprintf('%2.2f percent of the %d samples are outliers, which is equal to %d.', Percentage, numSamples, numOutliers);
disp(txt2)
11.22 percent of the 303 samples are outliers, which is equal to 34.
% Get Prediction 2
Ypred2 = Ypred1;
Ypred2(idx) = ~Ypred2(idx);
rmse2 = sqrt(mean((Y - Ypred2).^2));
txt3 = sprintf('RMSE of the prediction with outlier factor is %.4f.', rmse2);
disp(txt3)
RMSE of the prediction with outlier factor is 0.0000.

3 Comments
Show 1 older comment Hide 1 older comment

Ahmad
Ahmad on 11 Oct 2023
Thanks for the response sir. actually the dataset consist of 13 input, but because of the warning I received from matlab that " the inputs are large" i reduced the number of input from 13 to 6. The original dataset is provided in this attacement.
Sam Chak
Sam Chak on 11 Oct 2023
Hi @Ahmad
You are welcome, @Ahmad. However, I'm not a medical doctor and I believe I have already exhausted my mental resources today in attempting to solve the original data training problem as posted. If you find the guided solution and explanation about ANFIS helpful, please consider clicking 'Accept' ✔ on the answer and voting 👍 for it. Thanks a bunch!
As for the new data training problem, which involves significantly higher dimensions with 13 inputs, could you attempt to work through it using the guidance provided earlier? Please share the results or any error messages if encountered.
Ahmad
Ahmad on 11 Oct 2023
ok Sir , I will consider doing so. Thank you so much

Sign in to comment.

More Answers (2)

Sam Chak
Sam Chak on 27 Sep 2023
Ran in:

2 votes

Ran in:
Hi @Ahmad
To achieve a lower prescribed error measure, you can manually increase the number of Membership Functions until the desired error measure is reached. Naturally, I believe that PSO (Particle Swarm Optimization) can be used to find the optimal value for this parameter. However, since this is a relatively simple problem, manual tuning or tuning through iteration can also produce the desired result quickly, as demonstrated in the following example.
x = (0:0.1:10)';
y = sin(2*x)./exp(x/5);
% settings for FIS 1
genOpt1 = genfisOptions('GridPartition');
genOpt1.NumMembershipFunctions = 5; % <-- initial number of MFs
genOpt1.InputMembershipFunctionType = 'gaussmf';
inFIS1 = genfis(x, y, genOpt1);
opt = anfisOptions('InitialFIS', inFIS1);
opt.DisplayANFISInformation = 0;
opt.DisplayErrorValues = 0;
opt.DisplayStepSize = 0;
opt.DisplayFinalResults = 0;
% compute RMSE of FIS 1
outFIS1 = anfis([x y], opt);
yFIS1 = evalfis(outFIS1, x);
Error1 = rmse(yFIS1, y)
Error1 = 0.0095
% settings for FIS 2
genOpt2 = genfisOptions('GridPartition');
genOpt2.NumMembershipFunctions = 25; % <-- number of MFs is increased 5x
genOpt2.InputMembershipFunctionType = 'gaussmf';
inFIS2 = genfis(x, y, genOpt2);
opt = anfisOptions('InitialFIS', inFIS2);
opt.DisplayANFISInformation = 0;
opt.DisplayErrorValues = 0;
opt.DisplayStepSize = 0;
opt.DisplayFinalResults = 0;
% compute RMSE of FIS 2
outFIS2 = anfis([x y], opt);
yFIS2 = evalfis(outFIS2, x);
Error2 = rmse(yFIS2, y)
Error2 = 0.0019
% plots
plot(x, abs(y - yFIS1), x, abs(y - yFIS2)), grid on
xlabel('x')
ylabel('Absolute Error')
legend('FIS 1', 'FIS 2')

7 Comments
Show 5 older comments Hide 5 older comments

Ahmad
Ahmad on 9 Oct 2023
thank you sir, but now i have an issue with my PSO code for more than a week am not able to resolve it. If it is possible, I want to have your contact so that I will explain better Please. am a student, am doing my research now.my email is aarumah234@gmail.com
Sam Chak
Sam Chak on 9 Oct 2023
Hi @Ahmad
Thank you for reaching out and seeking help with your PSO code. I'm happy to help you right here on the forum whenever I feel free. Please feel free to describe the issue you're facing.
I also understand that you're doing research as a student, and I encourage you to share your problem openly on the forum. That way, not only can other experts help you, but others in the forum facing the same issue can also benefit from the discussion.
If you have any specific questions or issues, don't hesitate to ask.
Ahmad
Ahmad on 9 Oct 2023
Edited: Ahmad on 10 Oct 2023
@Sam Chak
ok thank you sir!
I developed my Anfis using a heart disease dataset that has 6 input and 1 output. I want to optimize the ANFIS using PSO. But I kept getting issue that returns an error message and am helpless because i dont know matlab that very much. I will provide the code, the code is separated into two: the main labelled as anfis_optimization_script.m code and function part labelled as setMembershipFunctionsCustom.m code.
The anfis_optimization_script.m code is given below:
% Create a MATLAB script and put the entire code inside it.
% Load your dataset and create an initial ANFIS model
data = readtable('hearts.csv');
X = table2array(data(:, 1:6)); % Input features
Y = table2array(data(:, 7)); % Output
% Set the percentage for training and testing
trainPercentage = 0.8;
testPercentage = 0.2;
% Calculate the number of samples for training and testing
numSamples = size(X, 1);
numTrain = round(trainPercentage * numSamples);
numTest = numSamples - numTrain;
% Split the dataset into training and testing sets
X_train = X(1:numTrain, :);
Y_train = Y(1:numTrain, :);
X_test = X(numTrain+1:end, :);
Y_test = Y(numTrain+1:end, :);
% Define PSO parameters
numParticles = 20; % Number of particles
maxIterations = 30; % Maximum number of iterations
w = 0.5; % Inertia weight
c1 = 1.5; % Cognitive coefficient
c2 = 1.5; % Social coefficient
% Number of membership functions for each input (adjust as needed)
numMFsPerInput = [3 3 2 2 3 3];
% Calculate the total number of membership functions
totalNumMFs = sum(numMFsPerInput);
% PSO initialization
particlePositions = rand(numParticles, totalNumMFs);
particleVelocities = zeros(numParticles, totalNumMFs);
particleBestPositions = particlePositions;
globalBestError = inf;
% Initialize sigma values with a minimum value of 0.1
sigma = 0.1 + 0.1 * rand(1, totalNumMFs);
% Initialize globalBestPosition with a reasonable initial value
globalBestPosition = rand(1, totalNumMFs);
% PSO optimization loop
for iteration = 1:maxIterations
for particle = 1:numParticles
% Initialize startIndex and endIndex here
startIndex = 1;
endIndex = numMFsPerInput(1); % Initialize endIndex to the number of MFs for the first input
currentMFParams = cell(1, numel(numMFsPerInput));
% .....
for i = 1:numel(numMFsPerInput)
if ~isempty(globalBestPosition) % Check if globalBestPosition is not empty
params = globalBestPosition(startIndex:endIndex);
mu = params(1:2:end); % Extract mus
% Assign sigma values from the pre-initialized array
sigma(i) = params(2); % Extract the corresponding sigma
% Debugging: Display sigma values after initialization
disp(['Sigma value after initialization for input ', num2str(i), ':']);
disp(sigma(i));
% Debugging: Display mu values after initialization
disp(['Mu values after initialization for input ', num2str(i), ':']);
disp(mu); % Display the entire array of mu values
else
mu = rand(1, numMFsPerInput(i)); % Initialize mu randomly
% Initialize sigma values only once here
for j = 1:numMFsPerInput(i)
sigma(j) = rand * 0.1 + 0.1; % Random value between 0.1 and 0.2
end
% Debugging: Display mu values after random initialization
disp(['Mu values after random initialization for input ', num2str(i), ':']);
disp(mu); % Display the entire array of mu values
end
% Debugging: Display sigma values after initialization
disp(['Sigma value after initialization for input ', num2str(i), ':']);
disp(sigma(i));
% Store the parameters in the cell array separately
currentMFParams{i} = mu;
currentMFParams{i + numel(numMFsPerInput)} = sigma(i); % Use the correct sigma
% Debugging: Display sigma values before calling setMembershipFunctionsCustom
disp('Before calling setMembershipFunctionsCustom:');
disp(sigma);
% Create an ANFIS model with the current particle's parameters
currentOpt = genfisOptions('GridPartition');
currentOpt.NumMembershipFunctions = numMFsPerInput;
currentOpt.InputMembershipFunctionType = 'gaussmf';
currentFis = genfis(X_train, Y_train, currentOpt);
% Call setMembershipFunctionsCustom
setMembershipFunctionsCustom(currentFis, mu, sigma, numMFsPerInput);
% Debugging: Display sigma values after calling setMembershipFunctionsCustom
disp('After calling setMembershipFunctionsCustom:');
disp(sigma);
% Train the ANFIS model for a fixed number of epochs (epoch)
epoch = 20;
currentFis = anfis([X_train Y_train], currentFis, epoch);
% Evaluate the trained ANFIS model on the training data
predictions = evalfis(X_train, currentFis);
% Calculate the mean squared error (MSE) for the training data
error = mean((Y_train - predictions).^2);
% Update particle's best position and global best position
if error < globalBestError
globalBestError = error;
globalBestPosition = particlePositions(particle, :);
end
if error < mean((Y_train - evalfis(X_train, currentFis)).^2)
particleBestPositions(particle, :) = particlePositions(particle, :);
end
end
% Update particle velocities and positions
for anotherParticle = 1:numParticles % Use a different variable for the nested loop
particleVelocities(anotherParticle, :) = w * particleVelocities(anotherParticle, :) + ...
c1 * rand(1, totalNumMFs) .* (particleBestPositions(anotherParticle, :) - particlePositions(anotherParticle, :)) + ...
c2 * rand(1, totalNumMFs) .* (globalBestPosition - particlePositions(anotherParticle, :));
particlePositions(anotherParticle, :) = particlePositions(anotherParticle, :) + particleVelocities(anotherParticle, :);
end
end
end
% Reshape the global best position to match the size of the MF parameters
startIndex = 1;
endIndex = 0;
currentMFParams = cell(1, numel(numMFsPerInput));
for i = 1:numel(numMFsPerInput)
endIndex = endIndex + numMFsPerInput(i);
params = globalBestPosition(startIndex:endIndex);
mu = params(1:2:end); % Extract mus
sigma = params(2:2:end); % Extract sigmas
% Store the parameters in the cell array
currentMFParams{i} = mu;
currentMFParams{i + numel(numMFsPerInput)} = sigma;
startIndex = endIndex + 1;
end
% Create the final ANFIS model with the optimized parameters
finalOpt = genfisOptions('GridPartition');
finalOpt.NumMembershipFunctions = numMFsPerInput;
finalOpt.InputMembershipFunctionType = 'gaussmf';
finalFis = genfis(X_train, Y_train, finalOpt);
% Use the custom function to set the membership functions
setMembershipFunctionsCustom(currentFis, currentMFParams{1}, currentMFParams{2}, numMFsPerInput);
% Train the final ANFIS model
epoch = 2; % You can adjust the number of epochs as needed
finalFis = anfis([X_train Y_train], finalFis, epoch);
% Evaluate the trained ANFIS model on the testing data (X_test and Y_test)
predictions = evalfis(X_test, finalFis);
% Calculate Mean Squared Error (MSE)
mse = sqrt(mean((Y_test - predictions).^2));
% Display results
fprintf('Optimized ANFIS model MSE: %.4f\n', mse);
The setMembershipFunctionsCustom.m code is :
function setMembershipFunctionsCustom(fis, mu, sigma, numMFsPerInput)
for i = 1:numel(fis.input)
for j = 1:numel(fis.input(i).mf)
% Check if j exceeds the number of membership functions for the current input
if j > numMFsPerInput(i)
error('Index exceeds the number of membership functions for input %d.', i);
end
% Ensure sigma is positive and within bounds
if sigma(j) < 0.1
error('Sigma value is less than 0.1. Check your random initialization.');
end
fis.input(i).mf(j).params = [mu(j), sigma(j)];
end
end
end
The output after running the code is:
ANFIS_Optimization_Script
Sigma value after initialization for input 1:
0.8593
Mu values after initialization for input 1:
0.6311 0.9742
Sigma value after initialization for input 1:
0.8593
Before calling setMembershipFunctionsCustom:
Columns 1 through 13
0.8593 0.1658 0.1951 0.1722 0.1400 0.1832 0.1134 0.1060 0.1084 0.1164 0.1324 0.1302 0.1012
Columns 14 through 16
0.1540 0.1095 0.1147
Warning: Generated rule base is large. MATLAB may run out of memory if this FIS is tuned using ANFIS. > In genfis1 (line 162)
In genfis (line 61)
In ANFIS_Optimization_Script (line 102)
Index exceeds the number of array elements. Index must not exceed 2.
Error in setMembershipFunctionsCustom (line 14)
fis.input(i).mf(j).params = [mu(j), sigma(j)];
Error in ANFIS_Optimization_Script (line 105)
setMembershipFunctionsCustom(currentFis, mu, sigma, numMFsPerInput);
Please Sir, If this is not the best way to go about using PSO in optimizing the ANFIS, suggest a solution. am willing to learn. Thank you. I am happy for your help.
Sam Chak
Sam Chak on 10 Oct 2023
@Ahmad, Your developer's PSO code is fairly sophisticated, especially when you embed 'anfis()' into it, instead of calling 'pso()' to optimize the FIS. Anyway, if you disable 'setMembershipFunctionsCustom()', the code works. Do you know what the function customize when you developed this code?
Ahmad
Ahmad on 10 Oct 2023
@Sam Chak
Sir! concerning the function, you mean if I remove the setMembershipFunctionsCustom(), the code will work ?
Sam Chak
Sam Chak on 10 Oct 2023
I commented out setMembershipFunctionsCustom(), and it works on my data. Honestly, I don't know how your user-supplied PSO code works. The 'anfis()' function itself has a built-in hybrid optimization algorithm, 'Gradient Descent + Least Square Estimation'. If you want me to investigate the data training issue using anfis, please attach the data 'hearts.csv', provided that the patient data is anonymized.
Ahmad
Ahmad on 11 Oct 2023
@Sam Chak
Ok sir I will be happy to send the dataset, its an anonymized dataset. it consist of six inputs and one output. From column one to six is the inputs and the last column is the output. Thank you sir...

Sign in to comment.

Sam Chak
Sam Chak on 12 Oct 2023
Ran in:

0 votes

Ran in:
Hi @Ahmad
For the new data "heart_dataset.csv" with 13 inputs to 1 output, the Subtractive Clustering method will generate 209 rules because each fuzzy input has 209 membership functions. While the number of rules may seem daunting, if the Grid Partitioning method is used with the fewest number of MFs for each input, even more rules will be required—specifically, (2 MFs)^(13 Inputs) = 8192 rules. Nonetheless, with my proposed approach, the RMSE for predicting the 13-input Boolean function is ZERO (0). You can express the ANFIS-based Boolean function as follows:
where the firing strength of a rule k is given by
,
and the first-order conclusion is a linear function given by
,
with to are all constants.
% Observational Data
data = readtable('heart_dataset.csv');
X = table2array(data(:, 1:13)); % 13 Inputs of patient data
Y = table2array(data(:, 14)); % 1 Output (target)
data_Train = [X Y];
% Setting up the initial FIS using Subtractive Clustering method
genOpt = genfisOptions('SubtractiveClustering');
inFIS = genfis(X, Y, genOpt);
anOpt = anfisOptions('InitialFIS', inFIS, 'EpochNumber', 3);
% Training data with ANFIS
outFIS = anfis(data_Train, anOpt)
ANFIS info: Number of nodes: 5868 Number of linear parameters: 2926 Number of nonlinear parameters: 5434 Total number of parameters: 8360 Number of training data pairs: 303 Number of checking data pairs: 0 Number of fuzzy rules: 209
Warning: Number of training data is smaller than number of modifiable parameters.
Start training ANFIS ... 1 2.49751e-05 2 2.64993e-05 3 1.66338e-05 Designated epoch number reached. ANFIS training completed at epoch 3. Minimal training RMSE = 1.66338e-05
outFIS =
sugfis with properties: Name: "sug131" AndMethod: "prod" OrMethod: "probor" ImplicationMethod: "prod" AggregationMethod: "sum" DefuzzificationMethod: "wtaver" DisableStructuralChecks: 0 Inputs: [1×13 fisvar] Outputs: [1×1 fisvar] Rules: [1×209 fisrule] See 'getTunableSettings' method for parameter optimization.
% Predict using trained FIS & display RMSE
Ypred1 = round(evalfis(outFIS, X));
rmse1 = sqrt(mean((Y - Ypred1).^2));
txt1 = sprintf('RMSE of the prediction is %.4f.', rmse1);
disp(txt1)
RMSE of the prediction is 0.0000.

3 Comments
Show 1 older comment Hide 1 older comment

Ahmad
Ahmad on 13 Oct 2023
good evening @Sam Chak , thanks for your response Sir, I really appreciate your effort.
This is my ANFIS model code with membership functions. I want to use PSO algorithm to optimize it. Please take a look at the code. or Should I write it as another seperate question and then mention you? The code is shown below:
% Load your dataset and create an initial ANFIS model
data = readtable('hearts.csv');
X = table2array(data(:, 1:6)); % Input features
Y = table2array(data(:, 7)); % Output
% Set the percentage for training and testing
trainPercentage = 0.8;
testPercentage = 0.2;
% Calculate the number of samples for training and testing
numSamples = size(X, 1);
numTrain = round(trainPercentage * numSamples);
numTest = numSamples - numTrain;
% Split the dataset into training and testing sets
X_train = X(1:numTrain, :);
Y_train = Y(1:numTrain, :);
X_test = X(numTrain+1:end, :);
Y_test = Y(numTrain+1:end, :);
opt = genfisOptions('GridPartition');
opt.NumMembershipFunctions = [4 3 2 2 4 3];
opt.InputMembershipFunctionType = "gaussmf";
fis = genfis(X_train, Y_train, opt);
epoch = 30; % Number of epochs (iterations) for ANFIS training
fis = anfis([X_train Y_train], fis, epoch);
% evaluate the trained ANFIS model using the testing data (X_test and Y_test):
Y_pred_test = evalfis(X_test, fis);
% calculate performance metrics to assess how well the model performs on the testing data. For example, to calculate RMSE
rmse_test = sqrt(mean((Y_pred_test - Y_test).^2));
fprintf('RMSE on Testing Data: %.4f\n', rmse_test);
Sam Chak
Sam Chak on 14 Oct 2023
Hi @Ahmad
I'm feeling a bit confused about the approach that has been taken. The code is roughly 80% identical to the solution, except for the fact that the data is divided in a biased manner into training and cross-validation sets. Additionally, the initial FIS is set up in an unguided manner using the Grid Partitioning method with the user-specified number of Membership Functions (MFs) as [4 3 2 2 4 3] for the six fuzzy inputs.
Since the problem of training the full dataset using ANFIS has been satisfactorily resolved, and you are continuing to pursue the training solution for partial data using PSO, I encourage you to create a New Question and specify the specific problem you are facing. If your PSO works flawlessly for other test problems like the Rosenbrock Function, it is likely not a coding issue but a mathematical formulation problem.
Please ensure that the title of your question is clear, for example: "How can I formulate the optimization problem to train a user-specified model using PSO?" The model can be ANFIS, Convolutional Neural Networks (CNN), or Support Vector Machines (SVM).
Things to consider:
It is still not clear which parameters in the internally-optimized ANFIS are intended to be optimized using PSO. There are many parameters to consider, such as the partitioning technique, number of MFs, input and output MF types, as well as other options specific to the partitioning techniques. Please identify these parameters.
If the mathematical modeling of the patients' heart conditions is unimportant to you, and you don't need to explain the nature or mechanism of the model set, then consider training the data using Neural Networks, where you can employ binary classifiers.
Please be reminded to share your user-developed PSO code for optimization purposes and the MATLAB-built-in ANFIS code for training purposes in the New Question.
Ahmad
Ahmad on 14 Oct 2023
ok sir thank you, I will specify everything in the new question. Thank you

Sign in to comment.

Categories

Find more on Fuzzy Inference System Tuning in Help Center and File Exchange

Products

Release

R2023b

Tags

Asked:

Ahmad
Ahmad
on 27 Sep 2023

Commented:

Ahmad
Ahmad
on 14 Oct 2023

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!