Tune FIS with Training Data

In the example contained in the Fuzzy logic user guide documentation by mathworks, Tune Fuzzy Inference System at the Command Line, page 225, I understand the code used for tuning the FIS, but i dont know how they come up with tunedfismpgprediction.mat . below is the sample code:

[data,name] = loadGasData; 
X = data(:,1:6); 
Y = data(:,7);
trnX = X(1:2:end,:); % Training input data set 
trnY = Y(1:2:end,:); % Training output data set 
vldX = X(2:2:end,:); % Validation input data set
vldY = Y(2:2:end,:); % Validation output data set
dataRange = [min(data)' max(data)'];
fisin = mamfis;
for i = 1:6
    fisin = addInput(fisin,dataRange(i,:),'Name',name(i),'NumMFs',2); 
end 
fisin = addOutput(fisin,dataRange(7,:),'Name',name(7),'NumMFs',64);

figure 
plotfis(fisin)

options = tunefisOptions('Method','particleswarm',... 
'OptimizationType','learning', ... 
'NumMaxRules',64);
options.MethodOptions.MaxIterations = 20; 

rng('default') 

runtunefis = false;
%% This is the stage where am confused, I dont know how they get tunedfismpgprediction.mat 
if runtunefis 
    fisout1 = tunefis(fisin,[],trnX,trnY,options); %#ok
else 
    tunedfis = load('tunedfismpgprediction.mat');
    fisout1 = tunedfis.fisout1; 
    fprintf('Training RMSE = %.3f MPG\n',calculateRMSE(fisout1,trnX,trnY));
end
plotfis(fisout1) 

[Ahmad]

@Sam Chak

@Walter Roberson

Answer 1

[Sam Chak]

Hi @Ahmad

Long story short, the MATLAB data file 'tunedfismpgprediction.mat' contains 2 pre-trained FIS files because this is a two-stage tuning process. In the MATLAB Fuzzy Logic Toolbox, developers have already tuned the FIS so that you can directly load them to see the results. You only need those FIS files when the 'runtunefis' parameter is set to 'false'. See my annotations below. Took me more than an hour to find facts and prepare the annotated code.

You can read the article in this link for more details:

https://www.mathworks.com/help/fuzzy/tune-mamdani-fuzzy-inference-system.html

%% Load automobile fuel consumption data (https://archive.ics.uci.edu/dataset/9/auto+mpg)
[data,name] = loadGasData;  % previous MATLAB versions used loadgas
X           = data(:,1:6); 
Y           = data(:,7);
trnX        = X(1:2:end,:); % Training input data set 
trnY        = Y(1:2:end,:); % Training output data set 
vldX        = X(2:2:end,:); % Validation input data set
vldY        = Y(2:2:end,:); % Validation output data set
dataRange   = [min(data)' max(data)'];

%% Create a Mamdani FIS for tuning
fisin = mamfis;
for i = 1:6
    fisin = addInput(fisin, dataRange(i,:), 'Name', name(i), 'NumMFs', 2); 
end 
fisin = addOutput(fisin, dataRange(7,:), 'Name', name(7), 'NumMFs', 64);

figure 
plotfis(fisin)

%% Setting
runtunefis = true;      % set true if you really want to learn tuning FIS from scratch
% runtunefis = false;   % set false to load pre-trained results if you can't wait 5 min

%% Stage 1: Learn the rule base using Particle Swarm optimizer, while keeping the input and output MF parameters constant
options = tunefisOptions('Method', 'particleswarm', 'OptimizationType', 'learning', 'NumMaxRules', 64);
options.MethodOptions.MaxIterations = 20; 

if runtunefis 
    rng('default')
    fisout1  = tunefis(fisin, [], trnX, trnY, options); % carry out the tuning
else                    % if set false previously, it loads the already tuned FIS
    tunedfis = load('tunedfismpgprediction.mat');       % there are 2 FIS files
    fisout1  = tunedfis.fisout1;                        % the PSO-tuned FIS in Stage 1 is named 'fisout1.fis'
    fprintf('Training RMSE = %.3f MPG\n',calculateRMSE(fisout1, trnX, trnY));
end

figure
plotfis(fisout1)                % view the PSO-tuned FIS
[fisout1.Rules.Description]'    % view all tuned 64 Rules, if you like
plotActualAndExpectedResultsWithRMSE(fisout1, vldX, vldY)   % calculate the RMSE to check accuracy

%% Stage 2: Use the rule base from Stage 1 to tune the parameters of the input/output MFs and rules using the Pattern Search optimizer
[in, out, rule]                       = getTunableSettings(fisout1);
options.OptimizationType              = 'tuning';
options.Method                        = 'patternsearch';
options.MethodOptions.MaxIterations   = 60;
options.MethodOptions.UseCompletePoll = true;

if runtunefis   % if previously set true, it will tune Stage 1 FIS using Pattern Search optimizer
    rng('default')
    fisout = tunefis(fisout1, [in; out; rule], trnX, trnY, options); 
else            % if previously set false, it will retrieve the already tuned Stage 2 FIS 
    fisout = tunedfis.fisout;   % the Pattern Search-tuned FIS in Stage 2 is named 'fisout.fis'
    fprintf('Training RMSE = %.3f MPG\n',calculateRMSE(fisout, trnX, trnY));
end

figure
plotfis(fisout)                 % view the Pattern Search-tuned FIS
plotActualAndExpectedResultsWithRMSE(fisout, vldX, vldY);   % calculate the RMSE to check accuracy


%  There are 2 Local functions that you need to create
%% Local function #1
function plotActualAndExpectedResultsWithRMSE(fis, x, y)

    % Calculate RMSE bewteen actual and expected results
    [rmse, actY] = calculateRMSE(fis, x, y);
    
    % Plot results
    figure
    subplot(2,1,1)
    hold on
    bar(actY)
    bar(y)
    bar(min(actY, y),'FaceColor', [0.5 0.5 0.5])
    hold off
    axis([0 200 0 60])
    xlabel("Validation input dataset index"),
    ylabel("MPG")
    legend(["Actual MPG" "Expected MPG" "Minimum of actual and expected values"], 'Location', 'NorthWest')
    title("RMSE = " + num2str(rmse) + " MPG")
    
    subplot(2,1,2)
    bar(actY-y)
    xlabel("Validation input dataset index"),ylabel("Error (MPG)")
    title("Difference Between Actual and Expected Values")

end

%% Local function #2 (this one can be embedded in the local function #1)
function [rmse, actY] = calculateRMSE(fis, x, y)

    % Specify options for FIS evaluation
    persistent evalOptions
    if isempty(evalOptions)
        evalOptions = evalfisOptions("EmptyOutputFuzzySetMessage", "none", "NoRuleFiredMessage", "none", "OutOfRangeInputValueMessage", "none");
    end
    
    % Evaluate FIS
    actY = evalfis(fis, x, evalOptions);
    
    % Calculate RMSE 
    del  = actY - y;
    rmse = sqrt(mean(del.^2));  % the rmse() function was introduced in R2022b
    
    % See https://www.mathworks.com/help/matlab/ref/rmse.html

end

[Ahmad]

@Sam Chak

@@Walter Roberson

Thanks for your responses, both of your responses really help me, it works where you tell me to set the runtunefis to true and where you tell me to add save('tunedfismpgprediction.mat', 'fisout1');, but due to the time consuming in learning the rules, I want to know how to tune the FIS seperately, then set the runtunefis to false. I tried tuning the FIS on the fuzzy logic designer, but may be am getting the whole thing wrong because i received the error message:

" Error using load Unable to read file 'tunedfismpgprediction.mat'. Input must be a MAT-file or an ASCII file containing numeric data with same number of columns in each row. "

so my main concern is how to tune the FIS, how did they get (tunedfismpgprediction.mat), I want to know how they worked it out. because they just shown the final result (which is tunedfismpgprediction.mat) without showing how they tuned it. Thank you and I really appreciate both of your responses. I want you to help me answer my question then I marked the question all answered

[Sam Chak]

Hi @Ahmad

... you tell me to set the runtunefis to true and where you tell me to add save('tunedfismpgprediction.mat', 'fisout1');,

... I want to know how to tune the FIS seperately, then set the runtunefis to false.

I think you contradicted yourself in these two lines. But without seeing your code, it's just speculation. You have two options in the original MATLAB code, but I don't want to give you an option here because you will contradict yourself and won't properly learn how to tune the FIS.

Note that 'tunedfismpgprediction.mat' is not needed in learning the FIS-tuning process. Because of the reasons stated above, just follow this modified code to tune both FIS separately (no If–Else option to choose!):

%% Load automobile fuel consumption data (https://archive.ics.uci.edu/dataset/9/auto+mpg)
[data,name] = loadGasData;  % previous MATLAB versions used loadgas
X           = data(:,1:6); 
Y           = data(:,7);
trnX        = X(1:2:end,:); % Training input data set 
trnY        = Y(1:2:end,:); % Training output data set 
vldX        = X(2:2:end,:); % Validation input data set
vldY        = Y(2:2:end,:); % Validation output data set
dataRange   = [min(data)' max(data)'];

%% Create a Mamdani FIS for tuning
fisin = mamfis;
for i = 1:6
    fisin = addInput(fisin, dataRange(i,:), 'Name', name(i), 'NumMFs', 2); 
end 
fisin = addOutput(fisin, dataRange(7,:), 'Name', name(7), 'NumMFs', 64);

figure 
plotfis(fisin)

%% Stage 1: Learn the rule base using Particle Swarm optimizer, while keeping the input and output MF parameters constant
options = tunefisOptions('Method', 'particleswarm', 'OptimizationType', 'learning', 'NumMaxRules', 64);
options.MethodOptions.MaxIterations = 20; 

rng('default')
fisout1  = tunefis(fisin, [], trnX, trnY, options); % carry out the tuning
fprintf('Training RMSE = %.3f MPG\n', calculateRMSE(fisout1, trnX, trnY));

figure
plotfis(fisout1)                % view the PSO-tuned FIS
[fisout1.Rules.Description]'    % view all tuned 64 Rules, if you like
plotActualAndExpectedResultsWithRMSE(fisout1, vldX, vldY)   % calculate the RMSE to check accuracy

%% Stage 2: Use the rule base from Stage 1 to tune the parameters of the input/output MFs and rules using the Pattern Search optimizer
[in, out, rule]                       = getTunableSettings(fisout1);
options.OptimizationType              = 'tuning';
options.Method                        = 'patternsearch';
options.MethodOptions.MaxIterations   = 60;
options.MethodOptions.UseCompletePoll = true;

rng('default')
fisout = tunefis(fisout1, [in; out; rule], trnX, trnY, options); 
fprintf('Training RMSE = %.3f MPG\n',calculateRMSE(fisout, trnX, trnY));

figure
plotfis(fisout)                 % view the Pattern Search-tuned FIS
plotActualAndExpectedResultsWithRMSE(fisout, vldX, vldY);   % calculate the RMSE to check accuracy


%  There are 2 Local functions that you need to create
%% Local function #1
function plotActualAndExpectedResultsWithRMSE(fis, x, y)

    % Calculate RMSE bewteen actual and expected results
    [rmse, actY] = calculateRMSE(fis, x, y);
    
    % Plot results
    figure
    subplot(2,1,1)
    hold on
    bar(actY)
    bar(y)
    bar(min(actY, y),'FaceColor', [0.5 0.5 0.5])
    hold off
    axis([0 200 0 60])
    xlabel("Validation input dataset index"),
    ylabel("MPG")
    legend(["Actual MPG" "Expected MPG" "Minimum of actual and expected values"], 'Location', 'NorthWest')
    title("RMSE = " + num2str(rmse) + " MPG")
    
    subplot(2,1,2)
    bar(actY-y)
    xlabel("Validation input dataset index"),ylabel("Error (MPG)")
    title("Difference Between Actual and Expected Values")

end

%% Local function #2 (this one can be embedded in the local function #1)
function [rmse, actY] = calculateRMSE(fis, x, y)

    % Specify options for FIS evaluation
    persistent evalOptions
    if isempty(evalOptions)
        evalOptions = evalfisOptions("EmptyOutputFuzzySetMessage", "none", "NoRuleFiredMessage", "none", "OutOfRangeInputValueMessage", "none");
    end
    
    % Evaluate FIS
    actY = evalfis(fis, x, evalOptions);
    
    % Calculate RMSE 
    del  = actY - y;
    rmse = sqrt(mean(del.^2));  % the rmse() function was introduced in R2022b
    
    % See https://www.mathworks.com/help/matlab/ref/rmse.html

end

[Ahmad]

thanks for your response, you have answered my question, I will write a seperate question again concerning this code.

[Ahmad]

@Sam Chak concerning the last code you provided, can I use anfis at the end to tune the parameters of the genetic algorithm? if yes, then I will write a seperate question on that. or

can I first use anfis to train the dataset (learn the rules), then use either genetic algorithm or particle swarm to tune the parameters of the ANFIS? Thank you Sir

[Sam Chak]

Hi @Ahmad

It is unnecessary to open a new question because it is directly related to the issue of selecting the desired tuning algorithm. I have provided a second answer (with code) below. If you find the solution helpful, please consider giving it a 👍 vote. Thanks a bunch!

Answer 2

[Sam Chak]

Hi @Ahmad,

You have the option to select one of the five tuning algorithms as shown below:

• "ga" — genetic algorithm
• "particleswarm" — particle swarm
• "patternsearch" — pattern search
• "simulannealbnd" — simulated annealing algorithm
• "anfis" — adaptive neuro-fuzzy

Note that the first four tuning algorithms require the Global Optimization Toolbox, while the "anfis" method is a built-in algorithm in the Fuzzy Logic Toolbox.

In the following code, the "anfis" method is used to learn the rule base in Stage 1, and the result is employed in Stage 2 to tune the parameters of the fuzzy system using the "ga" method.

%% Load automobile fuel consumption data (https://archive.ics.uci.edu/dataset/9/auto+mpg)
[data,name] = loadGasData;  % previous MATLAB versions used loadgas
X           = data(:,1:6); 
Y           = data(:,7);
trnX        = X(1:2:end,:); % Training input data set 
trnY        = Y(1:2:end,:); % Training output data set 
vldX        = X(2:2:end,:); % Validation input data set
vldY        = Y(2:2:end,:); % Validation output data set
dataRange   = [min(data)' max(data)'];

%% Create a Mamdani FIS for tuning
fisin = mamfis;
for i = 1:6
    fisin = addInput(fisin, dataRange(i,:), 'Name', name(i), 'NumMFs', 2); 
end 
fisin = addOutput(fisin, dataRange(7,:), 'Name', name(7), 'NumMFs', 64);

figure 
plotfis(fisin)

%% Stage 1: Learn only the rule base of the FIS using ANFIS
options = tunefisOptions('Method', 'anfis', 'OptimizationType', 'learning', 'NumMaxRules', 64);
options.MethodOptions.MaxIterations = 20; 

rng('default')
fisout1  = tunefis(fisin, [], trnX, trnY, options); % carry out the tuning
fprintf('Training RMSE = %.3f MPG\n', calculateRMSE(fisout1, trnX, trnY));

figure
plotfis(fisout1)                % view the PSO-tuned FIS
[fisout1.Rules.Description]'    % view all tuned 64 Rules, if you like
plotActualAndExpectedResultsWithRMSE(fisout1, vldX, vldY)   % calculate the RMSE to check accuracy

%% Stage 2: Use rule base from Stage 1 to tune FIS parameters using Genetic Algorithm
[in, out, rule]                       = getTunableSettings(fisout1);
options.OptimizationType              = 'tuning';
options.Method                        = 'ga';       % Genetic Algorithm
options.MethodOptions.MaxIterations   = 60;
options.MethodOptions.UseCompletePoll = true;

rng('default')
fisout = tunefis(fisout1, [in; out; rule], trnX, trnY, options); 
fprintf('Training RMSE = %.3f MPG\n', calculateRMSE(fisout, trnX, trnY));

figure
plotfis(fisout)                 % view the Pattern Search-tuned FIS
plotActualAndExpectedResultsWithRMSE(fisout, vldX, vldY);   % calculate the RMSE to check accuracy


%  There are 2 Local functions that you need to create
%% Local function #1
function plotActualAndExpectedResultsWithRMSE(fis, x, y)

    % Calculate RMSE bewteen actual and expected results
    [rmse, actY] = calculateRMSE(fis, x, y);
    
    % Plot results
    figure
    subplot(2,1,1)
    hold on
    bar(actY)
    bar(y)
    bar(min(actY, y),'FaceColor', [0.5 0.5 0.5])
    hold off
    axis([0 200 0 60])
    xlabel("Validation input dataset index"),
    ylabel("MPG")
    legend(["Actual MPG" "Expected MPG" "Minimum of actual and expected values"], 'Location', 'NorthWest')
    title("RMSE = " + num2str(rmse) + " MPG")
    
    subplot(2,1,2)
    bar(actY-y)
    xlabel("Validation input dataset index"),ylabel("Error (MPG)")
    title("Difference Between Actual and Expected Values")

end

%% Local function #2 (this one can be embedded in the local function #1)
function [rmse, actY] = calculateRMSE(fis, x, y)

    % Specify options for FIS evaluation
    persistent evalOptions
    if isempty(evalOptions)
        evalOptions = evalfisOptions("EmptyOutputFuzzySetMessage", "none", "NoRuleFiredMessage", "none", "OutOfRangeInputValueMessage", "none");
    end
    
    % Evaluate FIS
    actY = evalfis(fis, x, evalOptions);
    
    % Calculate RMSE 
    del  = actY - y;
    rmse = sqrt(mean(del.^2));  % the rmse() function was introduced in R2022b
    
    % See https://www.mathworks.com/help/matlab/ref/rmse.html

end

[Ahmad]

@Sam Chak

After running this recent code, I received the following error message:

Warning: ANFIS uses only default value of 'OptimizationType' property. > In tunefisOptions>throwANFISWarning (line 770)

In tunefisOptions/throwWarningForOptimizationType (line 643)

In tunefisOptions/set.OptimizationType (line 457)

In tunefisOptions (line 372)

In samchakCode (line 23) Warning: ANFIS uses only default value of 'NumMaxRules' property. > In tunefisOptions>throwANFISWarning (line 770)

In tunefisOptions/throwWarningForNumMaxRules (line 649)

In tunefisOptions/set.NumMaxRules (line 476)

In tunefisOptions (line 373)

In samchakCode (line 23)

Unrecognized property 'MaxIterations' for class 'fuzzy.anfis.ANFISOptions'.

Error in samchakCode (line 24)

options.MethodOptions.MaxIterations = 20;

[Sam Chak]

@Ahmad, Apologies, I didn't test run the code previously. I forgot that "anfis" is a bit special.

Make the following changes, and it should work.

%% Stage 1: Learn only the rule base of the FIS using ANFIS
options = tunefisOptions('Method', 'anfis');
% options.MethodOptions.MaxIterations = 20;     % comment out this line

[Ahmad]

@Sam Chak

after editing it, its still showing an error message, I think the error is because there is no rules in the fisin, the anfis is supposed to come after using PSO to learn the rules. This is the error message:

Error using tunefis

FIS must have rules.

Error in samchakcode (line 27)

fisout1 = tunefis(fisin, [], trnX, trnY, options); % carry out the tuning

[Michael Bamidele]

%% Stage 2: Use rule base from Stage 1 to tune FIS parameters using Genetic Algorithm

[in, out, rule] = getTunableSettings(fisout1);

options.OptimizationType = 'tuning';

options.Method = 'ga'; % Genetic Algorithm

%options.MethodOptions.MaxIterations = 60;

%options.MethodOptions.UseCompletePoll = true;

%comment out these two lines for the codes to run well

Answer 3

[Walter Roberson]

After

if runtunefis 
    fisout1 = tunefis(fisin,[],trnX,trnY,options); %#ok

and before the else there is an implied

    save('tunedfismpgprediction.mat', 'fisout1');