using rlSimulinkEnv reset function: how to access and modify variables in the matlab workspace

I am training (or at least trying to) a DQN agent to act on a simulink model to react in some way to an input time series. I would like to train the model on a different random time series generated at the beginning of each training episode.

The time series is read from a Matlab workspace matrix with time column and value column as required and I have the matlab script that generates a random series.

The problem is that if I put my function in the initFcn callback for the system it seems that it is called only once at the beginning of the training process (so that I get always the same time series for each training episode), if I try to put the same function in the localResetFunction I defined for my rlSimulinkEnv, it does not seem to have access to the Matlab workspace.

Is there some way to do this ? I essentially need to call a script that modifies a Matlab workspace variable at the begining of each training episode.

Answer 1

[Matteo D'Ambrosio]

Hello,

After you generate the RL environment, i assume you are adding the environment reset function as

env = rlSimulinkEnv(...)
env.ResetFcn = @(in) myResetFunction(in, otherInputs)

Inside of your myResetFunction.m, you can generate your desired matrix at the start of each episode, and then it needs to be added to the Simulink workspace. I found this to be important especially when randomizing parameters in the environment at the start of each episode.

To do this, inside of your myResetFunction use:

function in = myResetFunction(in, otherInputs)

% Here run the function to generate your matrix
NewMatrix = GenerateMatrix(otherInputs) ;
    
% Then, add this matrix to your Simulink workspace
in = setVariable( in, 'NewMatrix', NewMatrix, 'Workspace', SimulinkModel) ;
% where SimulinkModel is the string containing the name of the Simulink
% model that you are using

end

You should now be able to access the variable NewMatrix inside of Simulink during training, randomized according to GenerateMatrix.m at the start of each episode.

If you wish to randomize other parameters as well, use the exact same procedure, and then add them to the Simulink workspace.

Hope this helps.

[Giancarlo Storti Gajani]

Thank you !

I did not know that one could pass the reset function additional variables and that one could specify the workspace in setVariable.

Now it is working as expected.