Sending episode number to simulink during training

Hello every one,

I am using reinforcement learning with two agents and I am going to laod the number of episode from workspace in simulink by "from workspace block". I tried trainingStats.EpisodeIndex(1) but I recieved error. As trainingStats would appear after terminating episode 1, I dont have any idea to find the episode number in simulink.

I would appreciate any help.

@Emmanouil Tzorakoleftherakis

[Esan freedom]

@Emmanouil Tzorakoleftherakis

[Benjamin Thompson]

Can you provide a listing of the error message and more examples of what you are doing and how you are doing it?

[Esan freedom]

Thanks at first.

For my reinforcement learning agent, I need to build a special reward for my reinforcement learning agent that starts giving a penalty after episode 100th. For that, I need to read the number of episode in Simulink and if it is below the 100th episode, then the reward is set to zero, and after the 100th episode, It starts getting a penalty.

I have to agents and for example after 4 episods in the Workspace I have this index in the below Fig.

Answer 1

[Emmanouil Tzorakoleftherakis]

The training stats are not available until after training is done, so you cannot use trainingStats.EpisodeIndex during training. The easiest way to accomplish what you want is by using the reset function. Take a look at this example.

The reset function is called before every episode, so you can use it to create a counter for the episode number and assign it either to a workspace variable, or directly to e.g. a Constant block in Simulink.

Hope that helps

[Esan freedom]

Thank you so much Emmanouil,

Exactly it is the way of reading the episode number. However, the counter I am using counts till the end, in every episode. For example here in every episode I have 101. I want only to count one for each episode.

• function in = localResetFcn(in)
• c=0;
• for i = 1:100
• c =c + 1;
• end
• blk = 'PV/Control_rll/counter';
• in = setBlockParameter(in,blk,'Value',num2str(c)); %%% c is the valcue Ive set in a constant in simulink.
• end

[Emmanouil Tzorakoleftherakis]

The reset function is called before every episode, so you don't need to create a for loop yourself. Either make c a persistent variable and increase its value by one, or you can read the value of the constant block, increase it by one, and write it again to the block. Both approaches should work

[Esan freedom]

Dear Emmanouil, you are the best.

I am grateful for your help and time. Thank you so much. By the following code, I have solved my problem.

• function in= localResetFcn(in)
• persistent n
• if isempty(n)
• n = 0;
• end
• n = n+1
• blk = 'PV/Control_rll/count';
• in = setBlockParameter(in,blk,'Value',num2str(n));
• end

[Emmanouil Tzorakoleftherakis]

Glad it was helpful

Answer 2

[Fangjun Jiang]

Try the Constant block and specify the value as trainingStats.EpisodeIndex(1). Read the document of the "From Workspace" block to see if it is what you needed.

[Esan freedom]

Thanks for your help. I received the following error:

Error evaluating parameter 'Value' in 'PV/Control_rll/Constant34'

Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 668)

Unable to resolve the name trainingStats.EpisodeIndex.