The problem of agent decision frequency during reinforcement learning assessment

I use reinforcement learning to interact with the simulink environment to output three discrete actions, and when I evaluate, the whole process model will only output one action fixedly, but when training, it can output different decisions in real time. I suspect that my agent has only one action due to the low decision-making frequency during evaluation, where can I set the decision-making frequency of the model during evaluation, just like the agent during training.

agent.SampleTime = 0.01;

Answer 1

[Sameer]

Hi @浩文

It seems like the issue is due to the "decision-making frequency" of your agent during evaluation.

In Simulink-based reinforcement learning, the agent chooses a new action only once every "SampleTime" seconds. This is controlled by the property:

agent.SampleTime = 0.01;  % Example: new action every 0.01 seconds

There is no separate "evaluation frequency" setting, evaluation uses the same "SampleTime" as training.

If the "SampleTime" is large, the agent will hold the same action for many simulation steps, which can look like it’s stuck on one action.

To match training behavior during evaluation:

1. Set the agent’s "SampleTime" before simulating:

agent.SampleTime = 0.01;  % Or your desired frequency

2. Match the model’s solver step size to the agent’s "SampleTime" (e.g., fixed-step solver with step size 0.01).

3. Run the evaluation using "sim" or "rlSimulationOptions" the agent will now take decisions at the same rate as in training.

Hope this helps!