ExperienceBufferLength in Reinforcement Learning Toolbox
17 views (last 30 days)
Show older comments
Hello, everyone,
I found a problem with the 'ExperienceBufferLength' property in 'rlDDPGAgentOptions' when specifying options for rl agents.
In this example, every episode has 600 (60/0.1) steps. Does the agent start to train when the experience buffer is filled up with the experiences (S,A,R,S'). If so, it would take at least 1667 (1000000/600 ) episodes before the agent starts to improve.
So I want to know how to determine this value.
0 Comments
Accepted Answer
Ari Biswas
on 17 Nov 2021
The agent will train until at least one minibatch can be sampled from the buffer. If your mini batch size is 64, then the first learn step will occur after the buffer has stored 64 experiences. The experience buffer is circular, i.e., it removes older experiences when full. The size of the buffer is hence important. You may lose important experiences if the buffer size is too small.
3 Comments
Ari Biswas
on 9 Aug 2022
https://www.mathworks.com/help/reinforcement-learning/ug/train-sac-agent-for-ball-balance-control.html
Arman Ali
on 27 Sep 2022
How about if we want to fill our buffer first and then start taking minibatches?? how to implement this in matlab?
More Answers (0)
See Also
Categories
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!