Reinforcement Learning Toolbox: defining Size of layers and other parameters.

9 views (last 30 days)
hello,
i am trying to use Reinforcement learning toolbox for a vehicle where i have two inputs /observations for the Reinforcement learning toolbox-DQN Agent and one action to be taken by the DQN Agent. The action has discrete 40 discrete value. Three hiddens layers are to be used with sizes of 20,50 and 100.
Questions:
  1. how to define hidden layers size in the script? is it the size of "fullyConnectedLayer(L,'Name','fc1')"
  2. what should be the size parameter for " imageInputLayer([2 1 1],'Normalization','none','Name','state')" ?
  3. my simulation gives error of
Error using rl.env.AbstractEnv/sim (line 121)
An error occurred while simulating "RL_hev_control"with the agent "agent".
Error in RL_HEVsimulink (line 79)
experience = sim(env,agent,simOptions);
Error using Simulink.SimulationInput/sim Not enough room in the buffer to store the new experiences. Make sure the bufferSize argument is big enough.
What could be the mistake i am doing ?
  1 Comment
Raunak Gupta
Raunak Gupta on 18 Mar 2020
Hi,
Can you share the script (‘.m’ file) and Model (‘.slx’ file) written by you that can reproduce the same issue. It will helpful for me to solve the problem.

Sign in to comment.

Answers (1)

Emmanouil Tzorakoleftherakis
Hello,
I would recomment looking at a similar example in thw documentation (e.g. this one). To your questions:
  1. Yes. Number of neurons are specified as an argument in the fullyConnectedLayer
  2. Reinforcement Learning Toolbox currently uses the imageInputLayer even for non-image inputs (there is ongoing work to add more descriptive layers here). In any case, similar to the example I link to above, your Q network will need one channel for the observations and another for the actions, so you will need to have two imageInputLayers. For the action one, dimension should be [1 1 1], and for the observation you should have [2 1 1].
  3. Is this after training or during training? Looks like it's after training, but at that point the experience buffer should not play a role. This post looks relevant. Have you called 'train' at all?
  1 Comment
Aysegul Kahraman
Aysegul Kahraman on 2 Nov 2021
Hello Emmanouil,
This problem seems quite common, however, there is no particular solution as far as I see. And error comes after training when it is wanted to simulate. The only solution is not applying the sim which does not seem like a proper solution, including the post that you highlighted.

Sign in to comment.

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!