How can I set the initial value of action space while using Simulink DDPG Agent?

I got a robot model in Simulink, now I want to train the robot using DDPG Agent.

My question is, how can I set the initial value of action space? I want the action starts from some specefic value such as zero.

Answer 1

[Shlok]

Hi Lei,

DDPG agents are designed to operate in continuous action spaces. Hence, to create a custom continuous action space for the DDPG agent, you can use the “rlNumericSpec” function. “rlNumericSpec” helps in creating specifications object for a numeric action or observation channel. By setting the “LowerLimit” to zero, you can specify that the action starts from there.

Here is a sample code:

actionInfo = rlNumericSpec([1,1], 'LowerLimit', 0, 'UpperLimit', 1); 
observationInfo = ... // define observation specifications 
agtInitOpts = ... // define agent options 
agent = rlDDPGAgent(observationInfo,actionInfo,agtInitOpts); 

To know more about “rlNumericSpec” function, you can refer to the following MATLAB Answer and documentations:

• https://www.mathworks.com/matlabcentral/answers/1962479-mixed-datatype-in-action-space
• https://www.mathworks.com/help/reinforcement-learning/ref/rl.util.rlnumericspec.html