How Does MATLAB Internally Format Actions as dlarray in DDPG with Recurrent Networks (LSTM)?

In MATLAB's RL toolbox, when using DDPG with LSTM-based actors/critics, the conversion of actions to dlarray is handled automatically. Since users cannot directly control this process:

Are actions formatted with 'T' (time) or 'C' (channel) dimensions when passed between the actor and critic networks?

How does MATLAB structure actions for compatibility with recurrent layers (e.g., aligning sequences for LSTM time steps)?

Answer 1

[praguna manvi]

Hi @Farid,

In the functions "getAction" and "getValue" for the "actor" and "critic" networks, respectively, the inputs/observations are reshaped and formatted into "CBT" format in the following case of sequential layer network inputs, such as when using "lstm" layer. This ensures the data is in the format that the networks expect in general. To explore this further, you can use the example below:

openExample('rl/CreateDDPGAgentUsingRecurrentNeuralNetworksExample

This example will provide more insights into how the data is structured and processed within these networks when we look underneath these functions.

[Farid]

Thank you for time and your help