MATLAB Answers

Create and Train DQN Agent with just a State Path and Not Action Path

10 views (last 30 days)
Every example I have seen of a DQN on MATLAB is with two inputs, the state and action. However, it is possible for DQN RL to be done with just one input, the state but there are no examples for that case. How can that be done on MATLAB? My Input would basically be a binary vector and my output would be that I can do two actions?


Sign in to comment.

Accepted Answer

Emmanouil Tzorakoleftherakis
This page shows how this can be done in 20a. We will have examples that show this workflow in the next release.
Hope that helps.


Show 6 older comments
Huzaifah Shamim
Huzaifah Shamim on 6 Jul 2020
Ok perfect I will look into the custom training loop. Is there any advice you have regarding that (to adjust/configure it for multiple actions)?
A question based off of that is also: Could I treat my agent as one (even if many) but it have multiple actions related to it, and based off of that it could be a sort of multi agent problem?
Emmanouil Tzorakoleftherakis
This sounds doable.You may even be able to do this without custom loops using built-in agents (something like centralized multi-agent RL). You can use a single agent and at each step extract the appropriate action and apply it to the appropriate part of the environment. The tricky part is (typical of multi-agent RL) to pick the right amount of observation to make sure your process is Markov. This will likely require observations from each 'subagent' etc.
Huzaifah Shamim
Huzaifah Shamim on 6 Jul 2020
Oh ok interesting.
One last question (Hopefully), in this link, LoggedSignals is used as the State. However lets say I wanted LoggedSignals to contain three matrices or three vectors, one of them being the actual state. Could I contain those three matrices in LoggedSignal and then just let observation have the actual next state. Additionally if that is allowed, how would I define my observation info, would I use rlFiniteSetSpec or rlNumericSpec (My offical state would just be a binary vector of size N but I want to have two other matrices, x and y lets say, so that I can utlilze them in myStepFunction.m)

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!