Trying to modify the custom class rl.env.MATLABEnvironment

Hello guys,

I am working on RL problem. however, I am using rlCreateEnvTemplate("MyEnvironment") and I want to modify the properties according to my scenario. however, I am following the steps exactly in the document but at any time I am running, again it takes me to Cart - Pole scenario. how to customize to my own scenario??

Answer 1

[Hari]

Hi Shahd,

I understand that you are trying to modify the properties of a custom class "rl.env.MATLABEnvironment" created using "rlCreateEnvTemplate("MyEnvironment")" for your RL problem.

To acheive this, you can follow the below steps:

1. Create a custom environment (say MyEnvironment) instead of using the built-in CartPole environment. "MyEnvironment" class can be made as a child class for "rl.env.MATLABEnvironment" class, which is a base class for creating custom MATLAB environments in Reinforcement Learning Toolbox. You can add variables in this custom class which could define the properties of your environment.

For example,

% Define your custom environment class
classdef MyEnvironment < rl.env.MATLABEnvironment
    properties
        % Modify properties according to your scenario
        Property1 = 10;
        Property2 = 'Sample';
    end
end

2. Initilize the environment using the constructor of "MyEnvironment":

function this = MyEnvironment()
    % Define the action and observation information
    actionInfo = rlNumericSpec([1, 1]);
    observationInfo = rlNumericSpec([1, 1]);
    
    % Call the superclass constructor with actionInfo and observationInfo
    this = [e-mail](actionInfo, observationInfo);
end
        

The superclass constructor [e-mail](actionInfo, observationInfo) is called to initialize the environment with the specified action and observation information.

3. Step through the environment: Modify the "step" method to define the behavior of your environment. In the below example, the "step" function takes an action as input and calculates the next observation, reward, and whether the episode is done.

function [observation, reward, isDone, info] = step(this, action)
    % Modify the step function to define the behavior of your environment
    % e.g., Update the state based on the action, calculate the reward, etc.
    observation = action * this.Property1;
    reward = this.Property1 / action;
    isDone = false;
    info = 'Step completed successfully.';
end

4. The reset method can be modified to reset the environment to its initial state.

function reset(this)
    % Modify the reset function if needed
    % e.g., Reset the state to the initial state
    disp('Environment reset.');

After defining the MyEnvironment class, an instance of the environment is created using env = MyEnvironment();. The environment can be tested by stepping through it in a loop until the episode is done. By following this workflow, you can define your custom environment and interact with it using the provided methods.

Refer to the documentation of "rl.env.MATLABEnvironment" for more information on how to create and customize your own environment: https://www.mathworks.com/help/reinforcement-learning/ug/create-custom-environment-from-class-template.html

Hope this helps!