rlVectorEnv

R2026b

Create a vectorized environment from a scalar environment

Since R2026b

Description

Use rlVectorEnv to create a vectorized reinforcement learning environment by supplying an environment factory function that creates a scalar MATLAB^® environment. Vectorized environments use built-in vectorization features to step and reset many environment instances in batch. This capability can improve hardware utilization and enable higher data throughput compared to non-vectorized (that is, single-instance, or scalar) environments. For agents that use large experience batches, training against a vectorized environment can be much faster than training against a corresponding scalar version of the environment. After creating the environment, call validateEnvironment to check that the environment is configured correctly.

Note

Vectorized environments created using rlFunctionVectorEnv typically perform better than equivalent environments created with rlVectorEnv.

Creation

Syntax

venv = rlVectorEnv(envFactoryFcn)

venv = rlFunctionVectorEnv(envFactoryFcn,NumEnv=N)

Description

venv = rlVectorEnv(envFactoryFcn) creates the vectorized reinforcement learning environment venv using the factory function envFactoryFcn.

venv = rlFunctionVectorEnv(envFactoryFcn,NumEnv=N) also specifies the number of environment instances.

Input Arguments

envFactoryFcn — Factory function for scalar environment

function handle

Factory function for scalar environment, specified as a function handle.

The factory function is a function that creates a scalar MATLAB environment. The function must have one input and one output, as illustrated by the following signature.

senv = myEnvFactoryFcn(info)

Here, info is a structure containing the fields EnvIdx (this is the index of the specific environment instance that is returned) and NumEnv (the total number of environments).

You can use info.EnvIdx and info.NumEnv in the factory function to return an environment with different characteristics (as long as the observation and action specifications remain the same). For example, you can return an environment with a different parameter depending on the specific value of the environment instance index info.NumEnv

Example: @(info) rlPredefinedEnv("CartPole-Continuous")

N — Number of environment instances

2 (default) | positive integer

Number of environment instances, specified as a positive integer. This argument sets the NumEnv property. If you omit this argument, the function uses the default value of 2.

Example: NumEnv=128

Properties

NumEnv — Number of environment instances

2 (default) | positive integer

Number of environment instances, specified as a positive integer.

Example: 128

Object Functions

rlFunctionVectorEnv	Create a vectorized reinforcement learning environment using your setup, reset, and step functions
convertToScalarEnv	Convert a vectorized environment to a scalar environment
validateEnvironment	Validate custom reinforcement learning environment

Examples

Create Vectorized Environment from Scalar Environment

For this example, use rlPredefinedEnv to create a cart-pole environment with a continuous action space, and then use rlVectorEnv to create a vectorized version of that environment.

As a first argument, define in place an anonymous function that calls the environment factory function rlPredefinedEnv. This anonymous function returns the predefined cart-pole environment independently of info.

Use the name-value NumEnv argument to create 128 environment instances.

vcpenv = rlVectorEnv(@(info) rlPredefinedEnv("CartPole-Continuous"),NumEnv=128)

vcpenv = 
  rlVectorEnv with properties:

    NumEnv: 128

You can now create agents for vcpenv and train or simulate them as you would for any other environment.

Alternatively, to returned environment instances that have different parameters, use a handle to a custom factory function, customFactoryFcn, defined at the end of the example. This function returns environment instances with varying mass ranging from 0.874 to 1.128. It uses the variable info.EnvIdx to create a different mass every time it is invoked.

n = 128;
vcpenv_massrange = rlVectorEnv(@(info) customFactoryFcn(info),NumEnv=n)

vcpenv_massrange = 
  rlVectorEnv with properties:

    NumEnv: 128

You can use this environment to train an agent that balances the pole with a policy that is robust to uncertainties in cart mass.

Custom Factory Function

function env = customFactoryFcn(info)
    env = rlPredefinedEnv("CartPole-Continuous");
    env.MassCart = 1 + 1e-3*(2*info.EnvIdx-info.NumEnv);
end

Limitations

• Setting the UseParallel training option to on is not supported with vectorized environments.

• Training an agent using an evolution strategy (trainWithEvolutionStrategy) is not supported with vectorized environments.

• Training Q-Learning, SARSA, LSPI, PG and AC agents is not supported with vectorized environments.

• Training agents with actors or critics that use recursive neural networks is not supported with vectorized environments.

Version History

Introduced in R2026b