rlTable

Value table or Q table

Description

Value tables and Q-tables are one of the approximation models that can be used within value functions and Q-value functions, respectively. Value tables store rewards for a finite set of observations. Q tables store rewards for corresponding finite observation-action pairs.

To create a value function approximator using an rlTable object, use an rlValueFunction, rlQValueFunction, or rlVectorQValueFunction object.

Creation

Syntax

T = rlTable(obsinfo)

T = rlTable(obsinfo,actinfo)

Description

T = rlTable(obsinfo) creates a value table for the given discrete observations.

T = rlTable(obsinfo,actinfo) creates a Q table for the given discrete observations and actions.

Input Arguments

obsinfo — Observation specification

rlFiniteSetSpec object

Observation specification, specified as an rlFiniteSetSpec object.

actinfo — Action specification

rlFiniteSetSpec object

Action specification, specified as an rlFiniteSetSpec object.

Properties

Table — Reward table

array

Reward table, returned as an array. When Table is a:

• Value table, it contains N_O rows, where N_O is the number of finite observation values.

• Q table, it contains N_O rows and N_A columns, where N_A is the number of possible finite actions.

Object Functions

rlValueFunction	Value function approximator object for reinforcement learning agents
rlQValueFunction	Q-Value function approximator object for reinforcement learning agents
rlVectorQValueFunction	Vector Q-value function approximator for reinforcement learning agents

Examples

Create a Value Table

Create an environment interface, and obtain its observation specifications.

env = rlPredefinedEnv("BasicGridWorld");
obsInfo = getObservationInfo(env)

obsInfo = 
  rlFiniteSetSpec with properties:

       Elements: [25x1 double]
           Name: "MDP Observations"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

Create the value table using the observation specification.

vTable = rlTable(obsInfo)

vTable = 
  rlTable with properties:

    Table: [25x1 double]

You can now use your table as an approximation model for a value function for an agent with a discrete observation space. For more information, see rlValueFunction.

Create a Q Table

This example shows how to use rlTable to create a Q table. Such a table could be used to represent the actor or critic of an agent with finite observation and action spaces.

Create an environment interface, and obtain its observation and action specifications.

env=rlMDPEnv(createMDP(8,["up";"down"]));
obsInfo = getObservationInfo(env)

obsInfo = 
  rlFiniteSetSpec with properties:

       Elements: [8x1 double]
           Name: "MDP Observations"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

actInfo = getActionInfo(env)

actInfo = 
  rlFiniteSetSpec with properties:

       Elements: [2x1 double]
           Name: "MDP Actions"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

Create the Q table using the observation and action specifications.

qTable = rlTable(obsInfo,actInfo)

qTable = 
  rlTable with properties:

    Table: [8x2 double]

You can now use your table as an approximation model for a Q-value function for an agent with discrete action and observation spaces. For more information, see rlQValueFunction.

Version History

Introduced in R2019a