plannerBenchmark

Benchmark path planners using generated metrics

Since R2022a

Description

plannerBenchmark object benchmarks the 2-D path planners by running them on a specified environment with specified start and goal poses.

The plannerBenchmark object calculates the following metrics:

• clearance — Minimum distance to obstacles in the environment

• executionTime — Time taken by plan function to execute

• initializationTime — Time taken by initialization function to execute

• isPathValid — If true represent the path exists and is collision free

• pathLength — Length of the generated path

• smoothness — Smoothness of the path for all poses

The metrics like executionTime and initializationTime are calculated during the execution of planners. The metrics like clearance, isPathValid, pathLength, and smoothness are calculated from the resulting path outputs after executing planners. Calculated metrics are statistically summarized as a table and can be visualized as plots.

Creation

Syntax

plannerBM = plannerBenchmark(environment,start,goal)

Description

plannerBM = plannerBenchmark(environment,start,goal) creates a plannerBenchmark object with the specified environment, start and goal poses. The inputs environment, start, and goal sets the Environment, Start, and Goal properties, respectively.

Properties

Environment — Environment for benchmarking path planners

occupancyMap object | binaryOccupancyMap object | validatorOccupancyMap object

Environment for benchmarking path planners, specified as occupancyMap, binaryOccupancyMap, or validatorOccupancyMap object.

Start — Start pose of path for all planners

vector of the form [x y] | vector of the form [x y theta]

Start pose of path for all planners, specified as a vector of the form [x y] or [x y theta].

Data Types: single | double

Goal — Goal pose of path for all planners

vector of the form [x y] | vector of the form [x y theta]

Goal pose of path for all planners, specified as a vector of the form [x y] or [x y theta].

Data Types: single | double

PlannerOutput — Output of planners after execution

Read-only: structure

This property is read-only.

Output of planners after execution, returned as a structure that contains the initialization function output and plan function output for all planners. The fields of the structure are named by planner name specified in addPlanner function.

Each structure contains an initialization output and a plan function output. The plan function output is further a structure containing plan function output for each run of the planner.

Data Types: struct

Object Functions

addPlanner	Add path planner for benchmarking
copy	Create deep copy of plannerBenchmark object
metric	Return path planner metrics
report	Create benchmark report
runPlanner	Run path planners
show	Visualize path planner metrics

Examples

Benchmark 2-D Path Planners

Create an occupancy map from an example map.

load("exampleMaps.mat","simpleMap");
map = occupancyMap(simpleMap);

Create a state validator with stateSpaceSE2 using the map.

sv = validatorOccupancyMap(stateSpaceSE2,Map=map);

Specify the start and goal states.

start = [5 8 pi/2];
goal = [7 18 pi/2];

Create a plannerBenchmark object.

pbo = plannerBenchmark(sv,start,goal);

Define the function handles for the initialization functions of the planners.

plannerHAFcn = @(sv)plannerHybridAStar(sv);
plannerRRTSFcn = @(sv)plannerRRTStar(sv.StateSpace,sv);

Define the function handle for the plan function, which is common for both planners.

plnFcn = @(initOut,s,g)plan(initOut,s,g);

Add the path planners for benchmarking.

addPlanner(pbo,plnFcn,plannerHAFcn);
addPlanner(pbo,plnFcn,plannerRRTSFcn,PlannerName="ppRRTStar");

Set the rng for repetitive results.

rng('default')

Run the path planners for the number of times specified in runCount to collect metrics.

runCount = 5;
runPlanner(pbo,runCount)

Initializing plannerHAFcn_plnFcn ...
Done.
Planning a path from the start pose (5 8 1.5708) to the goal pose (7 18 1.5708) using plannerHAFcn_plnFcn.
Executing run 1.
Executing run 2.
Executing run 3.
Executing run 4.
Executing run 5.
Initializing ppRRTStar ...
Done.
Planning a path from the start pose (5 8 1.5708) to the goal pose (7 18 1.5708) using ppRRTStar.
Executing run 1.
Executing run 2.
Executing run 3.
Executing run 4.
Executing run 5.

Access path length metric for all the runs on the environment.

[pLenSummary,pLenData] = metric(pbo,"pathLength")

pLenSummary=2×4 table
                            Mean     Median    StdDev     sampleSize
                           ______    ______    _______    __________

    plannerHAFcn_plnFcn    10.349    10.349          0        5     
    ppRRTStar               12.84      12.9    0.67112        5     

pLenData=2×5 table
                            Run1      Run2      Run3      Run4      Run5 
                           ______    ______    ______    ______    ______

    plannerHAFcn_plnFcn    10.349    10.349    10.349    10.349    10.349
    ppRRTStar                13.8      12.9      12.2        12      13.3

Visualize all the metrics.

show(pbo)

Closely inspect the clearance metric.

figure
show(pbo,"clearance")

Version History

Introduced in R2022a