Experiment Manager

Explore parameters in your MATLAB code and compare experiment results

Since R2023b

Description

You can use the Experiment Manager app to create experiments to run your MATLAB^® code using various parameter values and compare results. For example, you can use Experiment Manager to explore how the solution to a system of differential equations responds to different coefficient values or how it evolves from different initial conditions.

Experiment Manager helps you set up, execute, and interpret experiments by:

• Organizing multiple experiments and the experiment artifacts and results in projects

• Providing visualizations, filters, and annotations for comparing results

• Storing the experiment definition and parameter combinations for each experiment result

For more information about setting up experiments, watch How to Set Up and Manage Experiments in MATLAB.

×

Open the Experiment Manager App

• MATLAB Toolstrip: On the Apps tab, under MATLAB, click the Experiment Manager icon.

• MATLAB command prompt: Enter experimentManager.

Examples

Create Experiment

Create an experiment for a function with two parameters. Run the experiment with different combinations of parameter values. In this case, calculate the speed of an object from two parameters, distance and time. Then, categorize the speed as either fast or slow.

speed = params.distance / params.time;
if speed > 100
   info = "Fast";
else
   info = "Slow";
end

First, create a new project for the experiment. Open the Experiment Manager app and select Blank Project > General Purpose. Specify a filename for the new project file.

Enter a description for the experiment in the Description box, such as:

Calculate the speed of an object.
Then, categorize the speed as either fast or slow.

Specify the parameters for the experiment, in this case, distance and time. In the Parameters section, click Add. Specify the first parameter name as distance and the first parameter value as 100:100:300. Similarly, add another parameter with the name time and the value 1:3. Experiment Manager will run the experiment once for each unique combination of these parameters.

Then, create the function that defines the experiment. Under Experiment Function, click Edit. An .mlx file named Experiment1Function1 opens in the MATLAB Live Editor. Copy the following code into the function definition. The experiment function uses the params input to access the values of the distance and time parameters you just specified in the Parameters section. The experiment function returns the outputs speed and info and displays their values in the table of results.

function [speed,info] = Experiment1Function1(params)
speed = params.distance / params.time;
if speed > 100
   info = "Fast";
else
   info = "Slow";
end
end

Save and close the .mlx file. Then, run the experiment by clicking Run in the Experiment Manager toolstrip.

Control Experiment Execution

As Experiment Manager runs each trial, a table displays the execution status, supported actions, elapsed time, parameters, and outputs. The experiment contains a set of results for each time you run the experiment, and each row in the table corresponds to a different combination of parameters for your experiment.

You can use the table to control the execution of the experiment. For example, cancel a single trial that has not yet been run using the Cancel button in the Actions column. Alternatively, to end experiment execution before any queued trial is run, click Stop in the Experiment Manager toolstrip.

After your experiment is finished running, you can reduce the size of your experiment by discarding the result for any trial that is no longer relevant by clicking the Discard button in the Actions column.

You can customize the table of results by using the Show or hide columns button above the table.

Analyze Results

You can analyze experiment results in various ways, such as displaying a visualization, sorting a column, adding an annotation, or applying a filter.

Display Visualization

You can display a visualization to interpret the result of a single trial. For example, to visualize the difference between the distance and time parameters, edit the experiment function to include code that creates a plot for each trial. Specify the visualization name by setting the Name property of the figure. Then, rerun your experiment.

function [speed,info] = Experiment1Function1(params)
speed = params.distance / params.time;
if speed > 100
   info = "Fast";
   color = "green";
else
   info = "Slow";
   color = "red";
end

figure(Name="Object Speed")
plot([0 params.time],[0 params.distance],Color=color)
legend(info)
xlim([0 3])
ylim([0 300])
title("Object Speed Is " + speed)
xlabel("Time")
ylabel("Distance")

end

Then, select a trial to visualize from the table of results. In the Review Results section of the Experiment Manager toolstrip, click a visualization name, and the plot appears in the Visualizations panel. To update the visualization to show results for a different trial, select the trial in the table of results.

Sort Column

You can interpret the values in a column of the table of results by sorting its values. For example, determine which trial yields the maximum object speed by sorting the speed output in descending order. In the table of results, point to the header of the speed variable, click the triangle icon, and select the sorting order.

Add Annotation

You can record an observation by adding an annotation to the table of results. For example, record the combination of parameters that yields the maximum speed. Select the table cell that contains the maximum value of speed, and in the Experiment Manager toolstrip, select Annotations > Add Annotation. Enter the annotation text for the trial, such as:

Maximum object speed.

Apply Filter

You can display only a subset of trials in the table of results by applying a filter to a column. For example, display only trials where the object speed is fast. In the Experiment Manager toolstrip, click Filters. To apply a filter, in the info variable section of the Filters panel, type the text Fast.

Access Artifacts for Experiment Result

Each time you run an experiment, you can change parameter values and modify functions. This flexibility allows you to explore different configurations and their effects on your results. After running an experiment, you can access or revert to the parameter values and the specific version of the functions that produced those results.

To access the artifacts for an experiment result, in the Experiment Browser panel, double-click the name of the set of results. Then, in the Result tab, click View Experiment Source.

Open files located in the project folder that are used by the result by clicking the links at the bottom of the Source tab. These files are read-only, but you can copy them to the project folder, rerun the experiment, and reproduce your results.

Related Examples

• Convert MATLAB Code into Experiment
• Compare Air Resistance Models for Projectile Motion
• Experiment with Predator-Prey Equations

Parameters

Description — Experiment description

text

Enter a description of the experiment.

Initialization Function — Function to call before trial runs

.mlx file | .m file

The initialization function is a function that you create to configure data or other experiment details before initiating the trial runs. Create a new blank initialization function by clicking New, or open and modify an existing initialization function by clicking Edit.

The function cannot accept any inputs. The function must return one output argument, such as a scalar value or a structure array. Access the initialization function output in your experiment function by using params.InitializationFunctionOutput.

Parameters — Parameters used in experiment function

names and values of parameters

Enter the names and values of the parameters used in the experiment function.

Parameter names must start with a letter, followed by letters, digits, or underscores.

Example: distance

Example: a_2

Parameter values must be scalars or vectors. Experiment Manager performs an exhaustive sweep by executing a trial for each unique combination of parameter values in the table.

Example: 0.01

Example: 0.01:0.01:0.05

Example: [0.01 0.02 0.04 0.08]

Example: ["alpha" "beta" "gamma"]

Data Types: double | single | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | logical | string | char

Experiment Function — Function to call for each trial

.mlx file | .m file

The experiment function is a function that you create to define your experiment and parameters. Experiment Manager runs the experiment function once for each unique combination of the parameter values. Create or modify the experiment function by clicking Edit.

The input variable is a structure params with fields from the Parameters table. Access the parameter values using dot notation. For example, if you add a parameter to the Parameters table with the name distance, then use params.distance to access it.

You can return outputs from your function. The variables names of your outputs appear as column headers in the results table. For example, if your experiment function returns the speed and info outputs, then the results table also uses those variable names.

[speed,info] = Experiment1Function1(params)

You can also add visualizations for your experiment by creating a figure in the experiment function. You can specify the name of the visualization by setting the Name property of the figure.

Supporting Files — Files required by experiment

filename

Identify, add, or remove files required by your experiment.

In most cases, Experiment Manager automatically detects required files and adds them to your project. The Detected Files list updates after you run the experiment or you click Refresh. The list displays the relative path for files in the project folder and the full path for files outside of the project folder.

If Experiment Manager does not detect some of your supporting files, your trials will produce an error. You can manually select files to include by clicking Add in the Additional Files section.

More About

AI Workflows

If you have Deep Learning Toolbox™, you can configure an experiment for your AI workflow. For more information, see Experiment Manager (Deep Learning Toolbox).

Machine Learning Workflows

If you have Statistics and Machine Learning Toolbox™, you can configure an experiment using the Classification Learner or the Regression Learner app. You can also execute your experiment using Bayesian optimization. For more information, see Experiment Manager (Statistics and Machine Learning Toolbox).

Parallel Workflows

If you have Parallel Computing Toolbox™, you can run multiple trials at the same time, or you can run a single trial on multiple GPUs, on a cluster, or in the cloud. For more information, see Run Experiments in Parallel.

If you have MATLAB Parallel Server™, you can offload experiments as batch jobs to a remote cluster. For more information, see Offload Experiments as Batch Jobs to a Cluster.

Tips

You can select one of these general-purpose experiment templates from the start page, indicated by the icon:

• General Purpose — Create a blank general-purpose experiment.

• Solve System of Ordinary Differential Equations — Create a general-purpose experiment that solves a system of ordinary differential equations by sweeping over the values of two parameters. (since R2024b)

Version History

Introduced in R2023b

R2024b: Improved interface for experiment setup and result analysis

You can set up your experiment using an improved interface.

• Create an initialization function that configures data or other experiment details before initiating the trial runs to reduce trial runtime. Previously, setup code was rerun for each trial.

• Incorporate a suggested parameter for your experiment for some experiment templates.

You can also efficiently analyze experiment results.

• Your experiment function for a general-purpose experiment can return an output containing multiple data types.

• When an experiment runs using the sequential execution mode, you can stop or cancel the experiment. Clicking the Stop button finishes the current trial and ends experiment execution. Clicking the Cancel button immediately ends experiment execution.

• When you select a trial in the results table, you can export the selected trial row to the MATLAB workspace as a table.

R2024a: Add supporting files

Add files required by your experiment using the Supporting Files section of the experiment definition tab.

After running an experiment, the Detected Files section lists detected supporting files. If Experiment Manager does not detect some of your supporting files, your trials will produce an error. You can manually select files to include by clicking Add in the Additional Files section. You can also update the Detected Files section by clicking Refresh.

R2024a: Apply and reset filters for numeric, character array, and string variables in results table

Apply or reset a filter for a variable in the results table by selecting Filters in the toolstrip and configuring the corresponding section of the Filters panel. Set the bounds for a numeric variable, or specify the contents of a character array or string variable. Previously, you could apply a filter only to numeric variables and you could not reset the numeric filter.

R2023b: Moved to MATLAB from Deep Learning Toolbox

Previously, Experiment Manager required Deep Learning Toolbox.