Experiment Manager - Explore parameters in your MATLAB code and compare experiment results - MATLAB (original) (raw)

Explore parameters in your MATLAB code and compare experiment results

Since R2023b

Description

You can use the Experiment Manager app to create experiments to run your MATLAB® code using various parameter values and compare results. For example, you can use Experiment Manager to explore how the solution to a system of differential equations responds to different coefficient values or how it evolves from different initial conditions.

Experiment Manager helps you set up, execute, and interpret experiments by:

For more information about setting up experiments, watch How to Set Up and Manage Experiments in MATLAB.

Experiment Manager app

Open the Experiment Manager App

Examples

expand all

Create an experiment for a function with two parameters. Run the experiment with different combinations of parameter values. In this case, calculate the speed of an object from two parameters, distance andtime. Then, categorize the speed as either fast or slow.

speed = params.distance / params.time; if speed > 100 info = "Fast"; else info = "Slow"; end

First, create a new project for the experiment. Open the Experiment Manager app and select Blank Project > General Purpose. Specify a filename for the new project file.

Enter a description for the experiment in the Description box, such as:

Calculate the speed of an object. Then, categorize the speed as either fast or slow.

Specify the parameters for the experiment, in this case, distance and time. In the Parameters section, clickAdd. Specify the first parameter name asdistance and the first parameter value as100💯300. Similarly, add another parameter with the nametime and the value 1:3. Experiment Manager will run the experiment once for each unique combination of these parameters.

Then, create the function that defines the experiment. Under Experiment Function, click Edit. An .mlx file named Experiment1Function1 opens in the MATLAB Live Editor. Copy the following code into the function definition. The experiment function uses the params input to access the values of thedistance and time parameters you just specified in the Parameters section. The experiment function returns the outputs speed and info and displays their values in the table of results.

function [speed,info] = Experiment1Function1(params) speed = params.distance / params.time; if speed > 100 info = "Fast"; else info = "Slow"; end end

Save and close the .mlx file. Then, run the experiment by clicking Run in the Experiment Manager toolstrip.

Experiment definition tab containing the experiment description, optional initialization function, parameter names and values, and experiment function name

As Experiment Manager runs each trial, a table displays the execution status, supported actions, elapsed time, parameters, and outputs. The experiment contains a set of results for each time you run the experiment, and each row in the table corresponds to a different combination of parameters for your experiment.

You can use the table to control the execution of the experiment. For example, cancel a single trial that has not yet been run using the Cancel button in theActions column. Alternatively, to end experiment execution before any queued trial is run, click Stop in the Experiment Manager toolstrip.

Table of experiment results, where the status of two trials is Canceled

After your experiment is finished running, you can reduce the size of your experiment by discarding the result for any trial that is no longer relevant by clicking the Discard button in the Actions column.

You can customize the table of results by using the Show or hide columns button above the table.

You can analyze experiment results in various ways, such as displaying a visualization, sorting a column, adding an annotation, or applying a filter.

Display Visualization

You can display a visualization to interpret the result of a single trial. For example, to visualize the difference between the distance andtime parameters, edit the experiment function to include code that creates a plot for each trial. Specify the visualization name by setting theName property of the figure. Then, rerun your experiment.

function [speed,info] = Experiment1Function1(params) speed = params.distance / params.time; if speed > 100 info = "Fast"; color = "green"; else info = "Slow"; color = "red"; end

figure(Name="Object Speed") plot([0 params.time],[0 params.distance],Color=color) legend(info) xlim([0 3]) ylim([0 300]) title("Object Speed Is " + speed) xlabel("Time") ylabel("Distance")

end

Then, select a trial to visualize from the table of results. In the Review Results section of the Experiment Manager toolstrip, click a visualization name, and the plot appears in theVisualizations panel. To update the visualization to show results for a different trial, select the trial in the table of results.

Result tab showing trial 3 selected in the table of results and the corresponding visualization of distance against time in the Visualizations panel

Sort Column

You can interpret the values in a column of the table of results by sorting its values. For example, determine which trial yields the maximum object speed by sorting the speed output in descending order. In the table of results, point to the header of the speed variable, click the triangle icon, and select the sorting order.

Add Annotation

You can record an observation by adding an annotation to the table of results. For example, record the combination of parameters that yields the maximum speed. Select the table cell that contains the maximum value of speed, and in theExperiment Manager toolstrip, select Annotations > Add Annotation. Enter the annotation text for the trial, such as:

Annotations panel showing that the speed parameter of trial 3 has the maximum object speed

Apply Filter

You can display only a subset of trials in the table of results by applying a filter to a column. For example, display only trials where the object speed is fast. In theExperiment Manager toolstrip, click Filters. To apply a filter, in the info variable section of theFilters panel, type the text Fast.

Filters panel showing the info variable section with the text filter "Fast"

Each time you run an experiment, you can change parameter values and modify functions. This flexibility allows you to explore different configurations and their effects on your results. After running an experiment, you can access or revert to the parameter values and the specific version of the functions that produced those results.

To access the artifacts for an experiment result, in the Experiment Browser panel, double-click the name of the set of results. Then, in the Result tab, click View Experiment Source.

Open files located in the project folder that are used by the result by clicking the links at the bottom of the Source tab. These files are read-only, but you can copy them to the project folder, rerun the experiment, and reproduce your results.

Parameters

expand all

Enter a description of the experiment.

The initialization function is a function that you create to configure data or other experiment details before initiating the trial runs. Create a new blank initialization function by clicking New, or open and modify an existing initialization function by clicking Edit.

The function cannot accept any inputs. The function must return one output argument, such as a scalar value or a structure array. Access the initialization function output in your experiment function by usingparams.InitializationFunctionOutput.

Enter the names and values of the parameters used in the experiment function.

Parameter names must start with a letter, followed by letters, digits, or underscores.

Example: distance

Example: a_2

Parameter values must be scalars or vectors. Experiment Manager performs an exhaustive sweep by executing a trial for each unique combination of parameter values in the table.

Example: 0.01

Example: 0.01:0.01:0.05

Example: [0.01 0.02 0.04 0.08]

Example: ["alpha" "beta" "gamma"]

Data Types: double | single | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | logical | string | char

The experiment function is a function that you create to define your experiment and parameters. Experiment Manager runs the experiment function once for each unique combination of the parameter values. Create or modify the experiment function by clicking Edit.

The input variable is a structure params with fields from theParameters table. Access the parameter values using dot notation. For example, if you add a parameter to the Parameters table with the name distance, then use params.distance to access it.

You can return outputs from your function. The variables names of your outputs appear as column headers in the results table. For example, if your experiment function returns the speed and info outputs, then the results table also uses those variable names.

[speed,info] = Experiment1Function1(params)

The Outputs column in the table of experiment results contains two nested columns for the speed and info variables

You can also add visualizations for your experiment by creating a figure in the experiment function. You can specify the name of the visualization by setting theName property of the figure.

Identify, add, or remove files required by your experiment.

In most cases, Experiment Manager automatically detects required files and adds them to your project. The Detected Files list updates after you run the experiment or you click Refresh. The list displays the relative path for files in the project folder and the full path for files outside of the project folder.

If Experiment Manager does not detect some of your supporting files, your trials will produce an error. You can manually select files to include by clickingAdd in the Additional Files section.

More About

expand all

If you have Deep Learning Toolboxâ„¢, you can configure an experiment for your AI workflow. For more information, see Experiment Manager (Deep Learning Toolbox).

If you have Statistics and Machine Learning Toolboxâ„¢, you can configure an experiment using the Classification Learner or the Regression Learner app. You can also execute your experiment using the Bayesian optimization or random sampling strategy. For more information, see Experiment Manager (Statistics and Machine Learning Toolbox).

If you have Parallel Computing Toolboxâ„¢, you can run multiple trials at the same time, or you can run a single trial on multiple GPUs, on a cluster, or in the cloud. For more information, see Run Experiments in Parallel.

If you have MATLAB Parallel Serverâ„¢, you can offload experiments as batch jobs to a remote cluster. For more information, see Offload Experiments as Batch Jobs to a Cluster.

Tips

You can select one of these general-purpose experiment templates from the start page, indicated by the Orange round-bottom flask icon:

Version History

Introduced in R2023b

expand all

You can set up your experiment using an improved interface.

You can also efficiently analyze experiment results.

Add files required by your experiment using the Supporting Files section of the experiment definition tab.

After running an experiment, the Detected Files section lists detected supporting files. If Experiment Manager does not detect some of your supporting files, your trials will produce an error. You can manually select files to include by clicking Add in the Additional Files section. You can also update the Detected Files section by clickingRefresh.

Apply or reset a filter for a variable in the results table by selectingFilters in the toolstrip and configuring the corresponding section of the Filters panel. Set the bounds for a numeric variable, or specify the contents of a character array or string variable. Previously, you could apply a filter only to numeric variables and you could not reset the numeric filter.

Filters panel in Experiment Manager. A text filter for the value "Newton" is applied for the Model parameter, and a numeric filter for values less than 54.7917 is applied for the Theta parameter.

Previously, Experiment Manager required Deep Learning Toolbox.

See Also

Apps

Topics