Create a Deep Learning Experiment for Classification - MATLAB & Simulink (original) (raw)

This example shows how to train a deep learning network for classification by using Experiment Manager. In this example, you train two networks to classify images of MathWorks merchandise into five classes. Each network is trained using three algorithms. In each case, a confusion matrix compares the true classes for a set of validation images with the classes predicted by the trained network. For more information on training a network for image classification, see Retrain Neural Network to Classify New Images.

This experiment requires the Deep Learning Toolbox™ Model for GoogLeNet Network support package. Before you run the experiment, install this support package by calling the googlenet function and clicking the download link.

Open Experiment

First, open the example. Experiment Manager loads a project with a preconfigured experiment that you can inspect and run. To open the experiment, in the Experiment Browser pane, double-click ClassificationExperiment.

Built-in training experiments consist of a description, a table of hyperparameters, a setup function, and a collection of metric functions to evaluate the results of the experiment. For more information, see Train Network Using trainnet and Display Custom Metrics.

The Description field contains a textual description of the experiment. For this example, the description is:

Merchandise image classification using:

The Hyperparameters section specifies the strategy and hyperparameter values to use for the experiment. When you run the experiment, Experiment Manager trains the network using every combination of hyperparameter values specified in the hyperparameter table. This example uses two hyperparameters:

The Setup Function section specifies a function that configures the training data, network architecture, and training options for the experiment. To open this function in MATLAB® Editor, click Edit. The code for the function also appears in Setup Function. The input to the setup function is a structure with fields from the hyperparameter table. The function returns three outputs that you use to train a network for image classification problems. In this example, the setup function has these sections:

filename = "MerchData.zip"; dataFolder = fullfile(tempdir,"MerchData"); if ~exist(dataFolder,"dir") unzip(filename,tempdir); end

imdsTrain = imageDatastore(dataFolder, ... IncludeSubfolders=true, .... LabelSource="foldernames");

numTrainingFiles = 0.7; [imdsTrain,imdsValidation] = splitEachLabel(imdsTrain,numTrainingFiles);

switch params.Network case "default" inputSize = [227 227 3]; numClasses = 5;

    layers = [
        imageInputLayer(inputSize)
        convolution2dLayer(5,20)
        batchNormalizationLayer
        reluLayer
        fullyConnectedLayer(numClasses)
        softmaxLayer
        classificationLayer];
    
case "googlenet"
    inputSize = [224 224 3];
    numClasses = 5;
    
    imdsTrain = augmentedImageDatastore(inputSize(1:2),imdsTrain);
    imdsValidation = augmentedImageDatastore(inputSize(1:2), ...
        imdsValidation);
    
    net = googlenet;
    layers = layerGraph(net);
    
    newLearnableLayer = fullyConnectedLayer(numClasses, ...
        Name="new_fc", ...
        WeightLearnRateFactor=10, ...
        BiasLearnRateFactor=10);
    layers = replaceLayer(layers,"loss3-classifier",newLearnableLayer);
    
    newClassLayer = classificationLayer(Name="new_classoutput");
    layers = replaceLayer(layers,"output",newClassLayer);
   
otherwise
    error("Undefined network selection.");

end

options = trainingOptions(params.Solver, ... MiniBatchSize=10, ... MaxEpochs=8, ... InitialLearnRate=1e-4, ... Shuffle="every-epoch", ... ValidationData=imdsValidation, ... ValidationFrequency=5, ... Verbose=false);

The Metrics section specifies optional functions that evaluate the results of the experiment. This example does not include any custom metric functions.

Run Experiment

When you run the experiment, Experiment Manager trains the network defined by the setup function six times. Each trial uses a different combination of hyperparameter values. By default, Experiment Manager runs one trial at a time. If you have Parallel Computing Toolbox™, you can run multiple trials at the same time or offload your experiment as a batch job in a cluster:

A table of results displays the accuracy and loss for each trial.

To display the training plot and track the progress of each trial while the experiment is running, under Review Results, click Training Plot.

Evaluate Results

To find the best result for your experiment, sort the table of results by validation accuracy:

  1. Point to the Validation Accuracy column.
  2. Click the triangle icon.
  3. Select Sort in Descending Order.

The trial with the highest validation accuracy appears at the top of the results table.

To display the confusion matrix for this trial, select the top row in the results table and, under Review Results, click Validation Data.

To record observations about the results of your experiment, add an annotation:

  1. In the results table, right-click the Validation Accuracy cell of the best trial.
  2. Select Add Annotation.
  3. In the Annotations pane, enter your observations in the text box.

Close Experiment

In the Experiment Browser pane, right-click MerchandiseClassificationProject and select Close Project. Experiment Manager closes the experiment and results contained in the project.

Setup Function

This function configures the training data, network architecture, and training options for the experiment. The input to this function is a structure with fields from the hyperparameter table. The function returns three outputs that you use to train a network for image classification problems.

function [imdsTrain,layers,options] = ClassificationExperiment_setup(params)

Load Training Data

filename = "MerchData.zip"; dataFolder = fullfile(tempdir,"MerchData"); if ~exist(dataFolder,"dir") unzip(filename,tempdir); end

imdsTrain = imageDatastore(dataFolder, ... IncludeSubfolders=true, .... LabelSource="foldernames");

numTrainingFiles = 0.7; [imdsTrain,imdsValidation] = splitEachLabel(imdsTrain,numTrainingFiles);

Define Network Architecture

switch params.Network case "default" inputSize = [227 227 3]; numClasses = 5;

    layers = [
        imageInputLayer(inputSize)
        convolution2dLayer(5,20)
        batchNormalizationLayer
        reluLayer
        fullyConnectedLayer(numClasses)
        softmaxLayer
        classificationLayer];
    
case "googlenet"
    inputSize = [224 224 3];
    numClasses = 5;
    
    imdsTrain = augmentedImageDatastore(inputSize(1:2),imdsTrain);
    imdsValidation = augmentedImageDatastore(inputSize(1:2), ...
        imdsValidation);
    
    net = googlenet;
    layers = layerGraph(net);
    
    newLearnableLayer = fullyConnectedLayer(numClasses, ...
        Name="new_fc", ...
        WeightLearnRateFactor=10, ...
        BiasLearnRateFactor=10);
    layers = replaceLayer(layers,"loss3-classifier",newLearnableLayer);
    
    newClassLayer = classificationLayer(Name="new_classoutput");
    layers = replaceLayer(layers,"output",newClassLayer);
   
otherwise
    error("Undefined network selection.");

end

Specify Training Options

options = trainingOptions(params.Solver, ... MiniBatchSize=10, ... MaxEpochs=8, ... InitialLearnRate=1e-4, ... Shuffle="every-epoch", ... ValidationData=imdsValidation, ... ValidationFrequency=5, ... Verbose=false);

See Also

Apps

Functions