trainYOLOv2ObjectDetector - Train YOLO v2 object detector - MATLAB (original) (raw)

Train YOLO v2 object detector

Syntax

Description

[trainedDetector](#mw%5F0ff9d26a-9a79-4333-a1b1-ba03c8539ee3%5Fsep%5Fmw%5Ffa1e6d4d-90ec-44f5-a8e8-69fb6c1a8207) = trainYOLOv2ObjectDetector([trainingData](#mw%5Fc1779815-4489-49c9-9786-05ec692442df),[detector](#mw%5F728fbd77-7d10-4e02-ac83-6a90ba114e31),[options](#mw%5Ff678a727-b7cb-4bf9-a79a-f592e2c40411)) returns an object detector trained using the you only look once version 2 (YOLO v2) network specified by detector. The options argument specifies training parameters for the detection network.

You can use this syntax for training an untrained detector or for fine-tuning a pretrained detector.

example

[trainedDetector](#mw%5F0ff9d26a-9a79-4333-a1b1-ba03c8539ee3%5Fsep%5Fmw%5Ffa1e6d4d-90ec-44f5-a8e8-69fb6c1a8207) = trainYOLOv2ObjectDetector([trainingData](#mw%5Fc1779815-4489-49c9-9786-05ec692442df),[checkpoint](#mw%5F1eecf535-37d0-44f5-a563-4b5c3469090d),[options](#mw%5Ff678a727-b7cb-4bf9-a79a-f592e2c40411)) resumes training from the saved detector checkpoint.

You can use this syntax to:

• Add more training data and continue the training.
• Improve training accuracy by increasing the maximum number of iterations.

[[trainedDetector](#mw%5F0ff9d26a-9a79-4333-a1b1-ba03c8539ee3%5Fsep%5Fmw%5Ffa1e6d4d-90ec-44f5-a8e8-69fb6c1a8207),[info](#mw%5F0ff9d26a-9a79-4333-a1b1-ba03c8539ee3%5Fsep%5Fmw%5F548f86bc-a0e5-49c3-a5a3-330c0afeceeb)] = trainYOLOv2ObjectDetector(___) also returns information on the training progress, such as the training accuracy and learning rate for each iteration.

___ = trainYOLOv2ObjectDetector(___,[Name=Value](#namevaluepairarguments)) uses additional options specified by one or more name-value arguments and any of the previous inputs. For example, ExperimentMonitor=[] specifies not to track metrics with the Experiment Manager (Deep Learning Toolbox) app.

Examples

collapse all

Load the training data for vehicle detection into the workspace.

data = load("vehicleTrainingData.mat"); trainingData = data.vehicleTrainingData;

Specify the directory in which training samples are stored. Add full path to the file names in training data.

dataDir = fullfile(toolboxdir("vision"),"visiondata"); trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);

Randomly shuffle data for training.

rng(0) shuffledIdx = randperm(height(trainingData)); trainingData = trainingData(shuffledIdx,:);

Create an imageDatastore using the files from the table.

imds = imageDatastore(trainingData.imageFilename);

Create a boxLabelDatastore using the label columns from the table.

blds = boxLabelDatastore(trainingData(:,2:end));

Combine the datastores.

Specify the class names using the label columns from the table.

classes = trainingData.Properties.VariableNames(2:end);

Specify anchor boxes.

anchorBoxes = [8 8; 32 48; 40 24; 72 48];

Load a preinitialized YOLO v2 object detection network.

load("yolov2VehicleDetectorNet.mat","net");

Create the YOLO v2 object detection network.

detector = yolov2ObjectDetector(net,classes,anchorBoxes)

detector = yolov2ObjectDetector with properties:

              Network: [1×1 dlnetwork]
            InputSize: [128 128 3]
    TrainingImageSize: [128 128]
          AnchorBoxes: [4×2 double]
           ClassNames: vehicle
ReorganizeLayerSource: ''
          LossFactors: [5 1 1 1]
            ModelName: ''

Configure the network training options.

options = trainingOptions("sgdm", ... InitialLearnRate=0.001, ... Verbose=true, ... MiniBatchSize=16, ... MaxEpochs=30, ... Shuffle="never", ... VerboseFrequency=30, ... CheckpointPath=tempdir);

Train the YOLO v2 network.

[trainedDetector,info] = trainYOLOv2ObjectDetector(ds,detector,options);

Training a YOLO v2 Object Detector for the following object classes:

• vehicle

Training on single CPU. |========================================================================================| | Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Base Learning | | | | (hh:mm:ss) | RMSE | Loss | Rate | |========================================================================================| | 1 | 1 | 00:00:01 | 7.13 | 50.8 | 0.0010 | | 2 | 30 | 00:00:15 | 1.32 | 1.8 | 0.0010 | | 4 | 60 | 00:00:32 | 0.93 | 0.9 | 0.0010 | | 5 | 90 | 00:00:46 | 0.64 | 0.4 | 0.0010 | | 7 | 120 | 00:01:00 | 0.58 | 0.3 | 0.0010 | | 9 | 150 | 00:01:14 | 0.64 | 0.4 | 0.0010 | | 10 | 180 | 00:01:27 | 0.46 | 0.2 | 0.0010 | | 12 | 210 | 00:01:42 | 0.40 | 0.2 | 0.0010 | | 14 | 240 | 00:01:58 | 0.58 | 0.3 | 0.0010 | | 15 | 270 | 00:02:17 | 0.40 | 0.2 | 0.0010 | | 17 | 300 | 00:02:35 | 0.37 | 0.1 | 0.0010 | | 19 | 330 | 00:02:48 | 0.50 | 0.2 | 0.0010 | | 20 | 360 | 00:03:03 | 0.37 | 0.1 | 0.0010 | | 22 | 390 | 00:03:17 | 0.36 | 0.1 | 0.0010 | | 24 | 420 | 00:03:30 | 0.43 | 0.2 | 0.0010 | | 25 | 450 | 00:03:44 | 0.54 | 0.3 | 0.0010 | | 27 | 480 | 00:04:00 | 0.54 | 0.3 | 0.0010 | | 29 | 510 | 00:04:18 | 0.66 | 0.4 | 0.0010 | | 30 | 540 | 00:04:34 | 0.38 | 0.1 | 0.0010 | |========================================================================================| Training finished: Max epochs completed. Detector training complete.

Verify the training accuracy by inspecting the training loss for each iteration.

figure plot(info.TrainingLoss) grid on xlabel("Number of Iterations") ylabel("Training Loss for Each Iteration")

Read a test image into the workspace.

img = imread("detectcars.png");

Run the trained YOLO v2 object detector on the test image for vehicle detection.

[bboxes,scores] = detect(trainedDetector,img);

Display the detection results.

if(~isempty(bboxes)) img = insertObjectAnnotation(img,"rectangle",bboxes,scores); end figure imshow(img)

Input Arguments

collapse all

Labeled ground truth images, specified as a datastore or a table.

• If you use a datastore, your data must be set up so that calling the datastore with the read and readall functions returns a cell array or table with two or three columns. When the output contains two columns, the first column must contain bounding boxes, and the second column must contain labels, {boxes,labels}. When the output contains three columns, the second column must contain the bounding boxes, and the third column must contain the labels. In this case, the first column can contain any type of data. For example, the first column can contain images or point cloud data.

  data	boxes	labels
  The first column must be images.	_M_-by-4 matrices of bounding boxes of the form [x, y,width, _height_], where [x,y] represent the top-left coordinates of the bounding box.	The third column must be a cell array that contains_M_-by-1 categorical vectors containing object class names. All categorical data returned by the datastore must contain the same categories.
  For more information, see Datastores for Deep Learning (Deep Learning Toolbox).

• If you use a table, the table must have two or more columns. The first column of the table must contain image file names with paths. The images must be grayscale or truecolor (RGB) and they can be in any format supported by imread. Each of the remaining columns must be a cell vector that contains_M_-by-4 matrices that represent a single object class, such as_vehicle_, flower, or stop sign. The columns contain 4-element double arrays of M bounding boxes in the format [x,y,width,_height_]. The format specifies the upper-left corner location and size of the bounding box in the corresponding image. To create a ground truth table, you can use the Image Labeler app orVideo Labeler app. To create a table of training data from the generated ground truth, use the objectDetectorTrainingData function.
  

Note

When the training data is specified using a table, thetrainYOLOv2ObjectDetector function checks these conditions

• The bounding box values must be integers. Otherwise, the function automatically rounds each noninteger values to its nearest integer.
• The bounding box must not be empty and must be within the image region. While training the network, the function ignores empty bounding boxes and bounding boxes that lie partially or fully outside the image region.

Pretrained or untrained YOLO v2 object detector, specified as a yolov2ObjectDetector object. If detector is a pretrained detector, then you can continue training the detector with additional training data or perform more training iterations to improve detector accuracy.

Training options, specified as a TrainingOptionsSGDM,TrainingOptionsRMSProp, or TrainingOptionsADAM object returned by the trainingOptions (Deep Learning Toolbox) function. To specify the solver name and other options for network training, use the trainingOptions (Deep Learning Toolbox) function.

Note

The trainYOLOv2ObjectDetector function does not support these training options:

• The trainingOptions Shuffle values, "once" and"every-epoch" are not supported when you use a datastore input.
• Datastore inputs are not supported when you set theDispatchInBackground training option totrue.

Saved detector checkpoint, specified as a yolov2ObjectDetector object. To periodically save a detector checkpoint during training, specify CheckpointPath. To control how frequently check points are saved see the CheckPointFrequency andCheckPointFrequencyUnit training options.

To load a checkpoint for a previously trained detector, load the MAT file from the checkpoint path. For example, if the CheckpointPath property of the object specified by options is "/checkpath", you can load a checkpoint MAT file by using this code.

data = load("/checkpath/yolov2_checkpoint__216__2018_11_16__13_34_30.mat"); checkpoint = data.detector;

The name of the MAT file includes the iteration number and timestamp of when the detector checkpoint was saved. The detector is saved in the detector variable of the file. Pass this file back into thetrainYOLOv2ObjectDetector function:

yoloDetector = trainYOLOv2ObjectDetector(trainingData,checkpoint,options);

Name-Value Arguments

collapse all

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: ExperimentManager="none" specifies not to monitor the detector training.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: "ExperimentManager","none"

Detector training experiment monitoring, specified as an experiments.Monitor (Deep Learning Toolbox) object for use with the Experiment Manager (Deep Learning Toolbox) app. You can use this object to track the progress of training, update information fields in the training results table, record values of the metrics used by the training, and to produce training plots. For an example using this app, see Train Object Detectors in Experiment Manager.

Information monitored during training:

• Training loss at each iteration.
• Training accuracy at each iteration.
• Training root mean square error (RMSE) for the box regression layer.
• Learning rate at each iteration.

Validation information when the training options input contains validation data:

• Validation loss at each iteration.
• Validation accuracy at each iteration.
• Validation RMSE at each iteration.

Output Arguments

collapse all

Training progress information, returned as a structure array with seven fields. Each field corresponds to a stage of training.

• TrainingLoss — Training loss at each iteration is the mean squared error (MSE) calculated as the sum of localization error, confidence loss, and classification loss. For more information about the training loss function, see YOLO v2 Training Loss.
• TrainingRMSE — Training root mean squared error (RMSE) is the RMSE calculated from the training loss at each iteration.
• BaseLearnRate — Learning rate at each iteration.
• ValidationLoss — Validation loss at each iteration.
• ValidationRMSE — Validation RMSE at each iteration.
• FinalValidationLoss — Final validation loss at end of the training.
• FinalValidationRMSE — Final validation RMSE at end of the training.

Each field is a numeric vector with one element per training iteration. Values that have not been calculated at a specific iteration are assigned as NaN. The struct contains ValidationLoss,ValidationAccuracy, ValidationRMSE,FinalValidationLoss, and FinalValidationRMSE fields only when options specifies validation data.

More About

collapse all

By default, the trainYOLOv2ObjectDetector function preprocesses the training images by:

• Resizing the input images to match the input size of the network.
• Normalizing the pixel values of the input images to lie in the range [0, 1].

When you specify the training data by using a table, thetrainYOLOv2ObjectDetector function also augments the input dataset by:

• Reflecting the training data horizontally. The probability for horizontally flipping each image in the training data is 0.5.
• Uniformly scaling (resizing) the training data by a scale factor that is randomly picked from a continuous uniform distribution in the range [1, 1.1].
• Random color jittering for brightness, hue, saturation, and contrast.

When you specify the training data by using a datastore, thetrainYOLOv2ObjectDetector function does not perform data augmentation. Instead you can augment the training data in datastore by using the transform function and then, train the network with the augmented training data. For more information on how to apply augmentation while using datastores, see Preprocess Images for Deep Learning (Deep Learning Toolbox).

During training, the trainYOLOv2ObjectDetector function predicts refined bounding box locations by optimizing the mean squared error (MSE) loss between predicted bounding boxes and the ground truth. The loss function is defined as

K1∑i=0S2∑j=0B1ijobj[(xi−x^i)2+(yi−y^i)2] + K1∑i=0S2∑j=0B1ijobj[(wi−w^i)2+(hi−h^i)2] +K2∑i=0S2∑j=0B1ijobj(Ci−C^i)2 +K3∑i=0S2∑j=0B1ijnoobj(Ci−C^i)2 + K4∑i=0S21iobj∑c∈classes(pi(c)−p^i(c))2

where:

• S is the number of grid cells.
• B is the number of bounding boxes in each grid cell.
• 1ijobj is 1 if the jth bounding box in grid cell_i_ is responsible for detecting the object. Otherwise it is set to 0. A grid cell i is responsible for detecting the object, if the overlap between the ground truth and a bounding box in that grid cell is greater than or equal to 0.6.
• 1ijnoobj is 1 if the jth bounding box in grid cell_i_ does not contain any object. Otherwise it is set to 0.
• 1iobj is 1 if an object is detected in grid cell i. Otherwise it is set to 0.
• K1,K2,K3, and_K4_ are the weights. To adjust the weights, modify the [LossFactors](yolov2objectdetector.html#mw%5F5e3535de-ce11-4bd2-8b32-d385ca422fa9) property of the detector.

The loss function can be split into three parts:

• Localization loss
  The first and second terms in the loss function comprise the localization loss. It measures error between the predicted bounding box and the ground truth. The parameters for computing the localization loss include the position, size of the predicted bounding box, and the ground truth. The parameters are defined as follows.
  • (xi,yi), is the center of the jth bounding box relative to grid cell i.
  • (x^i,y^i), is the center of the ground truth relative to grid cell_i_.
  • wi and hi is the width and the height of the jth bounding box in grid cell i, respectively. The size of the predicted bounding box is specified relative to the input image size.
  • w^i and h^i is the width and the height of the ground truth in grid cell_i_, respectively.
  • K1 is the weight for localization loss. Increase this value to increase the weightage for bounding box prediction errors.
• Confidence loss
  The third and fourth terms in the loss function comprise the confidence loss. The third term measures the objectness (confidence score) error when an object is detected in the jth bounding box of grid cell_i_. The fourth term measures the objectness error when no object is detected in the jth bounding box of grid cell_i_. The parameters for computing the confidence loss are defined as follows.
  • Ci is the confidence score of the_jth_ bounding box in grid cell i.
  • Ĉi is the confidence score of the ground truth in grid cell i.
  • K2 is the weight for objectness error, when an object is detected in the predicted bounding box. You can adjust the value of K2 to weigh confidence scores from grid cells that contain objects.
  • K3 is the weight for objectness error, when an object is not detected in the predicted bounding box. You can adjust the value of K3 to weigh confidence scores from grid cells that do not contain objects.
    The confidence loss can cause the training to diverge when the number of grid cells that do not contain objects is more than the number of grid cells that contain objects. To remedy this, increase the value for_K2_ and decrease the value for_K3_.
• Classification loss
  The fifth term in the loss function comprises the classification loss. For example, suppose that an object is detected in the predicted bounding box contained in grid cell i. Then, the classification loss measures the squared error between the class conditional probabilities for each class in grid cell_i_. The parameters for computing the classification loss are defined as follows.
  • pi (c) is the estimated conditional class probability for object class c in grid cell i.
  • p^i(c) is the actual conditional class probability for object class_c_ in grid cell i.
  • K4 is the weight for classification error when an object is detected in the grid cell. Increase this value to increase the weightage for classification loss.

Tips

• To generate the ground truth, use the Image Labeler orVideo Labeler app. To create a table of training data from the generated ground truth, use the objectDetectorTrainingData function.
• To improve prediction accuracy:
  • Increase the number of images you can use to train the network. You can expand the training dataset through data augmentation. For information on how to apply data augmentation for preprocessing, see Preprocess Images for Deep Learning (Deep Learning Toolbox).
  • Perform multiscale training by specifying an input detector whose [TrainingImageSize](yolov2objectdetector.html#mw%5Ff9479232-e734-44a4-8169-fb3ff0e4e510) property is a matrix with two or more rows. For each training epoch, the trainYOLOv2ObjectDetector function randomly resizes the input training images to one of the specified training image sizes.
  • Choose anchor boxes appropriate to the dataset for training the network. You can use the estimateAnchorBoxes function to compute anchor boxes directly from the training data.

References

[1] Joseph. R, S. K. Divvala, R. B. Girshick, and F. Ali. "You Only Look Once: Unified, Real-Time Object Detection." In_Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pp. 779–788. Las Vegas, NV: CVPR, 2016.

[2] Joseph. R and F. Ali. "YOLO 9000: Better, Faster, Stronger." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525. Honolulu, HI: CVPR, 2017.

Version History

Introduced in R2019a

expand all

Support for using MATLAB® Compiler™ will be removed in a future release.

The trainYOLOv2ObjectDetector supports a new process to train ayolov2ObjectDetector object. The function now uses:

• Class names specified by the [ClassNames](yolov2objectdetector.html#mw%5F1377a106-d967-48d8-b2c8-f272162ef3de) property of the yolov2ObjectDetector object. The function no longer uses the class names from the trainingData table.
• Anchor boxes specified by the [AnchorBoxes](yolov2objectdetector.html#mw%5F71b01b40-1509-4f6c-aa09-9fa1b37d0171) property of the yolov2ObjectDetector object.
• MSE loss weights specified by the [LossFactors](yolov2objectdetector.html#mw%5F5e3535de-ce11-4bd2-8b32-d385ca422fa9) property of the yolov2ObjectDetector object.

The TrainingImageSize name-value argument is no longer recommended. Instead, specify training image sizes for multiscale training by using the [TrainingImageSize](yolov2objectdetector.html#mw%5Ff9479232-e734-44a4-8169-fb3ff0e4e510) name-value argument of the yolov2ObjectDetector object.

Starting in R2024b, DAGNetwork (Deep Learning Toolbox) objects are not recommended. Instead, specify the network architecture using a yolov2ObjectDetector object.

There are no plans to remove support for DAGNetwork objects.

See Also

Apps

• Image Labeler | Video Labeler

Functions

• trainingOptions (Deep Learning Toolbox) | objectDetectorTrainingData | trainYOLOv4ObjectDetector

Objects

• yolov2ObjectDetector | yolov4ObjectDetector | boxLabelDatastore

Topics

• Create Custom YOLO v2 Object Detection Network
• Object Detection Using YOLO v2 Deep Learning
• Estimate Anchor Boxes from Training Data
• Code Generation for Object Detection by Using YOLO v2
• Train Object Detectors in Experiment Manager
• Getting Started with YOLO v2
• Get Started with Object Detection Using Deep Learning
• Choose an Object Detector
• Anchor Boxes for Object Detection
• Datastores for Deep Learning (Deep Learning Toolbox)