summarize - Summarize object detection performance metrics at data set and class

        level - MATLAB ([original](https://in.mathworks.com/help/vision/ref/objectdetectionmetrics.summarize.html)) ([raw](?raw))

Summarize object detection performance metrics at data set and class level

Since R2024b

Syntax

Description

[[summaryDataset](#mw%5F0fc8a7fe-7307-4820-b5da-9b0f3fe59231),[summaryClass](#mw%5F55d5489e-aa0b-40a8-8d65-40d4a9a4e9aa)] = summarize([metrics](#mw%5Fa4d1f5db-f12c-4455-b847-50df32a5176e)) summarizes the object detection evaluation metrics across all classes of the data set summaryDataset, and for each class in the data setsummaryClass.

example

[[summaryDataset](#mw%5F0fc8a7fe-7307-4820-b5da-9b0f3fe59231),[summaryClass](#mw%5F55d5489e-aa0b-40a8-8d65-40d4a9a4e9aa)] = summarize([metrics](#mw%5Fa4d1f5db-f12c-4455-b847-50df32a5176e),MetricName=[metricName](#mw%5Fdbd7af3b-e3ab-4894-9717-d07ad1630cb9)) specifies the metrics category to return in the metrics summary.

Examples

collapse all

Load a table containing image file names and ground truth bounding box labels into the workspace. The first column contains the image file names, and the remaining columns contain the labeled bounding boxes.

data = load("vehicleTrainingData.mat"); trainingData = data.vehicleTrainingData;

Set the value of the dataDir variable as the location where the vehicleTrainingData.mat file is located. Load the test data into a local vehicle data folder.

dataDir = fullfile(toolboxdir("vision"),"visiondata"); trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);

Create an imageDatastore using the files from the table.

imds = imageDatastore(trainingData.imageFilename);

Create a boxLabelDatastore using the label column from the table.

blds = boxLabelDatastore(trainingData(:,2:end));

Load a pretrained YOLO v2 object detector, trained to detect vehicles, into the workspace.

vehicleDetector = load("yolov2VehicleDetector.mat"); detector = vehicleDetector.detector;

Run the detector on the test images. Set the detection threshold to a low value to detect as many objects as possible. This enables you to evaluate the detector precision across the full range of recall values.

results = detect(detector,imds,Threshold=0.01);

Compute metrics for evaluating the performance of the object detector.

metrics = evaluateObjectDetection(results,blds);

Summarize the object detection metrics for the vehicle class using the summarize object function.

[summaryDataset,summaryClass] = summarize(metrics); summaryClass

summaryClass=1×3 table NumObjects APOverlapAvg AP0.5 __________ ____________ _______

vehicle       336          0.99096       0.99096

Input Arguments

collapse all

Metrics category to return in the metrics summary, specified as one of these options.

• "all" – Returns the average precision and mean average precision metric values, aggregated over each overlap threshold, along with all additional metrics.
• "AP" – Returns only the average precision across all classes at each overlap threshold as a_numThresh_-by-1 numeric vector, and the mean average precision (mAP) computed across all overlap thresholds.numThresh is the number of overlap thresholds specified by the [OverlapThreshold](objectdetectionmetrics.html#mw%5Feca15534-c978-4775-b162-db632406f5f1) property of theobjectDetectionMetrics object.
• "LAMR" – Returns only the log-average miss rate (LAMR) computed at each overlap threshold as a_numThresh_-by-1 numeric vector, and the mean of the log-average miss rate (mLAMR) computed across all overlap thresholds.
• "AOS" – Returns only the average orientation similarity (AOS) computed at each overlap threshold as a_numThresh_-by-1 array, and the mean of the average orientation similarity (mAOS) computed across all overlap thresholds.
  To specify "AOS" as the metric to return, your input data must contain rotated bounding boxes.

Output Arguments

collapse all

Metric summary over the data set, returned as a 1-by-numMetrics table, where_numMetrics_ is the number of specified metrics.

The columns of the table depend on the value of themetricName argument.

• "all" —

  NumObjects	mAPOverlapAvg	mAP	mLAMROverlapAvg	mLAMR	mAOSOverlapAvg	mAOS
  Number of objects in the ground truth data, returned as a positive integer.	Mean average precision (mAP) averaged over all thresholds specified by theOverlapThreshold property ofmetrics, returned as a numeric scalar.	Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.numThresh is the number of overlap thresholds.	Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a numeric scalar.	Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.	Mean average orientation similarity (mAOS) averaged over all classes, and averaged over all overlap thresholds specified by theOverlapThreshold property ofmetrics, returned as a numeric scalar.	Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by theOverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.
  To return the "mAOSOverlapAvg" and"mAOS" columns, your input data must contain rotated bounding boxes.

• "AP" —

  NumObjects	mAPOverlapAvg	mAP
  Number of objects in the ground truth data, returned as a positive integer.	Mean average precision (mAP) averaged over all thresholds specified by theOverlapThreshold property ofmetrics, returned as a numeric scalar.	Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.numThresh is the number of overlap thresholds.

• "LAMR" —

  mLAMROverlapAvg	mLAMR
  Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a positive scalar.	Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.

• "AOS" —

  mAOSOverlapAvg	mAOS
  Mean average orientation similarity (mAOS) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a numeric scalar.	Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by theOverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.

Metrics summary per class, returned as a_numClasses_-by-numMetrics table, where numClasses is the number of classes, and_numMetrics_ is the number of specified metrics.

The columns of the table depend on the value of themetricName argument.

• "all" —

  NumObjects	APOverlapAvg	AP	LAMROverlapAvg	LAMR	AOSOverlapAvg	AOS
  Number of objects in the ground truth data for each class, returned as a positive integer.	Average precision (AP) for each class averaged over all thresholds specified by theOverlapThreshold property ofmetrics, returned as a numeric scalar.	Average precision (AP) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.numThresh is the number of overlap thresholds.	Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property ofmetrics.	Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.	Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a numeric scalar.	Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.
  To return the "AOSOverlapAvg" and"AOS" columns, your input data must contain rotated bounding boxes.

• "AP" —

  NumObjects	APOverlapAvg	AP
  Number of objects in the ground truth data for each class, returned as a positive integer.	Average precision (AP) for each class averaged over all thresholds specified by theOverlapThreshold property ofmetrics, returned as a numeric scalar.	Average precision (AP) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.numThresh is the number of overlap thresholds.

• "LAMR" —

  LAMROverlapAvg	LAMR
  Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property ofmetrics.	Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.

• "AOS" —

  AOSOverlapAvg	AOS
  Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a numeric scalar.	Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the OverlapThreshold property ofmetrics, returned as a_numThresh_-by-1 numeric vector.

Version History

Introduced in R2024b