evaluateAnomalyDetection - Evaluate anomaly detection results against ground truth - MATLAB (original) (raw)

Evaluate anomaly detection results against ground truth

Since R2022b

Syntax

Description

[metrics](#mw%5Fb8e7a1f0-e47f-40cc-8b94-04d5c5795751) = evaluateAnomalyDetection([predLabels](#predlabels),[gtLabels](#mw%5F5f0f95db-ed38-4731-a5d5-5930890ed27f%5Fsep%5Fgtlabels),[anomalyLabels](#mw%5F5f0f95db-ed38-4731-a5d5-5930890ed27f%5Fsep%5Fanomalylabels)) calculates various metrics to evaluate the quality of the anomaly detection resultspredLabels against the ground truth gtLabels.anomalyLabels indicates which class labels ingtLabels belong to the anomaly (positive) class

Note

This functionality requires the Automated Visual Inspection Library for Computer Vision Toolbox™. You can install the Automated Visual Inspection Library for Computer Vision Toolbox from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

example

[metrics](#mw%5Fb8e7a1f0-e47f-40cc-8b94-04d5c5795751) = evaluateAnomalyDetection([predLabels](#predlabels),[gtLabels](#mw%5F5f0f95db-ed38-4731-a5d5-5930890ed27f%5Fsep%5Fgtlabels),[anomalyLabels](#mw%5F5f0f95db-ed38-4731-a5d5-5930890ed27f%5Fsep%5Fanomalylabels),Verbose=[tf](#mw%5Ffbbd52d4-efb8-471f-b5dd-c1599e7c4b6c)) also displays the evaluation progress information in the command window.

Examples

collapse all

Load test images and corresponding labels, then create a datastore that reads the test data. The data set consists of grayscale images of handwritten digits 0–9.

[Xtest,gtLabels] = digitTest4DArrayData; dsTest = arrayDatastore(Xtest,IterationDimension=4);

Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, specify the set of anomaly labels as the set of digits between 0 and 9, excluding 8.

load("digit8AnomalyDetector.mat"); anomalyLabels = setdiff(string(0:9),"8");

Classify each test image as normal or anomalous.

predLabels = classify(detector,dsTest);

Calculate metrics for the anomaly detection results.

metrics = evaluateAnomalyDetection(predLabels,gtLabels,anomalyLabels)

Evaluating anomaly detection results

metrics = anomalyDetectionMetrics with properties:

          ConfusionMatrix: [2×2 table]
NormalizedConfusionMatrix: [2×2 table]
           DataSetMetrics: [1×8 table]
             ClassMetrics: [2×2 table]

Display the confusion matrix.

cm = metrics.ConfusionMatrix

cm=2×2 table Normal Anomaly ______ _______

Normal       465        35  
Anomaly     1134      3366

Input Arguments

collapse all

Predicted labels, specified as a logical vector. The label value 0 (false) corresponds to the normal (negative) class. The value1 (true) corresponds to the anomaly (positive) class. You can get the predicted labels using the classify function.

Data Types: logical

Ground truth labels for each image, specified as a numeric vector, logical vector, or categorical vector. The evaluateAnomalyDetection function converts the labels into a logical vector according to the set of anomaly labels inanomalyLabels.

Anomaly labels, specified as a vector of the same data type asgtLabels. When gtLabels is categorical,anomalyLabels can be of data type string whose values correspond to categories in gtLabels.

The evaluateAnomalyDetection function converts all ground truth labels ingtLabels that belong to the set of anomaly labels to a logicaltrue, indicating an anomaly (positive detection). The function converts all other ground truth labels to a logical false, indicating normality (negative detection).

Display evaluation progress information in the command window, specified as a numeric or logical 1 (true) or0 (false). The displayed information includes a progress bar, elapsed time, estimated time remaining, and data set metrics.

Data Types: logical

Output Arguments

Version History

Introduced in R2022b