anomalyDetectionMetrics - Anomaly detection metrics - MATLAB (original) (raw)

Main Content

Anomaly detection metrics

Since R2022b

Description

An anomalyDetectionMetrics object encapsulates anomaly detection quality metrics for a set of images. Images of anomalies belong to the positive (true) class. Normal images belong to the negative (false) class.

Note

This functionality requires the Automated Visual Inspection Library for Computer Vision Toolbox™. You can install the Automated Visual Inspection Library for Computer Vision Toolbox from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

Properties

expand all

This property is read-only.

Confusion matrix, specified as a 2-by-2 table. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j.

This property is read-only.

Normalized confusion matrix, specified as a 2-by-2 table.NormalizedConfusionMatrix represents a confusion matrix normalized by the number of images known to belong to each class. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j, divided by the total number of images predicted in class i. Elements are in the range [0, 1].

This property is read-only.

Metrics aggregated over the data set, specified as a 1-by-8 table with columns corresponding to these metrics:

• GlobalAccuracy — Ratio of correctly classified images to total images, regardless of class
• MeanAccuracy — Ratio of correctly classified images of each class to total images, averaged over all classes
• Precision — Positive predictive value
• Recall — True positive (TP) rate
• Specificity — True negative (TN) rate
• F1Score — Harmonic mean of precision and recall
• FalsePositiveRate — False positive (FP) rate, or fallout
• FalseNegativeRate — False negative (FN) rate, or miss

This property is read-only.

Anomaly detection metrics for each class, specified as a 2-by-2 table. The table has rows "Normal" and "Anomaly" and columns corresponding to these metrics:

• Accuracy — Fraction of correctly classified images for each class (normal or anomaly).
• AccuracyPerSubClass — Fraction of correctly classified images for each subclass within the normal and anomaly classes. The value ofAccuracyPerSubClass is a table with C rows and one column, where C is the number of subclasses.

Examples

collapse all

Load test images and corresponding labels, then create a datastore that reads the test data. The data set consists of grayscale images of handwritten digits 0–9.

[Xtest,gtLabels] = digitTest4DArrayData; dsTest = arrayDatastore(Xtest,IterationDimension=4);

Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, specify the set of anomaly labels as the set of digits between 0 and 9, excluding 8.

load("digit8AnomalyDetector.mat"); anomalyLabels = setdiff(string(0:9),"8");

Classify each test image as normal or anomalous.

predLabels = classify(detector,dsTest);

Calculate metrics for the anomaly detection results.

metrics = evaluateAnomalyDetection(predLabels,gtLabels,anomalyLabels)

Evaluating anomaly detection results

• Finalizing... Done.

• Data set metrics:

  GlobalAccuracy MeanAccuracy Precision Recall Specificity F1Score FalsePositiveRate FalseNegativeRate

  ---

    0.7662           0.839         0.98971     0.748        0.93        0.85204          0.07                 0.252

metrics = anomalyDetectionMetrics with properties:

          ConfusionMatrix: [2×2 table]
NormalizedConfusionMatrix: [2×2 table]
           DataSetMetrics: [1×8 table]
             ClassMetrics: [2×2 table]

Display the confusion matrix.

cm = metrics.ConfusionMatrix

cm=2×2 table Normal Anomaly ______ _______

Normal       465        35  
Anomaly     1134      3366  

Version History

Introduced in R2022b