brier_score_loss (original) (raw)
sklearn.metrics.brier_score_loss(y_true, y_proba, *, sample_weight=None, pos_label=None, labels=None, scale_by_half='auto')[source]#
Compute the Brier score loss.
The smaller the Brier score loss, the better, hence the naming with “loss”. The Brier score measures the mean squared difference between the predicted probability and the actual outcome. The Brier score is a strictly proper scoring rule.
Read more in the User Guide.
Parameters:
y_truearray-like of shape (n_samples,)
True targets.
y_probaarray-like of shape (n_samples,) or (n_samples, n_classes)
Predicted probabilities. If y_proba.shape = (n_samples,)the probabilities provided are assumed to be that of the positive class. If y_proba.shape = (n_samples, n_classes)the columns in y_proba are assumed to correspond to the labels in alphabetical order, as done byLabelBinarizer.
sample_weightarray-like of shape (n_samples,), default=None
Sample weights.
pos_labelint, float, bool or str, default=None
Label of the positive class when y_proba.shape = (n_samples,). If not provided, pos_label will be inferred in the following manner:
- if
y_truein {-1, 1} or {0, 1},pos_labeldefaults to 1; - else if
y_truecontains string, an error will be raised andpos_labelshould be explicitly specified; - otherwise,
pos_labeldefaults to the greater label, i.e.np.unique(y_true)[-1].
labelsarray-like of shape (n_classes,), default=None
Class labels when y_proba.shape = (n_samples, n_classes). If not provided, labels will be inferred from y_true.
Added in version 1.7.
scale_by_halfbool or “auto”, default=”auto”
When True, scale the Brier score by 1/2 to lie in the [0, 1] range instead of the [0, 2] range. The default “auto” option implements the rescaling to [0, 1] only for binary classification (as customary) but keeps the original [0, 2] range for multiclasss classification.
Added in version 1.7.
Returns:
scorefloat
Brier score loss.
Notes
For \(N\) observations labeled from \(C\) possible classes, the Brier score is defined as:
\[\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C}(y_{ic} - \hat{p}_{ic})^{2}\]
where \(y_{ic}\) is 1 if observation i belongs to class c, otherwise 0 and \(\hat{p}_{ic}\) is the predicted probability for observation i to belong to class c. The Brier score then ranges between \([0, 2]\).
In binary classification tasks the Brier score is usually divided by two and then ranges between \([0, 1]\). It can be alternatively written as:
\[\frac{1}{N}\sum_{i=1}^{N}(y_{i} - \hat{p}_{i})^{2}\]
where \(y_{i}\) is the binary target and \(\hat{p}_{i}\)is the predicted probability of the positive class.
References
Examples
import numpy as np from sklearn.metrics import brier_score_loss y_true = np.array([0, 1, 1, 0]) y_true_categorical = np.array(["spam", "ham", "ham", "spam"]) y_prob = np.array([0.1, 0.9, 0.8, 0.3]) brier_score_loss(y_true, y_prob) 0.0375 brier_score_loss(y_true, 1-y_prob, pos_label=0) 0.0375 brier_score_loss(y_true_categorical, y_prob, pos_label="ham") 0.0375 brier_score_loss(y_true, np.array(y_prob) > 0.5) 0.0 brier_score_loss(y_true, y_prob, scale_by_half=False) 0.075 brier_score_loss( ... ["eggs", "ham", "spam"], ... [[0.8, 0.1, 0.1], [0.2, 0.7, 0.1], [0.2, 0.2, 0.6]], ... labels=["eggs", "ham", "spam"] ... ) 0.146