deepbox/metrics

Clustering, Ranking & Pairwise

Cluster quality scores, ranking and multilabel metrics, losses and deviances, and pairwise distance matrices.

Clustering
Ranking

Overview

Internal cluster metrics (silhouetteScore, daviesBouldinScore, calinskiHarabaszScore) take the feature matrix and the labels. silhouetteScore is O(n^2), and it requires { sampleSize } as the fourth argument when n is above 2000.
External metrics (adjustedRandScore, normalizedMutualInfoScore, homogeneityScore, vMeasureScore) compare two label vectors.
pairwiseEuclidean, pairwiseManhattan and pairwiseCosine take X and an optional Y, and return a distance matrix. Without Y they compare X with itself.
Ranking functions read score matrices. ndcgScore(yTrue, yScore, k?) takes relevance scores, and topKAccuracyScore(yTrue, yScore, k) takes class indices and a score matrix, with k defaulting to min(5, nClasses). The page also covers hingeLoss, brierScoreLoss, meanPinballLoss, smape, d2TweedieScore and the Poisson and gamma deviances.

Types

AverageMethod

type

How the two label entropies are combined into the normalizer of NMI and AMI.

type AverageMethod = "min" | "geometric" | "arithmetic" | "max";

SilhouetteMetric

type

Distance input accepted by silhouetteScore and silhouetteSamples.

type SilhouetteMetric = "euclidean" | "precomputed";

SilhouetteScoreOptions

type

Sampling options of silhouetteScore.

sampleSize?: number
Number of samples drawn without replacement; required when n_samples > 2000.
randomState?: number
Seed of the sampler.

DetCurveResult

type

False positive and false negative rates of the DET curve at each threshold.

fpr: number[]
fnr: number[]
thresholds: number[]

MultilabelConfusionMatrixEntry

type

One-vs-rest confusion counts of a single class label.

label: number
tn: number
fp: number
fn: number
tp: number

Functions

adjustedMutualInfoScore

function adjustedMutualInfoScore(labelsTrue: Tensor, labelsPred: Tensor, averageMethod?: AverageMethod): number;

Computes the Adjusted Mutual Information (AMI) between two clusterings.

Details

AMI adjusts the Mutual Information score to account for chance, providing a normalized measure of agreement between two clusterings. Identical partitions score 1, independent labelings score about 0, and the score can be slightly negative for labelings that agree less than chance.

Time Complexity: O(n + k₁ * k₂ * min(k₁, k₂)) Space Complexity: O(k₁ * k₂ + n)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels
averageMethodoptional AverageMethod: Method to compute the normalizer: 'min', 'geometric', 'arithmetic' (default), or 'max'

Returns

numberAdjusted Mutual Information score, at most 1

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
InvalidParameterErrorIf averageMethod is not one of the supported values
DataValidationErrorIf labels contain non-finite or non-integer values

adjustedRandScore

function adjustedRandScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the Adjusted Rand Index (ARI).

The ARI measures the similarity between two clusterings, adjusted for chance. It ranges from -0.5 to 1, where 1 indicates perfect agreement, 0 indicates random labeling, and negative values indicate worse than random.

Time Complexity: O(n + k₁ * k₂) where k₁, k₂ are cluster counts Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels

Returns

numberAdjusted Rand Index (1 for identical partitions and for fewer than two samples)

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

calinskiHarabaszScore

function calinskiHarabaszScore(X: Tensor, labels: Tensor): number;

Computes the Calinski-Harabasz index (Variance Ratio Criterion).

The score is the ratio of between-cluster dispersion to within-cluster dispersion. Higher values indicate better-defined clusters. Returns 0 when k < 2, n_samples == 0, or within-group sum of squares is 0.

Time Complexity: O(n * d) where n is samples and d is features Space Complexity: O(n * d)

Parameters

X Tensor: Feature matrix of shape [n_samples, n_features]
labels Tensor: Cluster labels for each sample

Returns

numberCalinski-Harabasz index (higher is better)

Throws

ShapeErrorIf labels length doesn't match samples, or X shape is invalid
DTypeErrorIf labels are string or X is non-numeric
DataValidationErrorIf X contains non-finite values

completenessScore

function completenessScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the completeness score of a clustering.

A clustering result satisfies completeness if all data points that are members of a given class are assigned to the same cluster. Score ranges from 0 to 1, where 1 indicates perfectly complete clustering.

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth class labels
labelsPred Tensor: Predicted cluster labels

Returns

numberCompleteness score in range [0, 1]

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

daviesBouldinScore

function daviesBouldinScore(X: Tensor, labels: Tensor): number;

Computes the Davies-Bouldin index.

Details

The Davies-Bouldin index measures the average similarity ratio of each cluster with its most similar cluster. Lower values indicate better clustering. Returns 0 when k < 2 or n_samples == 0. Returns Infinity when two clusters with non-zero scatter share the same centroid.

Time Complexity: O(n * d + k² * d) where n is samples, d is features, k is clusters Space Complexity: O(n * d)

Parameters

X Tensor: Feature matrix of shape [n_samples, n_features]
labels Tensor: Cluster labels for each sample

Returns

numberDavies-Bouldin index (lower is better, 0 is minimum)

Throws

ShapeErrorIf labels length doesn't match samples, or X shape is invalid
DTypeErrorIf labels are string or X is non-numeric
DataValidationErrorIf X contains non-finite values

fowlkesMallowsScore

function fowlkesMallowsScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the Fowlkes-Mallows Index (FMI).

The FMI is the geometric mean of pairwise precision and recall. It ranges from 0 to 1, where 1 indicates perfect agreement.

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels

Returns

numberFowlkes-Mallows score in range [0, 1]

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

homogeneityScore

function homogeneityScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the homogeneity score of a clustering.

A clustering result satisfies homogeneity if all of its clusters contain only data points which are members of a single class. Score ranges from 0 to 1, where 1 indicates perfectly homogeneous clustering.

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth class labels
labelsPred Tensor: Predicted cluster labels

Returns

numberHomogeneity score in range [0, 1]

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

mutualInfoScore

function mutualInfoScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the Mutual Information between two clusterings, in nats.

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels

Returns

numberMutual information (non-negative; 0 for independent labelings)

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

ExampleTypeScript

import { mutualInfoScore } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; mutualInfoScore(tensor([0, 0, 1, 1]), tensor([0, 0, 1, 1])); // ln 2 = 0.6931471805599453

normalizedMutualInfoScore

function normalizedMutualInfoScore(labelsTrue: Tensor, labelsPred: Tensor, averageMethod?: AverageMethod): number;

Computes the Normalized Mutual Information (NMI) between two clusterings.

NMI normalizes the Mutual Information score to scale between 0 and 1, where 1 indicates perfect correlation between clusterings. It is not adjusted for chance; see adjustedMutualInfoScore.

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels
averageMethodoptional AverageMethod: Method to compute the normalizer: 'min', 'geometric', 'arithmetic' (default), or 'max'

Returns

numberNormalized Mutual Information score in range [0, 1]

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
InvalidParameterErrorIf averageMethod is not one of the supported values
DataValidationErrorIf labels contain non-finite or non-integer values

randScore

function randScore(labelsTrue: Tensor, labelsPred: Tensor): number;

Computes the Rand Index between two clusterings.

The Rand Index is the fraction of sample pairs on which the two clusterings agree (both put the pair in the same cluster, or both separate it). It is not adjusted for chance; see adjustedRandScore for the adjusted version.

Time Complexity: O(n + k₁ * k₂) where k₁, k₂ are cluster counts Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth cluster labels
labelsPred Tensor: Predicted cluster labels

Returns

numberRand Index in range [0, 1]; 1 for fewer than two samples

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
DataValidationErrorIf labels contain non-finite or non-integer values

ExampleTypeScript

import { randScore } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; randScore(tensor([0, 0, 1, 1]), tensor([0, 0, 1, 2])); // 0.8333333333333334

silhouetteSamples

function silhouetteSamples(X: Tensor, labels: Tensor, metric?: SilhouetteMetric): Tensor;

Computes the Silhouette Coefficient for each sample.

Returns a tensor of per-sample silhouette values. Useful for identifying samples that are well-clustered vs. poorly-clustered. Samples in singleton clusters score 0.

Time Complexity: O(n²) where n is the number of samples Space Complexity: O(n + k) where k is the number of clusters

Parameters

X Tensor: Feature matrix of shape [n_samples, n_features], or a precomputed [n_samples, n_samples] distance matrix when metric is 'precomputed'
labels Tensor: Cluster labels for each sample
metricoptional SilhouetteMetric: Distance metric: 'euclidean' (default) or 'precomputed'

Returns

TensorFloat64 tensor of shape [n_samples] with coefficients in range [-1, 1]

Throws

InvalidParameterErrorIf fewer than 2 samples, a cluster count outside [2, n_samples - 1], or unsupported metric
ShapeErrorIf labels length doesn't match samples, or X shape is invalid
DTypeErrorIf labels are string or X is non-numeric
DataValidationErrorIf X contains non-finite values, or a precomputed matrix has negative entries or a non-zero diagonal

silhouetteScore

function silhouetteScore(X: Tensor, labels: Tensor, metric?: SilhouetteMetric, options?: SilhouetteScoreOptions): number;

Computes the mean Silhouette Coefficient over all samples.

Details

The Silhouette Coefficient for a sample measures how similar it is to its own cluster compared to other clusters. Values range from -1 to 1, where higher values indicate better-defined clusters. Samples in singleton clusters score 0.

Formula: s(i) = (b(i) - a(i)) / max(a(i), b(i)) - a(i): mean intra-cluster distance for sample i - b(i): mean nearest-cluster distance for sample i

Time Complexity: O(n²) where n is the number of samples Space Complexity: O(n + k) where k is the number of clusters

Parameters

X Tensor: Feature matrix of shape [n_samples, n_features], or a precomputed [n_samples, n_samples] distance matrix when metric is 'precomputed'
labels Tensor: Cluster labels for each sample
metricoptional SilhouetteMetric: Distance metric: 'euclidean' (default) or 'precomputed'
optionsoptional SilhouetteScoreOptions: Optional parameters

Returns

numberMean silhouette coefficient in range [-1, 1]

Throws

InvalidParameterErrorIf fewer than 2 samples, invalid sampleSize or randomState, a cluster count outside [2, n_samples - 1], or unsupported metric
ShapeErrorIf labels length doesn't match samples, or X shape is invalid
DTypeErrorIf labels are string or X is non-numeric
DataValidationErrorIf X contains non-finite values, or a precomputed matrix has negative entries or a non-zero diagonal

vMeasureScore

function vMeasureScore(labelsTrue: Tensor, labelsPred: Tensor, beta?: number): number;

Computes the V-measure score of a clustering.

V-measure is the weighted harmonic mean of homogeneity and completeness. With beta > 1 completeness is weighted more; with beta < 1 homogeneity is weighted more.

Formula: v = (1 + beta) * h * c / (beta * h + c)

Time Complexity: O(n + k₁ * k₂) Space Complexity: O(k₁ * k₂)

Parameters

labelsTrue Tensor: Ground truth class labels
labelsPred Tensor: Predicted cluster labels
betaoptional number: Weight of completeness relative to homogeneity (default: 1.0)

Returns

numberV-measure score in range [0, 1]

Throws

ShapeErrorIf labelsTrue and labelsPred have different sizes
DTypeErrorIf labels are string
InvalidParameterErrorIf beta is not a positive finite number
DataValidationErrorIf labels contain non-finite or non-integer values

brierScoreLoss

function brierScoreLoss(yTrue: Tensor, yProb: Tensor, posLabel?: number): number;

Brier score loss for probability predictions.

Brier = (1/n) * Σ (1[y_true = posLabel] - y_prob)² Lower is better. yTrue holds at most two distinct labels; yProb holds the predicted probability of the positive label.

Parameters

yTrue Tensor: True binary labels of shape (n_samples,)
yProb Tensor: Predicted probabilities of shape (n_samples,), each in [0, 1]
posLabeloptional number: Label treated as positive. Defaults to 1 for {0, 1} and {-1, 1} labels (and a lone 0, 1 or -1), otherwise to the largest label.

Returns

numberBrier score in [0, 1]

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf yTrue has more than two distinct labels or a probability lies outside [0, 1]
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { brierScoreLoss } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; brierScoreLoss(tensor([0, 1, 1, 0]), tensor([0.1, 0.9, 0.8, 0.2])); // 0.025

coverageError

function coverageError(yTrue: Tensor, yScore: Tensor): number;

Coverage error for multi-label ranking.

Details

Compute how far we need to go through the ranked scores to cover all true labels. The best value is equal to the average number of labels per sample.

coverage(y, f) = (1/n) * Σ_i max_{j: y_ij=1} rank_ij

where rank_ij is the number of labels scored at least as high as label j in sample i (highest score = rank 1). Labels that tie with a true label count against the ranking, as in scikit-learn. Samples without a true label contribute 0.

Parameters

yTrue Tensor: Binary label indicators of shape (n_samples, n_labels)
yScore Tensor: Target scores of shape (n_samples, n_labels)

Returns

numberCoverage error (lower is better, minimum = avg number of true labels)

Throws

ShapeErrorIf an input is not 2D or the shapes differ
InvalidParameterErrorIf yTrue holds a value other than 0 or 1
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { coverageError } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; coverageError(tensor([[1, 0, 0], [0, 1, 1]]), tensor([[0.5, 0.5, 0.5], [0.2, 0.2, 0.2]])); // 3

d2TweedieScore

function d2TweedieScore(yTrue: Tensor, yPred: Tensor, power?: number): number;

D² score for Tweedie deviance regression.

Details

D² = 1 - deviance(y_true, y_pred) / deviance(y_true, y_mean)

Generalizes R² to Tweedie distributions. The power parameter selects the distribution family: - power=0: Normal (equivalent to R²) - power=1: Poisson - power=2: Gamma - power=3: Inverse Gaussian

Domain: y_pred must be positive for every power other than 0. For power in [1, 2) y_true must be non-negative, and for power >= 2 it must be positive.

Parameters

yTrue Tensor: True values of shape (n_samples,)
yPred Tensor: Predicted values of shape (n_samples,)
poweroptional number: Tweedie power parameter (default: 0)

Returns

numberD² score (1 is perfect, 0 is baseline)

Throws

ShapeErrorIf inputs have different lengths or are not 1D
InvalidParameterErrorIf power is in (0, 1), the inputs are empty, or values lie outside the domain of the chosen power
DataValidationErrorIf an input contains NaN or infinite values

detCurve

function detCurve(yTrue: Tensor, yScore: Tensor): DetCurveResult;

Detection Error Tradeoff (DET) curve.

Returns false positive rate (FPR) and false negative rate (FNR) at each distinct score threshold, with thresholds in decreasing order. A sample is predicted positive when its score is greater than or equal to the threshold. Unlike ROC curves, both axes represent error rates.

Returns empty arrays when yTrue is empty or contains only one class.

Parameters

yTrue Tensor: True binary labels (0 or 1) of shape (n_samples,)
yScore Tensor: Target scores of shape (n_samples,)

Returns

DetCurveResult{ fpr, fnr, thresholds } arrays of equal length

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf yTrue holds a label other than 0 or 1
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { detCurve } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; const { fpr, fnr, thresholds } = detCurve(tensor([0, 0, 1, 1]), tensor([0.1, 0.4, 0.35, 0.8]));// thresholds: [0.8, 0.4, 0.35, 0.1]

hingeLoss

function hingeLoss(yTrue: Tensor, yDecision: Tensor): number;

Average hinge loss (for SVM evaluation).

hinge_loss = (1/n) * Σ max(0, 1 - y_true * y_decision)

yTrue holds the two classes; the larger label is the positive class (+1) and the smaller one the negative class (-1), so {-1, +1} and {0, 1} labels both work.

Parameters

yTrue Tensor: True binary labels of shape (n_samples,)
yDecision Tensor: Decision function values of shape (n_samples,)

Returns

numberMean hinge loss

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf yTrue has more than two distinct labels, or a single label other than -1 or +1
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { hingeLoss } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; hingeLoss(tensor([0, 1, 1, 0]), tensor([-1, 2, 0.2, 0.3])); // 0.525

labelRankingLoss

function labelRankingLoss(yTrue: Tensor, yScore: Tensor): number;

Label ranking loss for multi-label classification.

Computes the average fraction of label pairs that are incorrectly ordered. A pair (i, j) is incorrectly ordered if y_true[i]=1, y_true[j]=0 but score[i] <= score[j]; tied scores therefore count as errors. Samples with no true label or with every label true contribute 0.

Best value is 0.

Parameters

yTrue Tensor: Binary label indicators of shape (n_samples, n_labels)
yScore Tensor: Target scores of shape (n_samples, n_labels)

Returns

numberLabel ranking loss in [0, 1]

Throws

ShapeErrorIf an input is not 2D or the shapes differ
InvalidParameterErrorIf yTrue holds a value other than 0 or 1
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { labelRankingLoss } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; labelRankingLoss(tensor([[1, 1, 0]]), tensor([[0.9, 0.1, 0.5]])); // 0.5

meanGammaDeviance

function meanGammaDeviance(yTrue: Tensor, yPred: Tensor): number;

Mean Gamma deviance regression loss.

Gamma deviance = (2/n) * Σ (log(y_pred / y_true) + y_true / y_pred - 1) Requires y_pred > 0 and y_true > 0. The terms are evaluated without the cancellation a direct evaluation suffers when predictions are close to the targets.

Parameters

yTrue Tensor: True positive target values of shape (n_samples,)
yPred Tensor: Predicted positive values of shape (n_samples,)

Returns

numberMean Gamma deviance (lower is better, 0 is perfect)

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf a value is not positive
DataValidationErrorIf an input contains NaN or infinite values

meanPinballLoss

function meanPinballLoss(yTrue: Tensor, yPred: Tensor, alpha?: number): number;

Mean pinball loss (quantile loss).

For a given quantile alpha: L(y, q) = alpha * max(y - q, 0) + (1 - alpha) * max(q - y, 0)

The pinball loss is the average of the quantile-specific losses. It is minimized by the alpha-quantile of the distribution.

Parameters

yTrue Tensor: True values of shape (n_samples,)
yPred Tensor: Predicted quantile values of shape (n_samples,)
alphaoptional number: Quantile level in (0, 1), default 0.5 (median)

Returns

numberMean pinball loss (non-negative)

Throws

ShapeErrorIf inputs have different lengths or are not 1D
InvalidParameterErrorIf alpha is not in (0, 1)
DataValidationErrorIf an input contains NaN or infinite values

meanPoissonDeviance

function meanPoissonDeviance(yTrue: Tensor, yPred: Tensor): number;

Mean Poisson deviance regression loss.

Poisson deviance = (2/n) * Σ (y_true * log(y_true / y_pred) - y_true + y_pred) Requires y_pred > 0 and y_true >= 0. The terms are evaluated without the cancellation a direct evaluation suffers when predictions are close to the targets.

Parameters

yTrue Tensor: True non-negative target values of shape (n_samples,)
yPred Tensor: Predicted positive values of shape (n_samples,)

Returns

numberMean Poisson deviance (lower is better, 0 is perfect)

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf yTrue has a negative value or yPred a non-positive value
DataValidationErrorIf an input contains NaN or infinite values

meanSquaredLogError

function meanSquaredLogError(yTrue: Tensor, yPred: Tensor, options?: WeightedMetricOptions): number;

Mean Squared Logarithmic Error (MSLE).

MSLE = (1/n) * Σ (log(1 + y_true) - log(1 + y_pred))² Only valid for non-negative values. With options.sampleWeight the mean is weighted: Σ w_i * term_i / Σ w_i, as in scikit-learn.

Parameters

yTrue Tensor: True non-negative values of shape (n_samples,)
yPred Tensor: Predicted non-negative values of shape (n_samples,)
optionsoptional WeightedMetricOptions: Optional sampleWeight, one weight per sample

Returns

numberMSLE score

Throws

ShapeErrorIf inputs are not 1D, have different lengths, or sampleWeight has the wrong length
InvalidParameterErrorIf a value is negative, or the weights sum to zero
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { meanSquaredLogError } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; const yTrue = tensor([3, 5, 2.5, 7]);const yPred = tensor([2.5, 5, 4, 8]);meanSquaredLogError(yTrue, yPred); // 0.03973012298459379meanSquaredLogError(yTrue, yPred, { sampleWeight: [1, 2, 3, 4] }); // 0.04549730536710696

multilabelConfusionMatrix

function multilabelConfusionMatrix(yTrue: Tensor, yPred: Tensor, labels?: number[]): MultilabelConfusionMatrixEntry[];

Compute a one-vs-rest confusion matrix for each class label.

For each label, returns the counts of a 2x2 confusion matrix: [[TN, FP], [FN, TP]], treating that label as the positive class.

Parameters

yTrue Tensor: True labels of shape (n_samples,)
yPred Tensor: Predicted labels of shape (n_samples,)
labelsoptional number[]: Optional label values to report, in the order returned. Defaults to the sorted union of the labels in yTrue and yPred.

Returns

MultilabelConfusionMatrixEntry[]Array of { label, tn, fp, fn, tp } for each label

Throws

ShapeErrorIf inputs are not 1D or have different lengths
InvalidParameterErrorIf labels contains a non-finite value
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { multilabelConfusionMatrix } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; multilabelConfusionMatrix(tensor([0, 1, 2, 1]), tensor([0, 2, 2, 1]), [2, 0, 1]);// [{ label: 2, tn: 2, fp: 1, fn: 0, tp: 1 }, { label: 0, tn: 3, fp: 0, fn: 0, tp: 1 }, ...]

smape

function smape(yTrue: Tensor, yPred: Tensor): number;

Symmetric Mean Absolute Percentage Error (SMAPE).

Parameters

yTrue Tensor: True values of shape (n_samples,)
yPred Tensor: Predicted values of shape (n_samples,)

Returns

numberSMAPE score in [0, 2]

Throws

ShapeErrorIf inputs are not 1D or have different lengths
DTypeErrorIf an input is a string tensor
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { smape } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; smape(tensor([100, 200]), tensor([110, 180])); // 0.1002506265664160

topKAccuracyScore

function topKAccuracyScore(yTrue: Tensor, yScore: Tensor, k?: number): number;

Top-K accuracy score.

A prediction is correct if the true label is among the K classes with the highest scores. Ties are resolved like scikit-learn: among equal scores the class with the larger index ranks first.

Parameters

yTrue Tensor: True labels of shape (n_samples,), integer class indices
yScore Tensor: Predicted scores of shape (n_samples, n_classes)
koptional number: Number of top predictions to consider (default: 5, capped at n_classes)

Returns

numberTop-K accuracy in [0, 1]

Throws

ShapeErrorIf yTrue is not 1D, yScore is not 2D, or the sample counts differ
InvalidParameterErrorIf k is not an integer in [1, n_classes], or a label is not an integer class index in [0, n_classes)
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { topKAccuracyScore } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; const yTrue = tensor([0, 1, 2]);const yScore = tensor([  [0.1, 0.5, 0.4],  [0.1, 0.8, 0.1],  [0.1, 0.1, 0.8],]);topKAccuracyScore(yTrue, yScore, 2); // 2 / 3

zeroOneLoss

function zeroOneLoss(yTrue: Tensor, yPred: Tensor, options?: { normalize?: boolean; }): number;

Zero-one classification loss (fraction of misclassifications).

Parameters

yTrue Tensor: True labels of shape (n_samples,); numeric, bool or string
yPred Tensor: Predicted labels of shape (n_samples,)
optionsoptional { normalize?: boolean; }: Optional parameters

Returns

numberFraction of misclassifications in [0, 1], or their count when normalize is false

Throws

ShapeErrorIf inputs are not 1D or have different lengths
DataValidationErrorIf a numeric input contains NaN or infinite values

ExampleTypeScript

import { zeroOneLoss } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; zeroOneLoss(tensor([0, 1, 2, 3]), tensor([0, 1, 0, 3])); // 0.25zeroOneLoss(tensor([0, 1, 2, 3]), tensor([0, 1, 0, 3]), { normalize: false }); // 1

ndcgScore

function ndcgScore(yTrue: Tensor, yScore: Tensor, k?: number): number;

Compute Normalized Discounted Cumulative Gain (NDCG) at rank k.

Details

Accepts either a single query (1-D tensors) or a batch of queries (2-D tensors of shape [nSamples, nLabels]), in which case the mean NDCG across samples is returned, matching scikit-learn's ndcg_score. Gains are linear (the relevance itself). Documents with equal predicted scores share their gain, so ties do not depend on input order. Queries without any relevant document score 0.

Parameters

yTrue Tensor: True non-negative relevance scores (1-D or 2-D tensor)
yScore Tensor: Predicted scores (1-D or 2-D tensor, same shape as yTrue)
koptional number: Number of top results to consider (default: all)

Returns

numberNDCG score in [0, 1]

Throws

ShapeErrorIf the inputs are not both 1D or both 2D, or their shapes differ
InvalidParameterErrorIf k is not a positive integer or yTrue has a negative value
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { ndcgScore } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; ndcgScore(tensor([[0, 1, 0, 2]]), tensor([[0.5, 0.5, 0.2, 0.5]]), 2); // 0.6199062332840657

pairwiseCosine

function pairwiseCosine(X: Tensor, Y?: Tensor): Tensor;

Compute pairwise cosine distances between rows.

cosine_distance = 1 - cosine_similarity, clipped to [0, 2].

A row of all zeros has no direction: its similarity to every other row is taken as 0, so its distance to them is 1 (as in scikit-learn). Without Y the diagonal is exactly 0.

Parameters

X Tensor: 2-D tensor of shape [n, d]
Yoptional Tensor: Optional 2-D tensor of shape [m, d]

Returns

TensorFloat64 tensor of shape [n, n] (or [n, m] with Y) containing cosine distances

Throws

ShapeErrorIf an input is not 2D or X and Y have different feature counts
DTypeErrorIf an input is a string tensor
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { pairwiseCosine } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; pairwiseCosine(tensor([[1, 0], [0, 1]])).toArray(); // [[0, 1], [1, 0]]

pairwiseEuclidean

function pairwiseEuclidean(X: Tensor, Y?: Tensor): Tensor;

Compute pairwise Euclidean distances between rows.

Without Y the result is the symmetric [n, n] matrix of distances between the rows of X; with Y it is the [n, m] matrix of distances from rows of X to rows of Y.

Parameters

X Tensor: 2-D tensor of shape [n, d]
Yoptional Tensor: Optional 2-D tensor of shape [m, d]

Returns

TensorFloat64 tensor of shape [n, n] (or [n, m] with Y) containing distances

Throws

ShapeErrorIf an input is not 2D or X and Y have different feature counts
DTypeErrorIf an input is a string tensor
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { pairwiseEuclidean } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; pairwiseEuclidean(tensor([[0, 0], [3, 4]])).toArray(); // [[0, 5], [5, 0]]

pairwiseManhattan

function pairwiseManhattan(X: Tensor, Y?: Tensor): Tensor;

Compute pairwise Manhattan (L1) distances between rows.

Without Y the result is the symmetric [n, n] matrix of distances between the rows of X; with Y it is the [n, m] matrix of distances from rows of X to rows of Y.

Parameters

X Tensor: 2-D tensor of shape [n, d]
Yoptional Tensor: Optional 2-D tensor of shape [m, d]

Returns

TensorFloat64 tensor of shape [n, n] (or [n, m] with Y) containing L1 distances

Throws

ShapeErrorIf an input is not 2D or X and Y have different feature counts
DTypeErrorIf an input is a string tensor
DataValidationErrorIf an input contains NaN or infinite values

reciprocalRank

function reciprocalRank(yTrue: Tensor, yScore: Tensor): number;

Compute the reciprocal rank (RR) of the first relevant item.

Items are ranked by decreasing score; equal scores keep their input order. For a batch of queries (2-D tensors of shape [nQueries, nItems]) the mean over queries is returned, which is the Mean Reciprocal Rank (MRR). Queries without a relevant item score 0.

Parameters

yTrue Tensor: Relevance labels (1-D or 2-D tensor); any positive value is relevant
yScore Tensor: Predicted scores (same shape as yTrue)

Returns

numberReciprocal rank in [0, 1] (mean reciprocal rank for 2-D input)

Throws

ShapeErrorIf the inputs are not both 1D or both 2D, or their shapes differ
DataValidationErrorIf an input contains NaN or infinite values

ExampleTypeScript

import { reciprocalRank } from 'deepbox/metrics';import { tensor } from 'deepbox/ndarray'; reciprocalRank(tensor([0, 0, 1]), tensor([0.3, 0.2, 0.1])); // 1 / 3

metrics-clustering.tsTypeScript

import {  adjustedRandScore,  ndcgScore,  pairwiseCosine,  silhouetteScore,  topKAccuracyScore,} from "deepbox/metrics";import { tensor } from "deepbox/ndarray"; const X = tensor([[1, 0], [0.9, 0.1], [0, 1], [0.1, 0.9]]);const labels = tensor([0, 0, 1, 1]); console.log(silhouetteScore(X, labels));console.log(adjustedRandScore(labels, tensor([1, 1, 0, 0]))); // 1, label names do not matterconsole.log(pairwiseCosine(X).toString());console.log(ndcgScore(tensor([[1, 0, 1]]), tensor([[0.9, 0.2, 0.7]])));console.log(topKAccuracyScore(tensor([2]), tensor([[0.1, 0.2, 0.9]]), 1));

Found a mistake? Open an issue.