Skip to content

deepbox/preprocess

Scalers

StandardScaler, MinMaxScaler, MaxAbsScaler, RobustScaler, Normalizer, QuantileTransformer and PowerTransformer for numeric feature matrices.

  • Scaling

Overview

  • Fit on the training split only, then call transform on every other split. The scalers have fit, transform, fitTransform and inverseTransform. StandardScaler and MinMaxScaler also have partialFit for batches. Normalizer is stateless and has no inverse.
  • Input is a 2-D tensor, and the output is a float64 tensor whatever the input dtype. Fitted values are getters without scikit-learn's trailing underscore: scaler.mean, scaler.scale, scaler.variance.
  • StandardScaler divides by the population standard deviation (divisor n), as scikit-learn does. A feature with zero variance is only centered. RobustScaler centers on the median and divides by the interquartile range.
  • PowerTransformer defaults to standardize: false, where scikit-learn defaults to true. Pass { standardize: true } to match it. The default method is "yeo-johnson", and "box-cox" needs strictly positive data.

Classes

7

MaxAbsScaler

Scale each feature by its maximum absolute value.

The result lies in [-1, 1]. No centering is applied, so zeros stay zeros. Features that are entirely zero are left unchanged.

Fitted attributes (read-only getters): maxAbs, scale, nSamplesSeen, nFeaturesIn.

Members (13)
  • constructor(options?: { copy?: boolean; })

    Creates a new MaxAbsScaler.

  • maxAbs: Tensor | undefined

    Per-feature maximum absolute value, or undefined if unfitted.

  • scale: Tensor | undefined

    Per-feature divisor applied by transform (1 for all-zero features), or undefined if unfitted.

  • nSamplesSeen: number

    Number of samples seen so far (0 when unfitted).

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Compute the maximum absolute value of each feature, discarding any previous fit.

  • partialFit(X: Tensor, _y?: Tensor): this

    Update the running maximum absolute values with another batch of samples.

  • transform(X: Tensor): Tensor

    Divide each feature by its fitted maximum absolute value.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • inverseTransform(X: Tensor): Tensor

    Multiply each feature by its fitted maximum absolute value.

  • getParams(): Record<string, unknown>

    Constructor options of this scaler.

  • setParams(_params: Record<string, unknown>): this

    This scaler has no options; present for API consistency.

  • clone(): MaxAbsScaler

    Create an unfitted scaler.

MinMaxScaler

Scale each feature to a given range, by default [0, 1].

Formula: X_scaled = (X - X.min) / (X.max - X.min) * (max - min) + min

A feature with a single distinct value is shifted so that this value maps to featureRange[0]. Values outside the fitted range map outside the target range unless clip is set.

Fitted attributes (read-only getters): dataMin, dataMax, dataRange, scale, nSamplesSeen, nFeaturesIn.

Members (15)
  • constructor(options?: { featureRange?: [ number, number ]; clip?: boolean; copy?: boolean; })

    Creates a new MinMaxScaler.

  • dataMin: Tensor | undefined

    Per-feature minimum seen during fitting, or undefined if unfitted.

  • dataMax: Tensor | undefined

    Per-feature maximum seen during fitting, or undefined if unfitted.

  • dataRange: Tensor | undefined

    Per-feature range (max - min) seen during fitting, or undefined if unfitted.

  • scale: Tensor | undefined

    Per-feature multiplier applied by transform, or undefined if unfitted.

  • nSamplesSeen: number

    Number of samples seen so far (0 when unfitted).

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Compute the per-feature minimum and maximum, discarding any previous fit.

  • partialFit(X: Tensor, _y?: Tensor): this

    Update the running minimum and maximum with another batch of samples.

  • transform(X: Tensor): Tensor

    Map X into the feature range using the fitted minimum and maximum.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • inverseTransform(X: Tensor): Tensor

    Map data from the feature range back to the original feature space.

  • getParams(): Record<string, unknown>

    Constructor options of this scaler.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): MinMaxScaler

    Create an unfitted scaler with the same options.

Normalizer

Scale each sample (row) to unit norm.

Rows whose norm is zero are left unchanged. The transformer is stateless: fit only validates its input.

Example
import { Normalizer } from 'deepbox/preprocess';import { tensor } from 'deepbox/ndarray'; const X = tensor([[3, 4], [1, 0]]);const Xn = new Normalizer({ norm: 'l2' }).transform(X); // [[0.6, 0.8], [1, 0]]
Members (7)
  • constructor(options?: { norm?: "l1" | "l2" | "max"; copy?: boolean; })

    Creates a new Normalizer.

  • fit(X: Tensor, _y?: Tensor): this

    Validate X.

  • transform(X: Tensor): Tensor

    Divide each row by its norm.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Transform X (there is nothing to fit).

  • getParams(): Record<string, unknown>

    Constructor options of this normalizer.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): Normalizer

    Create a normalizer with the same options.

PowerTransformer

Apply a power transform to make data more Gaussian-like.

Supports Box-Cox (strictly positive data) and Yeo-Johnson (any real data). The exponent λ of each feature is chosen by maximum likelihood on the training data, searching λ in [-5, 5] and widening the search up to [-160, 160] when the optimum lies at the edge. With standardize: true the transformed features are additionally centered and scaled to unit variance.

Difference from scikit-learn: standardize defaults to false here, while scikit-learn's PowerTransformer defaults to True. Pass { standardize: true } to match scikit-learn. The default is kept for compatibility with earlier Deepbox releases.

Fitted attributes (read-only getters): lambdas, mean, scale, nFeaturesIn.

Members (12)
  • constructor(options?: { method?: "box-cox" | "yeo-johnson"; standardize?: boolean; copy?: boolean; })

    Creates a new PowerTransformer.

  • lambdas: Tensor | undefined

    Fitted exponent of each feature, or undefined if unfitted.

  • mean: Tensor | undefined

    Mean of each transformed feature, or undefined if unfitted or standardize is false.

  • scale: Tensor | undefined

    Standard deviation of each transformed feature (1 if constant), or undefined if unfitted or standardize is false.

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Estimate the exponent of each feature by maximum likelihood.

  • transform(X: Tensor): Tensor

    Apply the fitted power transform.

  • inverseTransform(X: Tensor): Tensor

    Inverse transform data back to the original feature space.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • getParams(): Record<string, unknown>

    Constructor options of this transformer.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): PowerTransformer

    Create an unfitted transformer with the same options.

QuantileTransformer

Transform features so that their distribution is uniform or normal.

Each feature is mapped through its empirical quantile function, estimated from nQuantiles evenly spaced quantiles of the training data and interpolated linearly in between. Values outside the training range are mapped to the ends of the output range. Tied training values map to the average of the first and last quantile they cover. With outputDistribution: "normal" the output is clipped to about ±5.2.

Unlike scikit-learn, transform does not snap values lying within an absolute distance of 1e-7 of the training minimum or maximum to the ends of the output range, because that tolerance depends on the data scale. Results differ from scikit-learn only for such values, by at most the local slope times 1e-7.

Fitted attributes (read-only getters): quantiles, references, nFeaturesIn.

Members (11)
  • constructor(options?: { nQuantiles?: number; outputDistribution?: "uniform" | "normal"; subsample?: number; randomState?: number; copy?: boolean; })

    Creates a new QuantileTransformer.

  • quantiles: Tensor | undefined

    Fitted quantiles as a tensor of shape (n_quantiles, n_features), or undefined if unfitted.

  • references: Tensor | undefined

    Probabilities that the fitted quantiles correspond to, or undefined if unfitted.

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Estimate the quantiles of each feature, discarding any previous fit.

  • transform(X: Tensor): Tensor

    Map X to the output distribution.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • inverseTransform(X: Tensor): Tensor

    Inverse transform data back to the original feature space.

  • getParams(): Record<string, unknown>

    Constructor options of this transformer.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): QuantileTransformer

    Create an unfitted transformer with the same options.

RobustScaler

Scale features using statistics that are less sensitive to outliers.

Each feature has its median removed and is divided by an inter-quantile range (the IQR, from the 25th to the 75th percentile, by default). Quantiles use linear interpolation, like numpy.percentile. Features whose range is zero are only centered.

Fitted attributes (read-only getters): center, scale, nFeaturesIn.

Members (11)
  • constructor(options?: { withCentering?: boolean; withScaling?: boolean; quantileRange?: [ number, number ]; unitVariance?: boolean; copy?: boolean; })

    Creates a new RobustScaler.

  • center: Tensor | undefined

    Per-feature median, or undefined if unfitted or withCentering is false.

  • scale: Tensor | undefined

    Per-feature scale (1 where the range is zero), or undefined if unfitted or withScaling is false.

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Compute the median and quantile range of each feature.

  • transform(X: Tensor): Tensor

    Remove the median and divide by the quantile range.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • inverseTransform(X: Tensor): Tensor

    Undo the scaling: x * scale + center.

  • getParams(): Record<string, unknown>

    Constructor options of this scaler.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): RobustScaler

    Create an unfitted scaler with the same options.

StandardScaler

Standardize features by removing the mean and scaling to unit variance.

Formula: z = (x - μ) / σ, with σ the population standard deviation (divisor n, as in scikit-learn).

Features with zero variance are only centered; their scale is treated as 1. Statistics are accumulated in a numerically stable way, so StandardScaler.partialFit can be called on successive batches and gives the same result as a single fit on all rows.

Fitted attributes (read-only getters): mean, variance, scale, nSamplesSeen, nFeaturesIn.

Example
import { StandardScaler } from 'deepbox/preprocess';import { tensor } from 'deepbox/ndarray'; const X = tensor([[1, 2], [3, 4], [5, 6]]);const scaler = new StandardScaler();const XScaled = scaler.fitTransform(X);const XBack = scaler.inverseTransform(XScaled);
Members (14)
  • constructor(options?: { withMean?: boolean; withStd?: boolean; copy?: boolean; })

    Creates a new StandardScaler.

  • mean: Tensor | undefined

    Per-feature mean seen during fitting, or undefined if unfitted or withMean is false.

  • variance: Tensor | undefined

    Per-feature population variance, or undefined if unfitted or withStd is false.

  • scale: Tensor | undefined

    Per-feature scale (standard deviation, 1 for constant features), or undefined if unfitted or withStd is false.

  • nSamplesSeen: number

    Number of samples seen so far (0 when unfitted).

  • nFeaturesIn: number

    Number of features seen during fitting (0 when unfitted).

  • fit(X: Tensor, _y?: Tensor): this

    Compute the mean and standard deviation of each feature, discarding any previous fit.

  • partialFit(X: Tensor, _y?: Tensor): this

    Update the running mean and variance with another batch of samples.

  • transform(X: Tensor): Tensor

    Center and scale X with the fitted statistics.

  • fitTransform(X: Tensor, _y?: Tensor): Tensor

    Fit to X, then transform it.

  • inverseTransform(X: Tensor): Tensor

    Undo the scaling: x * scale + mean.

  • getParams(): Record<string, unknown>

    Constructor options of this scaler.

  • setParams(params: Record<string, unknown>): this

    Change options.

  • clone(): StandardScaler

    Create an unfitted scaler with the same options.

preprocess-scalers.ts
import {  MinMaxScaler,  PowerTransformer,  RobustScaler,  StandardScaler,} from "deepbox/preprocess";import { tensor } from "deepbox/ndarray"; const train = tensor([[1, 10], [2, 20], [3, 30]]);const test = tensor([[4, 40]]); const scaler = new StandardScaler().fit(train);console.log(scaler.mean?.toString(), scaler.scale?.toString());console.log(scaler.transform(test).toString()); // uses the training statisticsconsole.log(scaler.inverseTransform(scaler.transform(train)).toString()); console.log(new MinMaxScaler().fitTransform(train).toString());console.log(new RobustScaler().fitTransform(train).toString());console.log(new PowerTransformer({ standardize: true }).fitTransform(train).toString());

Found a mistake? Open an issue.