deepbox/dataframe
DataFrame & Series
Column-oriented tables and labeled vectors with filtering, grouping, joins, reshaping, window functions and missing-value handling.
- Tabular data
Overview
- A
DataFrameis built from an object of equal-length column arrays, and aSeriesfrom one array. Most methods return a new frame instead of changing the receiver.null,undefinedandNaNall count as missing, and numeric methods skip them. - There is no frame-level
mean()orsum(). Take a column withdf.get("score"), which returns aSerieswithsum,mean,stdand more. Usedescribe()for a summary table, or aggregate throughgroupBy.filter(row => ...)takes a row predicate and does not select by label as pandas does. - Defaults that differ from pandas:
groupBylists groups in order of first appearance and keeps missing keys, so pass{ sort: true, dropna: true }for pandas behavior.DataFrame.valueCountscounts missing values, whileSeries.valueCountsdrops them.ewmdefaults toadjust: falseandbias: true. Series.stdandSeries.varuseddof = 1like pandas. This differs fromstdindeepbox/ndarrayanddeepbox/stats, which default toddof = 0.- 1.5.0 added
fillnawith per-column values or amethod,ffill()andbfill(),corrwithmethodandminPeriods,samplewithfrac,replaceandweights,groupBywithgetGroup,nunique,quantile,transformand named aggregation inagg,rollingwithminPeriodsandcenter,concatwithjoinandignoreIndex, andvalueCountswithnormalizeanddropna. sampledraws from the global generator, sosetSeedmakes it repeatable, or passrandomState.toTensor()andDataFrame.fromTensorconvert to and from tensors.
AggregateFunction
typeSupported aggregation functions for DataFrame groupby operations.
Numeric aggregations skip null, undefined and NaN and throw on non-numeric values. With no values left, sum is 0 and the other numeric aggregations are NaN.
- sum: Sum of values - mean: Arithmetic mean - median: Median value - min: Minimum value - max: Maximum value - std: Sample standard deviation (ddof = 1; NaN for fewer than two values) - var: Sample variance (ddof = 1; NaN for fewer than two values) - count: Number of non-missing values - first: First value in the group (missing values included) - last: Last value in the group (missing values included) - nunique: Number of distinct values, not counting missing ones
type AggregateFunction = "sum" | "mean" | "median" | "min" | "max" | "std" | "var" | "count" | "first" | "last" | "nunique";ConcatOptions
typeOptions for DataFrame.concat.
readonly join?: "outer" | "inner"readonly ignoreIndex?: boolean
CorrelationMethod
typeCorrelation coefficient computed by DataFrame.corr.
type CorrelationMethod = "pearson" | "spearman" | "kendall";CorrOptions
typeOptions for DataFrame.corr.
readonly method?: CorrelationMethodreadonly minPeriods?: number
DataFrameData
typeColumn-oriented data for DataFrame construction.
Maps column names to arrays of values. All arrays must have the same length.
type DataFrameData = Record<string, unknown[]>;DataFrameOptions
typeConfiguration options for DataFrame construction.
index?: (string | number)[]columns?: string[]copy?: boolean
DataValue
typeScalar value that can be stored in a DataFrame or Series cell.
Columns are not limited to these types (dates, bigints and objects are accepted too), but these are the values every operation understands. null and undefined mark missing values.
type DataValue = number | string | boolean | null | undefined;FillMethod
typeDirection of a fill: "ffill" carries the last valid value forward, "bfill" the next one back.
type FillMethod = "ffill" | "bfill";FillnaMethodOptions
typeOptions for filling missing values by propagating neighbours.
readonly method: FillMethod | "pad" | "backfill"readonly limit?: numberreadonly axis?: number | "index" | "rows" | "columns"
FillOptions
typeOptions for DataFrame.ffill and DataFrame.bfill.
readonly limit?: numberreadonly axis?: number | "index" | "rows" | "columns"
GroupByOptions
typeOptions for DataFrame.groupBy.
The defaults keep the behavior of Deepbox 1.0: groups are listed in order of first appearance and missing keys form groups. pandas sorts keys and drops missing keys by default, so pass { sort: true, dropna: true } to match it.
readonly sort?: booleanreadonly dropna?: boolean
NamedAggregation
typeNamed aggregation for DataFrameGroupBy.agg: a [column, aggregation] pair that is stored under the output name used as its key, like pandas' agg(out=("col", "mean")). The aggregation is a name from AggregateFunction or a function that receives the values of the column inside one group (missing values included).
type NamedAggregation = readonly [ column: string, aggregation: AggregateFunction | ((values: unknown[]) => unknown)];RollingOptions
typeOptions for DataFrame.rolling.
readonly on?: stringreadonly minPeriods?: numberreadonly center?: boolean
SampleOptions
typeOptions for DataFrame.sample.
readonly n?: numberreadonly frac?: numberreadonly replace?: booleanreadonly weights?: readonly number[] | stringreadonly randomState?: number
SeriesOptions
typeConfiguration options for Series construction.
name?: stringindex?: (string | number)[]copy?: boolean
ValueCountsOptions
typeOptions for Series.valueCounts and DataFrame.valueCounts.
readonly normalize?: booleanreadonly dropna?: booleanreadonly sort?: booleanreadonly ascending?: boolean
DataFrame
Two-dimensional, size-mutable, potentially heterogeneous tabular data.
A DataFrame is like a spreadsheet or SQL table. It consists of: - Rows (observations) identified by an index - Columns (variables) identified by column names - Data stored in a columnar format for efficient access
A tabular data structure with labeled columns.
import { DataFrame } from 'deepbox/dataframe'; const df = new DataFrame({name: ['Alice', 'Bob', 'Charlie'],age: [25, 30, 35],score: [85.5, 92.0, 78.5]}); console.log(df.shape); // [3, 3]console.log(df.columns); // ['name', 'age', 'score']Members (107)
constructor(data: DataFrameData, options?: DataFrameOptions)Creates a new DataFrame instance.
shape: [ number, number ]Get the dimensions of the DataFrame.
plot: PlotAccessorPlotting accessor.
style: StyleAccessorStyle accessor.
columns: string[]Get the column names.
index: (string | number)[]Get the row index labels.
get(column: string): Series<unknown>Get a column as a Series.
get<T>(column: string, guard: (value: unknown) => value is T): Series<T>loc(row: string | number): Record<string, unknown>Access a row by label (label-based indexing).
iloc(position: number): Record<string, unknown>Access a row by integer position (position-based indexing).
head(n?: number): DataFrameReturn the first n rows.
tail(n?: number): DataFrameReturn the last n rows.
filter(predicate: (row: Record<string, any>) => boolean): DataFrameFilter rows based on a boolean predicate function.
select(columns: string[]): DataFrameSelect a subset of columns.
drop(columns: string[]): DataFrameDrop (remove) specified columns.
sort(by: string | string[], ascending?: boolean | boolean[]): DataFrameSort DataFrame by one or more columns.
groupBy(by: string | string[], options?: GroupByOptions): DataFrameGroupByGroup DataFrame by one or more columns.
join(other: DataFrame, on: string, how?: "inner" | "left" | "right" | "outer"): DataFrameJoin with another DataFrame using SQL-style join.
merge(other: DataFrame, options?: { readonly on?: string; readonly leftOn?: string; readonly rightOn?: string; readonly left_on?: string; readonly right_on?: string; readonly how?: "inner" | "left" | "right" | "outer"; readonly suffixes?: readonly [ string, string ]; }): DataFrameMerge with another DataFrame using SQL-style merge.
concat(other: DataFrame, axis?: Axis | ConcatOptions, options?: ConcatOptions): DataFrameConcatenate with another DataFrame.
fillna(value: unknown): DataFrameFill missing values (null, undefined or NaN).
ffill(options?: FillOptions): DataFrameFill missing values with the previous valid value (forward fill).
bfill(options?: FillOptions): DataFrameFill missing values with the next valid value (backward fill).
dropna(options?: { readonly how?: "any" | "all"; readonly subset?: readonly string[]; readonly thresh?: number; }): DataFrameDrop rows that contain missing values (null, undefined or NaN).
describe(): DataFrameGenerate descriptive statistics.
corr(method?: CorrelationMethod | CorrOptions, minPeriods?: number): DataFrameCompute the pairwise correlation matrix of the numeric columns.
cov(): DataFrameCompute covariance matrix.
apply<U = unknown>(fn: (series: Series<unknown>) => Series<U>, axis?: Axis): DataFrameApply a function along an axis of the DataFrame.
toTensor(options?: { readonly dtype?: "float32" | "float64"; }): TensorConvert DataFrame to a 2D Tensor.
toArray(): unknown[][]Convert DataFrame to a 2D JavaScript array.
static fromCsvString(csvString: string, options?: { readonly delimiter?: string; readonly quoteChar?: string; readonly hasHeader?: boolean; readonly skipRows?: number; }): DataFrameParse a CSV string into a DataFrame, inferring cell types and handling quoted fields.
static readCsv(path: string, options?: { readonly delimiter?: string; readonly quoteChar?: string; readonly hasHeader?: boolean; readonly skipRows?: number; }): Promise<DataFrame>Read CSV file - environment-aware (Node.js fs or browser fetch).
toCsvString(options?: { readonly delimiter?: string; readonly quoteChar?: string; readonly includeIndex?: boolean; readonly header?: boolean; }): stringConvert DataFrame to CSV string with proper quoting and escaping.
toCsv(path: string, options?: { readonly delimiter?: string; readonly quoteChar?: string; readonly includeIndex?: boolean; readonly header?: boolean; }): Promise<void>Write DataFrame to CSV file - environment-aware.
toJsonString(): stringSerialize DataFrame to a JSON string with
columns,indexanddatafields.static fromJsonString(jsonStr: string): DataFrameCreate DataFrame from JSON string.
static readJson(path: string): Promise<DataFrame>Read JSON file - environment-aware.
toJson(path: string): Promise<void>Write DataFrame to JSON file - environment-aware.
toParquet(options?: ParquetWriteOptions): Uint8ArraySerialize the DataFrame to Parquet and return the file bytes.
static fromParquet(buffer: Uint8Array, options?: ParquetReadOptions): DataFrameCreate a DataFrame from Parquet file bytes.
toXlsx(options?: XlsxWriteOptions): Uint8ArraySerialize the DataFrame to an XLSX workbook and return the file bytes.
static fromXlsx(buffer: Uint8Array, options?: XlsxReadOptions): DataFrameCreate a DataFrame from XLSX file bytes.
static fromTensor(source: Tensor, columns?: string[]): DataFrameCreate DataFrame from a Tensor.
dropDuplicates(subset?: string[], keep?: "first" | "last" | false): DataFrameRemove duplicate rows from DataFrame.
drop_duplicates(subset?: string[], keep?: "first" | "last" | false): DataFramedeprecatedSame as DataFrame.dropDuplicates.
duplicated(subset?: string[], keep?: "first" | "last" | false): Series<boolean>Return boolean Series indicating duplicate rows.
rename(mapper: Record<string, string> | ((name: string) => string), axis?: Axis): DataFrameRename columns or index labels.
resetIndex(drop?: boolean): DataFrameReset index to default integer index.
reset_index(drop?: boolean): DataFramedeprecatedSame as DataFrame.resetIndex.
setIndex(column: string, drop?: boolean): DataFrameSet a column as the index.
set_index(column: string, drop?: boolean): DataFramedeprecatedSame as DataFrame.setIndex.
isnull(): DataFrameReturn boolean DataFrame showing missing values (null, undefined and NaN).
notnull(): DataFrameReturn boolean DataFrame showing present values (not null, undefined or NaN).
replace(toReplace: unknown | unknown[], value: unknown): DataFrameReplace values in DataFrame.
clip(lower?: number, upper?: number): DataFrameClip (limit) values in a range.
sample(n?: number | SampleOptions, randomState?: number): DataFrameReturn a random sample of rows.
quantile(q: number): Series<number>Return values at the given quantile, using linear interpolation between the two nearest ranks (the NumPy and pandas default).
rank(method?: "average" | "min" | "max" | "first" | "dense", ascending?: boolean): DataFrameCompute numerical rank of values (1 through n) for each column.
diff(periods?: number): DataFrameCalculate the difference between consecutive rows.
pctChange(periods?: number): DataFrameCalculate percentage change between consecutive rows.
pct_change(periods?: number): DataFramedeprecatedSame as DataFrame.pctChange.
cumsum(): DataFrameReturn the cumulative sum of each column.
cumprod(): DataFrameReturn the cumulative product of each column.
cummax(): DataFrameReturn the cumulative maximum of each column.
cummin(): DataFrameReturn the cumulative minimum of each column.
shift(periods?: number, fillValue?: unknown): DataFrameShift index by desired number of periods.
pivot(index: string, columns: string, values: string): DataFramePivot DataFrame.
melt(idVars: string[], valueVars?: string[], varName?: string, valueName?: string): DataFrameMelt DataFrame.
stack(options?: { readonly columns?: string[]; readonly varName?: string; readonly valueName?: string; }): DataFrameStack (pivot) specified value columns into rows.
unstack(options: { readonly index: string; readonly column: string; readonly value: string; }): DataFrameUnstack (pivot) a column's values into new columns.
rolling(window: number, on?: string | RollingOptions): RollingCreate a Rolling object for window-based calculations.
applymap(fn: (value: unknown) => unknown): DataFrameApply a function element-wise to the DataFrame.
pipe<T>(fn: (df: DataFrame, ...args: unknown[]) => T, ...args: unknown[]): TPass the DataFrame through a function chain.
explode(column: string): DataFrameExplode a list-like column into separate rows.
getDummies(columns?: string | string[], options?: { prefix?: string; dropFirst?: boolean; dummyNa?: boolean; }): DataFrameConvert categorical column(s) into dummy/indicator variables.
toString(maxRows?: number): stringReturn a human-readable tabular string representation.
abs(): DataFrameElement-wise absolute value for all numeric columns.
round(decimals?: number): DataFrameRound numeric columns to the given number of decimal places.
nunique(): SeriesCount distinct values per column.
nlargest(n: number, column: string): DataFrameReturn the n rows with the largest values in a column, largest first.
nsmallest(n: number, column: string): DataFrameReturn the n rows with the smallest values in a column, smallest first.
idxmin(): SeriesIndex label of the first minimum value in each column.
idxmax(): SeriesIndex label of the first maximum value in each column.
between(column: string, left: number, right: number, inclusive?: boolean | "both" | "neither" | "left" | "right"): DataFrameKeep the rows whose value in
columnlies betweenleftandright.assign(columns: Record<string, unknown[] | ((row: Record<string, unknown>, i: number) => unknown) | string | number | boolean | bigint | Date | null>): DataFrameAdd or modify columns functionally.
where(cond: boolean[] | ((val: unknown, i: number) => boolean), other?: unknown): DataFrameReplace values where condition is false with other.
mask(cond: boolean[] | ((val: unknown, i: number) => boolean), other?: unknown): DataFrameReplace values where condition is true with other.
astype(column: string, dtype: "number" | "string" | "boolean"): DataFrameCast columns to a target type.
astype(dtypes: Record<string, "number" | "string" | "boolean">): DataFrameinfo(): stringSummarize the DataFrame as text: row count, and for every column the number of non-null values and a dtype guess taken from its first non-null value.
any(axis?: Axis): Series | boolean[]Whether any value is truthy.
all(axis?: Axis): Series | boolean[]Whether all values are truthy.
valueCounts(...args: (string | ValueCountsOptions)[]): DataFrameCount occurrences of unique value combinations across specified columns.
value_counts(...args: (string | ValueCountsOptions)[]): DataFramedeprecatedSame as DataFrame.valueCounts.
query(expr: string): DataFrameFilter rows using a simple expression string.
memoryUsage(): DataFrameReturn approximate memory usage in bytes per column.
memory_usage(): DataFramedeprecatedSame as DataFrame.memoryUsage.
interpolate(method?: "linear" | "nearest"): DataFrameFill NaN/null values by interpolation, like pandas' default settings.
pivotTable(options: { index: string; columns: string; values: string; aggFunc?: "mean" | "sum" | "count" | "min" | "max" | "median" | "first" | "last"; }): DataFramePivot with aggregation (unlike pivot() which requires unique index/column pairs).
pivot_table(options: { index: string; columns: string; values: string; aggFunc?: "mean" | "sum" | "count" | "min" | "max" | "median" | "first" | "last"; }): DataFramedeprecatedSame as DataFrame.pivotTable.
expanding(minPeriods?: number): ExpandingExpanding (cumulative) window calculations.
ewm(options: { span?: number; alpha?: number; com?: number; halflife?: number; adjust?: boolean; bias?: boolean; ignoreNa?: boolean; }): EWMExponentially weighted moving calculations.
copy(): DataFrameReturn a deep copy of the DataFrame.
isin(values: readonly unknown[]): DataFrameTest whether each element is contained in the given values.
transform(fn: (series: Series<unknown>) => Series<unknown>, axis?: Axis): DataFrameApply a function to each column (axis=0) or each row (axis=1), producing a result with the same shape as the input.
eval(expr: string): DataFrameEvaluate a string expression on the DataFrame, producing a new column or filtering rows.
crosstab(rowCol: string, colCol: string): DataFrameCompute a cross-tabulation of two columns.
DataFrameGroupBy
GroupBy object for aggregation operations.
Created by DataFrame.groupBy(). Used to perform aggregations on grouped data.
const df = new DataFrame({ category: ['A', 'B', 'A', 'B'], value: [10, 20, 30, 40]});const grouped = df.groupBy('category');grouped.sum(); // Sum by categorygrouped.mean(); // Mean by categoryMembers (18)
constructor(df: DataFrame, by: string | string[], options?: GroupByOptions)ngroups: numberNumber of distinct groups.
agg(operations: Record<string, AggregateFunction | AggregateFunction[] | NamedAggregation>): DataFrameAggregate grouped data.
sum(): DataFrameCompute sum for each group.
mean(): DataFrameCompute mean (average) for each group.
count(): DataFrameCount non-null values in each non-grouping column for every group.
min(): DataFrameCompute minimum value for each group.
max(): DataFrameCompute maximum value for each group.
std(): DataFrameCompute standard deviation for each group.
var(): DataFrameCompute variance for each group.
median(): DataFrameCompute median for each group.
first(): DataFrameFirst value of every non-grouping column in each group (missing values included).
last(): DataFrameLast value of every non-grouping column in each group (missing values included).
size(): DataFrameNumber of rows in each group, missing values included.
getGroup(key: unknown): DataFrameRows of one group, with all columns and the original row labels.
nunique(): DataFrameNumber of distinct values, not counting missing ones, of every non-grouping column in each group.
quantile(q: number): DataFrameQuantile of every numeric column in each group, by linear interpolation between ranks (the NumPy and pandas default).
transform(how: AggregateFunction | "cumsum" | "cumprod" | "cummax" | "cummin" | "ffill" | "bfill" | ((values: Series<unknown>) => unknown)): DataFrameTransform every non-grouping column group by group, returning a frame with the same rows (and row labels) as the original, so the result lines up with the source data.
EWM
Exponentially weighted moving calculations on DataFrame columns.
Created by DataFrame.ewm(). The weights follow pandas' algorithm, so results can be compared against Series.ewm(...) with the same adjust, bias and ignoreNa settings. NaN, null, infinite and non-numeric values are treated as missing observations.
Members (4)
constructor(df: DataFrame, alpha: number, settings?: { adjust: boolean; bias: boolean; ignoreNa: boolean; })mean(): DataFrameExponentially weighted mean.
std(): DataFrameExponentially weighted standard deviation.
var(): DataFrameExponentially weighted variance.
Expanding
Expanding (cumulative) window calculations on DataFrame columns.
Created by DataFrame.expanding(). NaN, null, infinite and non-numeric values are not observations (pandas treats infinities the same way in window functions); a row's result needs at least minPeriods observations so far.
Members (7)
constructor(df: DataFrame, minPeriods: number)mean(): DataFrameExpanding mean (compensated running sum).
sum(): DataFrameExpanding sum (compensated).
std(): DataFrameExpanding sample standard deviation (ddof = 1).
var(): DataFrameExpanding sample variance (ddof = 1).
min(): DataFrameExpanding minimum.
max(): DataFrameExpanding maximum.
Rolling
Rolling window calculations on DataFrame columns.
Created by DataFrame.rolling(). Provides mean, sum, std, var, min, max, median, apply, corr and cov. Like pandas with its default min_periods, the single-column statistics need a full window of valid numbers: windows that reach before the first row or contain a NaN, null or non-numeric value give null. With minPeriods they work from the valid values of a shorter window, and with center the window is placed around each row. corr and cov use the valid pairs inside each window and need at least two (or minPeriods, when given; the window must then be full only if minPeriods is not given).
Members (11)
constructor(df: DataFrame, window: number, on?: string, options?: { readonly minPeriods?: number; readonly center?: boolean; })mean(): DataFrameRolling mean.
sum(): DataFrameRolling sum.
var(): DataFrameRolling sample variance (ddof = 1); null for a window of one value.
std(): DataFrameRolling sample standard deviation (ddof = 1); null for a window of one value.
min(): DataFrameRolling minimum.
max(): DataFrameRolling maximum.
median(): DataFrameRolling median.
apply(fn: (vals: number[]) => number | null): DataFrameApply a function to every full window.
corr(col1: string, col2: string): DataFrameCompute rolling pairwise correlation between two columns.
cov(col1: string, col2: string): DataFrameCompute rolling pairwise covariance between two columns.
Series
One-dimensional labeled array capable of holding any data type.
A Series is like a column in a spreadsheet or database table. It combines: - An array of data values - An array of index labels (can be strings or numbers) - An optional name
// Create a numeric seriesconst s = new Series([1, 2, 3, 4], { name: 'numbers' }); // Create a series with custom indexconst s2 = new Series(['a', 'b', 'c'], { index: ['row1', 'row2', 'row3'], name: 'letters'});Members (33)
constructor(data: T[], options?: SeriesOptions)Creates a new Series instance.
data: readonly T[]Get the underlying data array.
index: readonly (string | number)[]Get the index labels.
name: string | undefinedGet the Series name.
str: StringAccessorAccess string methods on this Series.
dt: DateTimeAccessorAccess datetime methods on this Series.
length: numberGet the number of elements in the Series.
get(label: number | string): T | undefinedGet a value by label.
loc(label: string | number): T | undefinedAccess a value by label (label-based indexing).
iloc(position: number): T | undefinedAccess a value by integer position (position-based indexing).
head(n?: number): Series<T>Return the first n elements.
tail(n?: number): Series<T>Return the last n elements.
filter(predicate: (value: T, index: number) => boolean): Series<T>Filter Series by a boolean predicate function.
map<U>(fn: (value: T, index: number) => U): Series<U>Transform each element using a mapping function.
sort(ascending?: boolean): Series<T>Sort the Series values.
unique(): T[]Get unique values in the Series.
valueCounts(dropna?: boolean | ValueCountsOptions): Series<number>Count occurrences of unique values.
fillna(options: FillnaMethodOptions): Series<T>Fill missing values (null, undefined, NaN and invalid Dates).
fillna<U>(value: U): Series<T | U>ffill(limit?: number): Series<T>Fill missing values with the previous valid value (forward fill).
bfill(limit?: number): Series<T>Fill missing values with the next valid value (backward fill).
sum(): numberCalculate the sum of all values.
mean(): numberCalculate the arithmetic mean (average) of all values.
median(): numberCalculate the median (middle value) of all values.
std(ddof?: number): numberCalculate the standard deviation of all values.
var(ddof?: number): numberCalculate the variance of all values.
min(): numberFind the minimum value in the Series.
max(): numberFind the maximum value in the Series.
isnull(): Series<boolean>Boolean Series marking missing values (null, undefined, NaN and invalid Dates).
notnull(): Series<boolean>Boolean Series marking present values, the complement of Series.isnull.
toArray(): T[]Convert the Series to a plain JavaScript array.
toTensor(options?: { readonly dtype?: "float32" | "float64"; }): TensorConvert the Series to a 1D ndarray Tensor.
toString(maxRows?: number): stringReturn a human-readable string representation of this Series.
import { DataFrame, Series } from "deepbox/dataframe"; const df = new DataFrame({ name: ["Alice", "Bob", "Charlie", "Dana"], team: ["B", "A", "B", "A"], score: [91, 84, 96, null],}); const score = df.get("score");console.log(score.mean(), score.std()); // the missing value is skipped; std uses ddof = 1 console.log(df.groupBy("team", { sort: true, dropna: true }).mean().toString());console.log(df.groupBy("team").agg({ best: ["score", "max"] }).toString());console.log(df.fillna({ score: 0 }).filter((row) => Number(row.score) > 90).toString());console.log(df.describe().toString()); const s = new Series([1, 2, 3, 4], { name: "value" });console.log(s.mean(), s.std());Found a mistake? Open an issue.