Skip to content
Deepbox
33intermediate

DataFrame Advanced Features

Twelve short parts on Series and DataFrame features: string and datetime accessors, rolling, expanding and EWM windows, query and eval expressions, assign, pivot tables, crosstabs, nlargest, interpolation and gap filling.

What you will learn

  • Use deepbox/dataframe for Series.str, Series.dt, rolling, expanding, ewm, query, eval, assign, pivotTable, crosstab, nlargest, nsmallest, interpolate, ffill, bfill, fillna.
  • Twelve short parts on Series and DataFrame features: string and datetime accessors, rolling, expanding and EWM windows, query and eval expressions, assign, pivot tables, crosstabs, nlargest, interpolation and gap filling.

Source

index.ts
/** * Example 33: Advanced DataFrame Features * * String and DateTime accessors, rolling, expanding and EWM windows, * query/eval expressions, pivot tables, crosstabs, interpolation and gap filling. */ import { DataFrame, Series } from "deepbox/dataframe"; console.log("=".repeat(60));console.log("Example 33: Advanced DataFrame Features");console.log("=".repeat(60)); // ============================================================================// Part 1: Series String Accessor (str)// ============================================================================console.log("\nPart 1: Series String Accessor");console.log("-".repeat(60)); // The .str accessor applies string operations to every element of a Series. Missing values stay missing.const names = new Series(["Alice Smith", "bob jones", "CHARLIE BROWN", null, "diana prince"]); console.log("Original names:");console.log(names.toString()); // Case transformationsconst upper = names.str.upper();console.log("\n.str.upper():", upper.toString()); const lower = names.str.lower();console.log(".str.lower():", lower.toString()); const title = names.str.title();console.log(".str.title():", title.toString()); // String matchingconst containsI = names.str.contains("i");console.log("\n.str.contains('i'):", containsI.toString()); // String operationsconst lengths = names.str.len();console.log(".str.len():", lengths.toString()); const replaced = names.str.replace("o", "0");console.log(".str.replace('o', '0'):", replaced.toString()); const trimmed = new Series(["  hello  ", " world ", null]);console.log("\n.str.strip():", trimmed.str.strip().toString()); // Split and sliceconst emails = new Series(["alice@example.com", "bob@test.org", null]);console.log(".str.split('@'):", emails.str.split("@").toString()); const starts = names.str.startswith("A");console.log(".str.startswith('A'):", starts.toString()); const ends = names.str.endswith("e");console.log(".str.endswith('e'):", ends.toString()); // ============================================================================// Part 2: Series DateTime Accessor (dt)// ============================================================================console.log("\nPart 2: Series DateTime Accessor");console.log("-".repeat(60)); // The .dt accessor extracts date parts from every element of a Seriesconst dates = new Series([  new Date("2024-01-15T10:30:00"),  new Date("2024-06-20T14:45:30"),  new Date("2024-12-25T00:00:00"),  null,  new Date("2024-03-08T08:15:00"),]); console.log("Original dates:");console.log(dates.toString()); // Extract date componentsconst years = dates.dt.year();console.log("\n.dt.year():", years.toString()); const months = dates.dt.month();console.log(".dt.month():", months.toString()); const days = dates.dt.day();console.log(".dt.day():", days.toString()); const hours = dates.dt.hour();console.log(".dt.hour():", hours.toString()); const dayOfWeek = dates.dt.dayOfWeek();console.log(".dt.dayOfWeek():", dayOfWeek.toString()); const quarter = dates.dt.quarter();console.log(".dt.quarter():", quarter.toString()); // ============================================================================// Part 3: Rolling Window Calculations// ============================================================================console.log("\nPart 3: Rolling Window Calculations");console.log("-".repeat(60)); // Rolling windows compute statistics over a sliding windowconst stockPrices = new DataFrame({  price: [100, 102, 101, 105, 108, 107, 110, 112, 109, 115],  volume: [1000, 1200, 900, 1500, 1300, 1100, 1400, 1600, 1000, 1800],}); console.log("Stock price data:");console.log(stockPrices.toString()); // Rolling mean with window size 3const rollingMean = stockPrices.rolling(3).mean();console.log("\nRolling mean (window=3):");console.log(rollingMean.toString()); // Rolling standard deviationconst rollingStd = stockPrices.rolling(3).std();console.log("Rolling std (window=3):");console.log(rollingStd.toString()); // Rolling sumconst rollingSum = stockPrices.rolling(3).sum();console.log("Rolling sum (window=3):");console.log(rollingSum.toString()); // Rolling min/maxconst rollingMin = stockPrices.rolling(3).min();console.log("Rolling min (window=3):");console.log(rollingMin.toString()); const rollingMax = stockPrices.rolling(3).max();console.log("Rolling max (window=3):");console.log(rollingMax.toString()); // By default the first window - 1 rows are null. minPeriods lets a window report// once it holds that many values, and center aligns the window on the current row.const earlyMean = stockPrices.rolling(3, { minPeriods: 1 }).mean();console.log("Rolling mean (window=3, minPeriods=1), first 3 rows:");console.log(earlyMean.head(3).toString()); const centeredMean = stockPrices.rolling(3, { center: true }).mean();console.log("Rolling mean (window=3, center=true), first 3 rows:");console.log(centeredMean.head(3).toString()); // ============================================================================// Part 4: Expanding Window Calculations// ============================================================================console.log("\nPart 4: Expanding Window Calculations");console.log("-".repeat(60)); // Expanding windows compute cumulative statistics from the startconst values = new DataFrame({  revenue: [10, 20, 15, 25, 30, 18, 35, 40],}); const expandingMean = values.expanding().mean();console.log("Expanding mean (cumulative average):");console.log(expandingMean.toString()); const expandingSum = values.expanding().sum();console.log("Expanding sum (cumulative sum):");console.log(expandingSum.toString()); const expandingStd = values.expanding().std();console.log("Expanding std (cumulative std):");console.log(expandingStd.toString()); // ============================================================================// Part 5: Exponentially Weighted Moving (EWM) Calculations// ============================================================================console.log("\nPart 5: Exponentially Weighted Moving (EWM)");console.log("-".repeat(60)); // EWM gives more weight to recent observations.// Deepbox defaults to adjust=false and bias=true. pandas defaults to adjust=true and// bias=false, so pass { adjust: true, bias: false } to match pandas.const temperatures = new DataFrame({  temp: [20, 22, 21, 25, 24, 23, 26, 28, 27, 30],}); // Using span parameter (alpha = 2 / (span + 1))const ewmMean = temperatures.ewm({ span: 3 }).mean();console.log("EWM mean (span=3):");console.log(ewmMean.toString()); // Using direct alpha parameterconst ewmMeanAlpha = temperatures.ewm({ alpha: 0.3 }).mean();console.log("EWM mean (alpha=0.3):");console.log(ewmMeanAlpha.toString()); // EWM standard deviationconst ewmStd = temperatures.ewm({ span: 3 }).std();console.log("EWM std (span=3):");console.log(ewmStd.toString()); // pandas-compatible settingsconst ewmPandas = temperatures.ewm({ span: 3, adjust: true, bias: false }).mean();console.log("EWM mean (span=3, adjust=true, bias=false):");console.log(ewmPandas.toString()); // ============================================================================// Part 6: Query Expressions// ============================================================================console.log("\nPart 6: Query Expressions");console.log("-".repeat(60)); // Query filters rows using string expressionsconst employees = new DataFrame({  name: ["Alice", "Bob", "Charlie", "Diana", "Eve", "Frank"],  department: ["Engineering", "Sales", "Engineering", "Marketing", "Sales", "Engineering"],  salary: [95000, 65000, 88000, 72000, 61000, 105000],  experience: [5, 3, 4, 6, 2, 8],}); console.log("Employee data:");console.log(employees.toString()); // Simple queryconst highEarners = employees.query("salary > 80000");console.log("\nquery('salary > 80000'):");console.log(highEarners.toString()); // Compound query with ANDconst seniorHighEarners = employees.query("salary > 70000 and experience > 4");console.log("query('salary > 70000 and experience > 4'):");console.log(seniorHighEarners.toString()); // Query with ORconst salesOrMarketing = employees.query("department == Sales or department == Marketing");console.log("query('department == Sales or department == Marketing'):");console.log(salesOrMarketing.toString()); // ============================================================================// Part 7: Eval Expressions// ============================================================================console.log("\nPart 7: Eval Expressions");console.log("-".repeat(60)); // Eval creates computed columns or filters using expressionsconst metrics = new DataFrame({  a: [1, 2, 3, 4, 5],  b: [10, 20, 30, 40, 50],}); console.log("Original:");console.log(metrics.toString()); // Create a new column with evalconst withSum = metrics.eval("c = a + b");console.log("\neval('c = a + b'):");console.log(withSum.toString()); // Multiplicationconst withProduct = metrics.eval("product = a * b");console.log("eval('product = a * b'):");console.log(withProduct.toString()); // Filter with evalconst filtered = metrics.eval("a > 2");console.log("eval('a > 2') filters rows:");console.log(filtered.toString()); // ============================================================================// Part 8: Assign (Functional Column Creation)// ============================================================================console.log("\nPart 8: Assign");console.log("-".repeat(60)); // Assign creates new columns from arrays or functionsconst sales = new DataFrame({  product: ["Widget", "Gadget", "Doohickey", "Thingamajig"],  price: [10, 25, 15, 30],  quantity: [100, 50, 80, 30],}); console.log("Original sales:");console.log(sales.toString()); // Assign with a functionconst withRevenue = sales.assign({  revenue: (row: Record<string, unknown>) => {    const price = Number(row["price"] ?? 0);    const qty = Number(row["quantity"] ?? 0);    return price * qty;  },  discount: [0.1, 0.05, 0.15, 0.0],}); console.log("\nWith assigned columns (revenue, discount):");console.log(withRevenue.toString()); // ============================================================================// Part 9: Pivot Table// ============================================================================console.log("\nPart 9: Pivot Table");console.log("-".repeat(60)); // Pivot tables reshape data for cross-tabulation analysisconst salesData = new DataFrame({  region: ["North", "South", "North", "South", "North", "South", "North", "South"],  product: ["A", "A", "B", "B", "A", "A", "B", "B"],  revenue: [100, 150, 200, 120, 130, 160, 180, 140],}); console.log("Sales data:");console.log(salesData.toString()); // Pivot: regions as rows, products as columns, mean revenue as valuesconst pivoted = salesData.pivotTable({  index: "region",  columns: "product",  values: "revenue",  aggFunc: "mean",}); console.log("\nPivot table (mean revenue by region and product):");console.log(pivoted.toString()); // Pivot with sum aggregationconst pivotedSum = salesData.pivotTable({  index: "region",  columns: "product",  values: "revenue",  aggFunc: "sum",}); console.log("Pivot table (sum revenue by region and product):");console.log(pivotedSum.toString()); // ============================================================================// Part 10: Crosstab// ============================================================================console.log("\nPart 10: Crosstab");console.log("-".repeat(60)); // Crosstab computes frequency tables between two categorical columnsconst survey = new DataFrame({  gender: ["M", "F", "M", "F", "M", "F", "M", "F", "M", "F"],  preference: ["A", "B", "A", "A", "B", "B", "A", "A", "B", "B"],}); console.log("Survey data:");console.log(survey.toString()); const crossResult = survey.crosstab("gender", "preference");console.log("\nCrosstab (gender and preference):");console.log(crossResult.toString()); // ============================================================================// Part 11: nlargest / nsmallest// ============================================================================console.log("\nPart 11: nlargest / nsmallest");console.log("-".repeat(60)); const scores = new DataFrame({  student: ["Alice", "Bob", "Charlie", "Diana", "Eve", "Frank", "Grace"],  score: [92, 85, 78, 95, 88, 73, 91],  grade: ["A", "B", "C", "A", "B", "C", "A"],}); console.log("Student scores:");console.log(scores.toString()); const top3 = scores.nlargest(3, "score");console.log("\nTop 3 scores:");console.log(top3.toString()); const bottom3 = scores.nsmallest(3, "score");console.log("Bottom 3 scores:");console.log(bottom3.toString()); // ============================================================================// Part 12: Interpolation// ============================================================================console.log("\nPart 12: Interpolation");console.log("-".repeat(60)); // interpolate estimates missing values from their neighbors (linear or nearest).// ffill, bfill and fillna replace them with a copied or fixed value instead.const sensorData = new DataFrame({  time: [0, 1, 2, 3, 4, 5, 6, 7],  reading: [10, null, null, 25, 30, null, 42, 50],}); console.log("Sensor data with gaps:");console.log(sensorData.toString()); const linearInterp = sensorData.interpolate("linear");console.log("\nLinear interpolation:");console.log(linearInterp.toString()); const nearestInterp = sensorData.interpolate("nearest");console.log("Nearest interpolation:");console.log(nearestInterp.toString()); // ffill copies the last valid value forward, bfill copies the next valid value backwardconsole.log("Forward fill (ffill):");console.log(sensorData.ffill().toString()); console.log("Backward fill (bfill):");console.log(sensorData.bfill().toString()); // fillna takes a value per column, or a fill methodconsole.log("fillna({ reading: 0 }):");console.log(sensorData.fillna({ reading: 0 }).toString()); // ============================================================================// Summary// ============================================================================console.log("\nKey Takeaways");console.log("-".repeat(60));console.log(  "• .str accessor: string operations on every element (upper, lower, contains, split, ...)");console.log("• .dt accessor: date parts (year, month, day, hour, dayOfWeek, quarter)");console.log(  "• rolling(n): sliding window stats (mean, sum, std, var, min, max), with minPeriods and center");console.log("• expanding(): cumulative stats from the start of the data");console.log(  "• ewm({ span }): exponentially weighted moving statistics (adjust=false, bias=true by default)");console.log("• query(): filter rows with an expression string");console.log("• eval(): computed columns and expression-based filtering");console.log("• assign(): functional column creation with arrays or functions");console.log("• pivotTable(): aggregate values into a rows-by-columns table");console.log("• crosstab(): counts for each pair of categories");console.log("• nlargest/nsmallest: top/bottom N rows by column");console.log("• interpolate(), ffill(), bfill(), fillna(): fill missing values"); console.log("\nAdvanced DataFrame Features Example Complete!");console.log("=".repeat(60));

Output

Console output
Console output only: each part prints its input and its result.
`rolling` accepts `minPeriods` and `center`.
`ewm` defaults to `adjust: false` and `bias: true`, which differs from pandas. Pass `{ adjust: true, bias: false }` to match pandas.
The older names `pivot_table` and `dayofweek` still work. Use `pivotTable` and `dayOfWeek`.