Tree-Based & Ensemble Models

Tree-based models are among the most powerful and interpretable ML algorithms. This example trains DecisionTreeClassifier/Regressor, RandomForestClassifier/Regressor, GradientBoostingClassifier/Regressor, and LinearSVC/LinearSVR. Each model is evaluated on the Iris or Diabetes dataset with appropriate metrics.

Deepbox Modules Used

deepbox/datasetsdeepbox/mldeepbox/ndarraydeepbox/metricsdeepbox/preprocess

What You Will Learn

Decision Trees split features recursively — control depth to prevent overfitting
Random Forests bag many randomized trees for variance reduction
Gradient Boosting builds trees sequentially, each correcting the previous
LinearSVC finds the maximum-margin hyperplane for classification

Source Code

11-tree-ensemble-models/index.ts

1import { loadIris } from "deepbox/datasets";2import { accuracy, mse, r2Score } from "deepbox/metrics";3import {4  DecisionTreeClassifier,5  DecisionTreeRegressor,6  GradientBoostingClassifier,7  GradientBoostingRegressor,8  LinearSVC,9  LinearSVR,10  RandomForestClassifier,11  RandomForestRegressor,12} from "deepbox/ml";13import { slice, tensor } from "deepbox/ndarray";14import { trainTestSplit } from "deepbox/preprocess";1516console.log("=== Tree-Based & Ensemble Models ===\n");1718// ---------------------------------------------------------------------------19// Classification dataset (Iris — full 3-class for multi-class models)20// ---------------------------------------------------------------------------21const iris = loadIris();22const [XTrain, XTest, yTrain, yTest] = trainTestSplit(iris.data, iris.target, {23  testSize: 0.2,24  randomState: 42,25});2627// Binary subset (classes 0 and 1 only) for models that require binary labels28const XBin = slice(iris.data, { start: 0, end: 100 });29const yBin = slice(iris.target, { start: 0, end: 100 });30const [XBinTrain, XBinTest, yBinTrain, yBinTest] = trainTestSplit(XBin, yBin, {31  testSize: 0.2,32  randomState: 42,33});3435// ---------------------------------------------------------------------------36// Part 1: Decision Tree Classifier37// ---------------------------------------------------------------------------38console.log("--- Part 1: Decision Tree Classifier ---");3940const dtc = new DecisionTreeClassifier({ maxDepth: 5, minSamplesSplit: 2 });41dtc.fit(XTrain, yTrain);42const dtcPred = dtc.predict(XTest);43console.log("  Accuracy:", accuracy(yTest, dtcPred).toFixed(4));4445// ---------------------------------------------------------------------------46// Part 2: Random Forest Classifier47// ---------------------------------------------------------------------------48console.log("\n--- Part 2: Random Forest Classifier ---");4950const rfc = new RandomForestClassifier({51  nEstimators: 50,52  maxDepth: 5,53  randomState: 42,54});55rfc.fit(XTrain, yTrain);56const rfcPred = rfc.predict(XTest);57console.log("  Accuracy:", accuracy(yTest, rfcPred).toFixed(4));5859// ---------------------------------------------------------------------------60// Part 3: Gradient Boosting Classifier61// ---------------------------------------------------------------------------62console.log("\n--- Part 3: Gradient Boosting Classifier ---");6364const gbc = new GradientBoostingClassifier({65  nEstimators: 50,66  learningRate: 0.1,67  maxDepth: 3,68});69gbc.fit(XBinTrain, yBinTrain);70const gbcPred = gbc.predict(XBinTest);71console.log("  Accuracy:", accuracy(yBinTest, gbcPred).toFixed(4));7273// ---------------------------------------------------------------------------74// Part 4: Linear SVC75// ---------------------------------------------------------------------------76console.log("\n--- Part 4: Linear SVC ---");7778const svc = new LinearSVC({ C: 1.0 });79svc.fit(XBinTrain, yBinTrain);80const svcPred = svc.predict(XBinTest);81console.log("  Accuracy:", accuracy(yBinTest, svcPred).toFixed(4));8283// ---------------------------------------------------------------------------84// Regression dataset (synthetic y = x0 + 2*x1 + noise)85// ---------------------------------------------------------------------------86console.log("\n--- Regression Models ---");8788const XReg = tensor([89  [1, 2],90  [2, 3],91  [3, 4],92  [4, 5],93  [5, 6],94  [6, 7],95  [7, 8],96  [8, 9],97  [9, 10],98  [10, 11],99  [1, 3],100  [2, 5],101  [3, 2],102  [4, 1],103  [5, 4],104  [6, 3],105  [7, 6],106  [8, 5],107  [9, 8],108  [10, 7],109]);110const yReg = tensor([5, 8, 11, 14, 17, 20, 23, 26, 29, 32, 7, 12, 7, 6, 13, 12, 19, 18, 25, 24]);111112const [XRegTrain, XRegTest, yRegTrain, yRegTest] = trainTestSplit(XReg, yReg, {113  testSize: 0.2,114  randomState: 42,115});116117// ---------------------------------------------------------------------------118// Part 5: Decision Tree Regressor119// ---------------------------------------------------------------------------120console.log("\n--- Part 5: Decision Tree Regressor ---");121122const dtr = new DecisionTreeRegressor({ maxDepth: 5 });123dtr.fit(XRegTrain, yRegTrain);124const dtrPred = dtr.predict(XRegTest);125console.log("  MSE:", mse(yRegTest, dtrPred).toFixed(4));126console.log("  R²: ", r2Score(yRegTest, dtrPred).toFixed(4));127128// ---------------------------------------------------------------------------129// Part 6: Random Forest Regressor130// ---------------------------------------------------------------------------131console.log("\n--- Part 6: Random Forest Regressor ---");132133const rfr = new RandomForestRegressor({134  nEstimators: 50,135  maxDepth: 5,136  randomState: 42,137});138rfr.fit(XRegTrain, yRegTrain);139const rfrPred = rfr.predict(XRegTest);140console.log("  MSE:", mse(yRegTest, rfrPred).toFixed(4));141console.log("  R²: ", r2Score(yRegTest, rfrPred).toFixed(4));142143// ---------------------------------------------------------------------------144// Part 7: Gradient Boosting Regressor145// ---------------------------------------------------------------------------146console.log("\n--- Part 7: Gradient Boosting Regressor ---");147148const gbr = new GradientBoostingRegressor({149  nEstimators: 50,150  learningRate: 0.1,151  maxDepth: 3,152});153gbr.fit(XRegTrain, yRegTrain);154const gbrPred = gbr.predict(XRegTest);155console.log("  MSE:", mse(yRegTest, gbrPred).toFixed(4));156console.log("  R²: ", r2Score(yRegTest, gbrPred).toFixed(4));157158// ---------------------------------------------------------------------------159// Part 8: Linear SVR160// ---------------------------------------------------------------------------161console.log("\n--- Part 8: Linear SVR ---");162163const svr = new LinearSVR({ C: 1.0 });164svr.fit(XRegTrain, yRegTrain);165const svrPred = svr.predict(XRegTest);166console.log("  MSE:", mse(yRegTest, svrPred).toFixed(4));167console.log("  R²: ", r2Score(yRegTest, svrPred).toFixed(4));168169console.log("\n=== Tree-Based & Ensemble Models Complete ===");

Console Output

$ npx tsx 11-tree-ensemble-models/index.ts

=== Tree-Based & Ensemble Models ===

--- Part 1: Decision Tree Classifier ---
  Accuracy: 1.0000

--- Part 2: Random Forest Classifier ---
  Accuracy: 1.0000

--- Part 3: Gradient Boosting Classifier ---
  Accuracy: 1.0000

--- Part 4: Linear SVC ---
  Accuracy: 1.0000

--- Regression Models ---

--- Part 5: Decision Tree Regressor ---
  MSE: 7.5000
  R²:  0.8590

--- Part 6: Random Forest Regressor ---
  MSE: 5.0926
  R²:  0.9043

--- Part 7: Gradient Boosting Regressor ---
  MSE: 4.0332
  R²:  0.9242

--- Part 8: Linear SVR ---
  MSE: 0.9344
  R²:  0.9824

=== Tree-Based & Ensemble Models Complete ===

Advanced ML ModelsPrevious Complete ML PipelineNext