![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "3Ncn0J2aX3CN"
+ },
+ "source": [
+ "\n"
+ ],
+ "id": "3Ncn0J2aX3CN"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "B8jHkzm_X3CO"
+ },
+ "source": [
+ "# xgboost2ww Covertype Hyperparameter Sweep (Underfit → Tuned → Mild Overfit)\n",
+ "\n",
+ "## What this notebook does\n",
+ "- **Dataset**: UCI/Kaggle-style **Forest CoverType** via `sklearn.datasets.fetch_covtype` (581,012 rows, 54 features, 7 classes). Multiclass accuracy is expected to be lower than 90%+ binary examples.\n",
+ "- **Multiclass objective**: XGBoost `objective='multi:softprob'` with `num_class=7`, evaluating `mlogloss` and `merror`.\n",
+ "- **Split protocol**: deterministic stratified 80/20 train-test split (`RNG=123`) plus a stratified validation split from train for early stopping.\n",
+ "- **Runtime controls**: configurable `SAMPLE_N`, `NFOLDS_SWEEP`, `T_POINTS_SWEEP` and optional heavier best-model verification settings so Colab runs remain practical.\n",
+ "- **xgboost2ww multiclass mode**: main sweep uses `multiclass='stack'` with `return_type='torch'` for a single WeightWatcher-friendly matrix; optional representative configs use `multiclass='per_class'` with `return_type='numpy'` because per-class + torch is unsupported.\n",
+ "- **WeightWatcher metrics**: `alpha`, `rand_num_spikes` (all configs, always `randomize=True`), and optional `detX` for representative configs only.\n",
+ "- **Sweep strategy**: Stage A regime sweep (underfit/tuned/overfit seeds + perturbations), Stage B local refinement around top configs.\n",
+ "- **Expected qualitative pattern**: as in Adult/Spambase, better-generalizing models often have healthier spectral structure (alpha nearer 2, fewer traps), while underfit tends to larger alpha and overfit/unstable regimes often show alpha < 2 and/or more traps.\n",
+ "- **Reproducibility and persistence**: fixed seeds, explicit requested vs achieved boosting rounds, checkpointed CSV/plots under a Covertype-specific folder.\n",
+ "\n",
+ "> We expect to observe the same **qualitative** alpha/performance pattern as binary notebooks, but absolute accuracy values differ because this is a larger 7-class multiclass task.\n"
+ ],
+ "id": "B8jHkzm_X3CO"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9eskbjYkX3CO"
+ },
+ "source": [
+ "## Google Drive checkpoint locations (duplicated here for run notes)\n",
+ "- **Checkpoint root path:** `/content/drive/MyDrive/xgboost2ww_checkpoints`\n",
+ "- **This run folder path:** `/content/drive/MyDrive/xgboost2ww_checkpoints/covertype_hparam_sweep`\n",
+ "- **Results CSV path:** `/content/drive/MyDrive/xgboost2ww_checkpoints/covertype_hparam_sweep/covertype_hyperparameter_sweep_results.csv`\n",
+ "\n",
+ "When running outside Colab, the notebook uses `./checkpoints/covertype_hparam_sweep/` as the local fallback.\n"
+ ],
+ "id": "9eskbjYkX3CO"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "hYMhlRoGX3CO",
+ "outputId": "533868ed-a26f-467a-dbf9-5a3fded2dca0",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "view-in-github",
- "colab_type": "text"
- },
- "source": [
- ""
- ]
- },
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n",
+ "IN_COLAB: True\n",
+ "CHECKPOINT_DIR: /content/drive/MyDrive/xgboost2ww_checkpoints/covertype_hparam_sweep\n",
+ "CHECKPOINT_RESULTS_CSV: /content/drive/MyDrive/xgboost2ww_checkpoints/covertype_hparam_sweep/covertype_hyperparameter_sweep_results.csv\n"
+ ]
+ }
+ ],
+ "source": [
+ "from pathlib import Path\n",
+ "import os\n",
+ "\n",
+ "IN_COLAB = False\n",
+ "try:\n",
+ " from google.colab import drive\n",
+ " IN_COLAB = True\n",
+ "except Exception:\n",
+ " IN_COLAB = False\n",
+ "\n",
+ "CHECKPOINT_ROOT_COLAB = Path('/content/drive/MyDrive/xgboost2ww_checkpoints')\n",
+ "CHECKPOINT_ROOT_LOCAL = Path('./checkpoints')\n",
+ "EXPERIMENT_NAME = 'covertype_hparam_sweep'\n",
+ "OUTPUT_FILENAME = 'covertype_hyperparameter_sweep_results.csv'\n",
+ "\n",
+ "if IN_COLAB:\n",
+ " drive.mount('/content/drive')\n",
+ " CHECKPOINT_ROOT = CHECKPOINT_ROOT_COLAB\n",
+ "else:\n",
+ " CHECKPOINT_ROOT = CHECKPOINT_ROOT_LOCAL\n",
+ "\n",
+ "CHECKPOINT_DIR = CHECKPOINT_ROOT / EXPERIMENT_NAME\n",
+ "CHECKPOINT_DIR.mkdir(parents=True, exist_ok=True)\n",
+ "CHECKPOINT_RESULTS_CSV = CHECKPOINT_DIR / OUTPUT_FILENAME\n",
+ "\n",
+ "print('IN_COLAB:', IN_COLAB)\n",
+ "print('CHECKPOINT_DIR:', CHECKPOINT_DIR)\n",
+ "print('CHECKPOINT_RESULTS_CSV:', CHECKPOINT_RESULTS_CSV)\n"
+ ],
+ "id": "hYMhlRoGX3CO"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "crKUqaFRX3CP"
+ },
+ "outputs": [],
+ "source": [
+ "# Optional install cell (useful in Colab)\n",
+ "!pip -q install xgboost weightwatcher xgboost2ww scikit-learn pandas matplotlib\n"
+ ],
+ "id": "crKUqaFRX3CP"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "UBEE7EP1X3CP"
+ },
+ "source": [
+ "## Imports and reproducibility controls\n"
+ ],
+ "id": "UBEE7EP1X3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "7O6hz-jUX3CP"
+ },
+ "outputs": [],
+ "source": [
+ "import warnings\n",
+ "warnings.filterwarnings('ignore')\n",
+ "\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "from sklearn.datasets import fetch_covtype, fetch_openml\n",
+ "from sklearn.model_selection import train_test_split, StratifiedShuffleSplit\n",
+ "from sklearn.metrics import accuracy_score, log_loss, balanced_accuracy_score, f1_score\n",
+ "\n",
+ "import xgboost as xgb\n",
+ "import weightwatcher as ww\n",
+ "from xgboost2ww import convert, to_linear_layer\n"
+ ],
+ "id": "7O6hz-jUX3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "d12_Yyg1X3CP"
+ },
+ "outputs": [],
+ "source": [
+ "RNG = 123\n",
+ "TEST_SIZE = 0.20\n",
+ "VAL_SIZE_WITHIN_TRAIN = 0.20\n",
+ "\n",
+ "# Runtime knobs (safe defaults for Colab)\n",
+ "USE_SAMPLING = False\n",
+ "SAMPLE_N = 120_000\n",
+ "SWEEP_SEEDS = [123, 231, 777]\n",
+ "NFOLDS_SWEEP = 3\n",
+ "NFOLDS_BEST = 5\n",
+ "T_POINTS_SWEEP = 32\n",
+ "T_POINTS_BEST = 64\n",
+ "RUN_EXPENSIVE_BEST_VERIFICATION = False\n",
+ "\n",
+ "MULTICLASS_MODE_MAIN = 'stack' # 'stack' preferred; fallback to 'avg' if memory is tight\n",
+ "W_MATRIX_NAME = 'W7'\n",
+ "\n",
+ "# Increase WeightWatcher matrix-size cap for large stacked multiclass matrices\n",
+ "WW_MAX_N = 800000\n"
+ ],
+ "id": "d12_Yyg1X3CP"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Drd0saLCX3CP"
+ },
+ "source": [
+ "## Load Covertype with deterministic fallback and run sanity checks\n"
+ ],
+ "id": "Drd0saLCX3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LtJfTue2X3CP"
+ },
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " data = fetch_covtype()\n",
+ " X, y = data.data, data.target\n",
+ " loader_used = 'fetch_covtype'\n",
+ "except Exception:\n",
+ " data = fetch_openml(name='covertype', version=3, as_frame=False)\n",
+ " X, y = data.data, data.target\n",
+ " loader_used = 'fetch_openml(covertype)'\n",
+ "\n",
+ "X = np.asarray(X, dtype=np.float32)\n",
+ "y = np.asarray(y)\n",
+ "if y.dtype.kind in {'U', 'S', 'O'}:\n",
+ " y = y.astype(int)\n",
+ "y = y.astype(np.int64)\n",
+ "if y.min() == 1:\n",
+ " y = y - 1\n",
+ "\n",
+ "n_classes = int(np.unique(y).shape[0])\n",
+ "print('loader:', loader_used)\n",
+ "print('X.shape:', X.shape, 'y.shape:', y.shape)\n",
+ "print('n_classes:', n_classes)\n",
+ "\n",
+ "class_counts = pd.Series(y).value_counts().sort_index()\n",
+ "print('counts per class:')\n",
+ "print(class_counts)\n",
+ "\n",
+ "if USE_SAMPLING and SAMPLE_N < len(y):\n",
+ " sss = StratifiedShuffleSplit(n_splits=1, train_size=SAMPLE_N, random_state=RNG)\n",
+ " idx, _ = next(sss.split(X, y))\n",
+ " X = X[idx]\n",
+ " y = y[idx]\n",
+ " print(f'Applied stratified sample: SAMPLE_N={SAMPLE_N}, new shape={X.shape}')\n"
+ ],
+ "id": "LtJfTue2X3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "K2oMbwTbX3CP"
+ },
+ "outputs": [],
+ "source": [
+ "def build_split(seed):\n",
+ " idx_train, idx_test = train_test_split(\n",
+ " np.arange(len(y)), test_size=TEST_SIZE, random_state=seed, stratify=y\n",
+ " )\n",
+ " X_train, X_test = X[idx_train], X[idx_test]\n",
+ " y_train, y_test = y[idx_train], y[idx_test]\n",
+ "\n",
+ " idx_tr, idx_val = train_test_split(\n",
+ " np.arange(len(y_train)), test_size=VAL_SIZE_WITHIN_TRAIN, random_state=seed, stratify=y_train\n",
+ " )\n",
+ " return {\n",
+ " 'seed': seed,\n",
+ " 'X_train': X_train,\n",
+ " 'X_test': X_test,\n",
+ " 'y_train': y_train,\n",
+ " 'y_test': y_test,\n",
+ " 'idx_tr': idx_tr,\n",
+ " 'idx_val': idx_val,\n",
+ " }\n",
+ "\n",
+ "base_split = build_split(RNG)\n",
+ "X_train, X_test = base_split['X_train'], base_split['X_test']\n",
+ "y_train, y_test = base_split['y_train'], base_split['y_test']\n",
+ "idx_tr, idx_val = base_split['idx_tr'], base_split['idx_val']\n",
+ "\n",
+ "print('Train/Test sizes:', X_train.shape, X_test.shape)\n",
+ "print('Train class counts:\\n', pd.Series(y_train).value_counts().sort_index().to_string())\n",
+ "print('Test class counts:\\n', pd.Series(y_test).value_counts().sort_index().to_string())\n",
+ "print('Inner train/val sizes:', len(idx_tr), len(idx_val))\n",
+ "print('Sweep seeds:', SWEEP_SEEDS)\n"
+ ],
+ "id": "K2oMbwTbX3CP"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "oafvwxynX3CP"
+ },
+ "source": [
+ "## Helpers: train/evaluate + xgboost2ww + WeightWatcher (graceful degradation)\n"
+ ],
+ "id": "oafvwxynX3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "n5rbUhgOX3CP"
+ },
+ "outputs": [],
+ "source": [
+ "def _run_weightwatcher_summary(row, booster, split_data, run_detx=False):\n",
+ " X_train_local, y_train_local = split_data['X_train'], split_data['y_train']\n",
+ " try:\n",
+ " model_ww = convert(booster, X_train_local, y_train_local, multiclass=MULTICLASS_MODE_MAIN, return_type='torch')\n",
+ " watcher = ww.WeightWatcher(model=model_ww, log_level='ERROR')\n",
+ " details = watcher.analyze(\n",
+ " randomize=True,\n",
+ " min_evals=10,\n",
+ " max_evals=200,\n",
+ " mp_fit=False,\n",
+ " plot=False,\n",
+ " savefig=False,\n",
+ " detX=run_detx,\n",
+ " max_N=WW_MAX_N,\n",
+ " )\n",
+ " details = pd.DataFrame(details)\n",
+ " if 'alpha' in details.columns:\n",
+ " row['alpha'] = float(details['alpha'].dropna().iloc[0]) if details['alpha'].notna().any() else np.nan\n",
+ " if 'rand_num_spikes' in details.columns:\n",
+ " row['rand_num_spikes'] = float(details['rand_num_spikes'].dropna().iloc[0]) if details['rand_num_spikes'].notna().any() else np.nan\n",
+ " if run_detx and 'detX_val' in details.columns and details['detX_val'].notna().any():\n",
+ " row['detX_val'] = float(details['detX_val'].dropna().iloc[0])\n",
+ " except Exception as e:\n",
+ " row['ww_note'] = f'skipped/failed: {type(e).__name__}: {e}'\n",
+ " try:\n",
+ " model_ww = convert(booster, X_train_local, y_train_local, multiclass='avg', return_type='torch')\n",
+ " watcher = ww.WeightWatcher(model=model_ww, log_level='ERROR')\n",
+ " details = pd.DataFrame(\n",
+ " watcher.analyze(randomize=True, min_evals=10, max_evals=120, mp_fit=False, plot=False, savefig=False, max_N=WW_MAX_N)\n",
+ " )\n",
+ " row['multiclass_mode'] = 'avg'\n",
+ " if 'alpha' in details.columns and details['alpha'].notna().any():\n",
+ " row['alpha'] = float(details['alpha'].dropna().iloc[0])\n",
+ " if 'rand_num_spikes' in details.columns and details['rand_num_spikes'].notna().any():\n",
+ " row['rand_num_spikes'] = float(details['rand_num_spikes'].dropna().iloc[0])\n",
+ " row['ww_note'] += ' | fallback avg mode applied'\n",
+ " except Exception as e2:\n",
+ " row['ww_note'] += f' | fallback failed: {type(e2).__name__}: {e2}'\n",
+ " return row\n",
+ "\n",
+ "\n",
+ "def fit_eval_config_once(tag, cfg, split_seed, run_ww=False, run_detx=False):\n",
+ " split_data = build_split(split_seed)\n",
+ " X_train_local, X_test_local = split_data['X_train'], split_data['X_test']\n",
+ " y_train_local, y_test_local = split_data['y_train'], split_data['y_test']\n",
+ " idx_tr_local, idx_val_local = split_data['idx_tr'], split_data['idx_val']\n",
+ "\n",
+ " params = dict(\n",
+ " objective='multi:softprob',\n",
+ " num_class=n_classes,\n",
+ " eval_metric=['mlogloss', 'merror'],\n",
+ " tree_method='hist',\n",
+ " seed=split_seed,\n",
+ " verbosity=0,\n",
+ " )\n",
+ " params.update(cfg)\n",
+ " num_boost_round = int(params.pop('num_boost_round', 2000))\n",
+ " early_stopping_rounds = params.pop('early_stopping_rounds', 100)\n",
+ "\n",
+ " dtr = xgb.DMatrix(X_train_local[idx_tr_local], label=y_train_local[idx_tr_local])\n",
+ " dva = xgb.DMatrix(X_train_local[idx_val_local], label=y_train_local[idx_val_local])\n",
+ " dtrain_full = xgb.DMatrix(X_train_local, label=y_train_local)\n",
+ " dtest = xgb.DMatrix(X_test_local, label=y_test_local)\n",
+ "\n",
+ " evals_result = {}\n",
+ " booster = xgb.train(\n",
+ " params=params,\n",
+ " dtrain=dtr,\n",
+ " num_boost_round=num_boost_round,\n",
+ " evals=[(dtr, 'tr'), (dva, 'va')],\n",
+ " early_stopping_rounds=early_stopping_rounds,\n",
+ " evals_result=evals_result,\n",
+ " verbose_eval=False,\n",
+ " )\n",
+ "\n",
+ " best_iteration = int(getattr(booster, 'best_iteration', num_boost_round - 1))\n",
+ " train_proba = booster.predict(dtrain_full, iteration_range=(0, best_iteration + 1))\n",
+ " test_proba = booster.predict(dtest, iteration_range=(0, best_iteration + 1))\n",
+ "\n",
+ " train_pred = np.argmax(train_proba, axis=1)\n",
+ " test_pred = np.argmax(test_proba, axis=1)\n",
+ "\n",
+ " row = {\n",
+ " 'tag': tag,\n",
+ " 'split_seed': split_seed,\n",
+ " 'num_boost_round_requested': num_boost_round,\n",
+ " 'best_iteration': best_iteration,\n",
+ " 'train_acc': accuracy_score(y_train_local, train_pred),\n",
+ " 'test_acc': accuracy_score(y_test_local, test_pred),\n",
+ " 'train_bal_acc': balanced_accuracy_score(y_train_local, train_pred),\n",
+ " 'test_bal_acc': balanced_accuracy_score(y_test_local, test_pred),\n",
+ " 'test_macro_f1': f1_score(y_test_local, test_pred, average='macro'),\n",
+ " 'train_mlogloss': log_loss(y_train_local, train_proba, labels=np.arange(n_classes)),\n",
+ " 'test_mlogloss': log_loss(y_test_local, test_proba, labels=np.arange(n_classes)),\n",
+ " 'multiclass_mode': MULTICLASS_MODE_MAIN,\n",
+ " 'alpha': np.nan,\n",
+ " 'rand_num_spikes': np.nan,\n",
+ " 'detX_val': np.nan,\n",
+ " 'ww_note': '',\n",
+ " }\n",
+ " for k, v in cfg.items():\n",
+ " row[f'hp_{k}'] = v\n",
+ "\n",
+ " if run_ww:\n",
+ " row = _run_weightwatcher_summary(row, booster, split_data, run_detx=run_detx)\n",
+ "\n",
+ " print(f\"[{tag}] seed={split_seed} requested={num_boost_round} best_iteration={best_iteration} train_acc={row['train_acc']:.4f} test_acc={row['test_acc']:.4f}\")\n",
+ " return row, booster\n",
+ "\n",
+ "\n",
+ "def fit_eval_config(tag, cfg, t_points=T_POINTS_SWEEP, run_detx=False):\n",
+ " per_seed_rows, per_seed_boosters = [], {}\n",
+ " for seed in SWEEP_SEEDS:\n",
+ " row, booster = fit_eval_config_once(tag, cfg, split_seed=seed, run_ww=False, run_detx=False)\n",
+ " per_seed_rows.append(row)\n",
+ " per_seed_boosters[seed] = booster\n",
+ "\n",
+ " per_seed_df = pd.DataFrame(per_seed_rows)\n",
+ " rep_seed = int(per_seed_df.loc[per_seed_df['test_mlogloss'].idxmin(), 'split_seed'])\n",
+ " rep_row = per_seed_df.loc[per_seed_df['split_seed'] == rep_seed].iloc[0].to_dict()\n",
+ "\n",
+ " for metric in ['train_acc', 'test_acc', 'train_bal_acc', 'test_bal_acc', 'test_macro_f1', 'train_mlogloss', 'test_mlogloss', 'best_iteration']:\n",
+ " rep_row[metric] = float(per_seed_df[metric].mean())\n",
+ " rep_row[f'{metric}_std'] = float(per_seed_df[metric].std(ddof=0))\n",
+ "\n",
+ " rep_row['rep_split_seed'] = rep_seed\n",
+ " rep_row['n_sweep_seeds'] = len(SWEEP_SEEDS)\n",
+ "\n",
+ " rep_booster = per_seed_boosters[rep_seed]\n",
+ " rep_row = _run_weightwatcher_summary(rep_row, rep_booster, build_split(rep_seed), run_detx=run_detx)\n",
+ "\n",
+ " print(f\"[{tag}] aggregated across seeds={SWEEP_SEEDS}: test_acc={rep_row['test_acc']:.4f}±{rep_row['test_acc_std']:.4f}, test_mlogloss={rep_row['test_mlogloss']:.4f}±{rep_row['test_mlogloss_std']:.4f}, rep_seed={rep_seed}\")\n",
+ " return rep_row, rep_booster\n"
+ ],
+ "id": "n5rbUhgOX3CP"
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "yLhrndzDX3CP"
+ },
+ "source": [
+ "## Debug cell: single-model convert + WeightWatcher deep diagnostics\n",
+ "\n",
+ "Use this when `alpha` is NaN or conversion/WW unexpectedly fails. It trains one selected config, runs `xgboost2ww.convert`, and then runs WeightWatcher with `plot=True` plus extra diagnostics."
+ ],
+ "id": "yLhrndzDX3CP"
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "id": "sY-Hq3GkX3CQ",
+ "outputId": "03e0f815-c5b0-44d3-988d-1b0fa0764f22",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
+ }
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "metadata": {
- "id": "3Ncn0J2aX3CN"
- },
- "source": [
- "\n"
- ],
- "id": "3Ncn0J2aX3CN"
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Debug config: U1 {'max_depth': 2, 'eta': 0.3, 'num_boost_round': 150, 'early_stopping_rounds': 40, 'min_child_weight': 20, 'gamma': 5, 'subsample': 0.6, 'colsample_bytree': 0.6, 'reg_lambda': 50, 'reg_alpha': 0}\n",
+ "best_iteration: 147 requested: 150\n",
+ "debug test_acc: 0.7439166666666667\n",
+ "debug test_mlogloss: 0.585806952159488\n",
+ "Converting with multiclass= stack return_type= torch\n"
+ ]
},
{
- "cell_type": "markdown",
- "metadata": {
- "id": "B8jHkzm_X3CO"
- },
- "source": [
- "# xgboost2ww Covertype Hyperparameter Sweep (Underfit → Tuned → Mild Overfit)\n",
- "\n",
- "## What this notebook does\n",
- "- **Dataset**: UCI/Kaggle-style **Forest CoverType** via `sklearn.datasets.fetch_covtype` (581,012 rows, 54 features, 7 classes). Multiclass accuracy is expected to be lower than 90%+ binary examples.\n",
- "- **Multiclass objective**: XGBoost `objective='multi:softprob'` with `num_class=7`, evaluating `mlogloss` and `merror`.\n",
- "- **Split protocol**: deterministic stratified 80/20 train-test split (`RNG=123`) plus a stratified validation split from train for early stopping.\n",
- "- **Runtime controls**: configurable `SAMPLE_N`, `NFOLDS_SWEEP`, `T_POINTS_SWEEP` and optional heavier best-model verification settings so Colab runs remain practical.\n",
- "- **xgboost2ww multiclass mode**: main sweep uses `multiclass='stack'` with `return_type='torch'` for a single WeightWatcher-friendly matrix; optional representative configs use `multiclass='per_class'` with `return_type='numpy'` because per-class + torch is unsupported.\n",
- "- **WeightWatcher metrics**: `alpha`, `rand_num_spikes` (all configs, always `randomize=True`), and optional `detX` for representative configs only.\n",
- "- **Sweep strategy**: Stage A regime sweep (underfit/tuned/overfit seeds + perturbations), Stage B local refinement around top configs.\n",
- "- **Expected qualitative pattern**: as in Adult/Spambase, better-generalizing models often have healthier spectral structure (alpha nearer 2, fewer traps), while underfit tends to larger alpha and overfit/unstable regimes often show alpha < 2 and/or more traps.\n",
- "- **Reproducibility and persistence**: fixed seeds, explicit requested vs achieved boosting rounds, checkpointed CSV/plots under a Covertype-specific folder.\n",
- "\n",
- "> We expect to observe the same **qualitative** alpha/performance pattern as binary notebooks, but absolute accuracy values differ because this is a larger 7-class multiclass task.\n"
- ],
- "id": "B8jHkzm_X3CO"
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "INFO:weightwatcher:\n",
+ "\n",
+ "python version 3.12.12 (main, Oct 10 2025, 08:52:57) [GCC 11.4.0]\n",
+ "numpy version 2.0.2\n",
+ "torch version 2.10.0+cu128\n",
+ "INFO:weightwatcher:SET MODEL: None, None\n",
+ "INFO:weightwatcher:SET MODEL: Sequential(\n",
+ " (0): Linear(in_features=150, out_features=672000, bias=False)\n",
+ "), None\n",
+ "INFO:weightwatcher:params {'glorot_fix': False, 'normalize': False, 'conv2d_norm': True, 'randomize': True, 'savedir': 'ww-img', 'savefig': False, 'rescale': True, 'plot': True, 'deltaEs': False, 'intra': False, 'channels': None, 'fft': False, 'conv2d_fft': False, 'ww2x': False, 'pool': True, 'vectors': False, 'smooth': None, 'stacked': False, 'svd_method': 'fast', 'fix_fingers': False, 'fit': 'power_law', 'MAX_FINGERS': 10, 'finger_thresh': 1.0, 'sparsify': True, 'detX': True, 'mp_fit': False, 'min_evals': 10, 'max_evals': 200, 'max_N': 800000, 'tolerance': 1e-06, 'layer_ids_start': 0, 'add_biases': False, 'xmin_max': None, 'pl_package': 'ww', 'xmax': None, 'peft': False, 'inverse': False, 'glorot_fit': False, 'layers': []}\n",
+ "INFO:weightwatcher:Using | \n", - " | layer_id | \n", - "name | \n", - "N | \n", - "M | \n", - "alpha | \n", - "rand_num_spikes | \n", - "detX_val | \n", - "status | \n", - "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", - "1 | \n", - "0 | \n", - "672000 | \n", - "150 | \n", - "2.664018 | \n", - "0 | \n", - "167.990234 | \n", - "success | \n", - "
| \n", - " | class_id | \n", - "rows | \n", - "alpha_non_nan | \n", - "
|---|---|---|---|
| 0 | \n", - "0 | \n", - "0 | \n", - "0 | \n", - "
| 1 | \n", - "1 | \n", - "0 | \n", - "0 | \n", - "
| 2 | \n", - "2 | \n", - "0 | \n", - "0 | \n", - "
| 3 | \n", - "3 | \n", - "0 | \n", - "0 | \n", - "
| 4 | \n", - "4 | \n", - "0 | \n", - "0 | \n", - "
| 5 | \n", - "5 | \n", - "0 | \n", - "0 | \n", - "
| 6 | \n", - "6 | \n", - "0 | \n", - "0 | \n", - "
| \n", + " | layer_id | \n", + "name | \n", + "N | \n", + "M | \n", + "alpha | \n", + "rand_num_spikes | \n", + "detX_val | \n", + "status | \n", + "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "1 | \n", + "0 | \n", + "672000 | \n", + "150 | \n", + "2.664018 | \n", + "0 | \n", + "167.990234 | \n", + "success | \n", + "
| \n", - " | tag | \n", - "num_boost_round_requested | \n", - "best_iteration | \n", - "train_acc | \n", - "test_acc | \n", - "train_bal_acc | \n", - "test_bal_acc | \n", - "test_macro_f1 | \n", - "train_mlogloss | \n", - "test_mlogloss | \n", - "... | \n", - "hp_max_depth | \n", - "hp_eta | \n", - "hp_num_boost_round | \n", - "hp_early_stopping_rounds | \n", - "hp_min_child_weight | \n", - "hp_gamma | \n", - "hp_subsample | \n", - "hp_colsample_bytree | \n", - "hp_reg_lambda | \n", - "hp_reg_alpha | \n", - "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", - "BEST | \n", - "6000 | \n", - "1304 | \n", - "0.985479 | \n", - "0.927125 | \n", - "0.971639 | \n", - "0.870948 | \n", - "0.890698 | \n", - "0.063135 | \n", - "0.192197 | \n", - "... | \n", - "10 | \n", - "0.05 | \n", - "6000 | \n", - "150 | \n", - "1 | \n", - "0 | \n", - "0.9 | \n", - "0.9 | \n", - "5 | \n", - "0.5 | \n", - "
1 rows × 25 columns
\n", - "| \n", - " | tag | \n", - "test_acc | \n", - "train_acc | \n", - "alpha | \n", - "rand_num_spikes | \n", - "best_iteration | \n", - "num_boost_round_requested | \n", - "ww_note | \n", - "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", - "O1 | \n", - "0.932083 | \n", - "0.986667 | \n", - "2.415733 | \n", - "0.0 | \n", - "347 | \n", - "8000 | \n", - "\n", - " |
| 1 | \n", - "O3 | \n", - "0.931750 | \n", - "0.986354 | \n", - "1.708937 | \n", - "3.0 | \n", - "374 | \n", - "7000 | \n", - "\n", - " |
| 2 | \n", - "O2 | \n", - "0.930583 | \n", - "0.986219 | \n", - "2.389655 | \n", - "34.0 | \n", - "271 | \n", - "6000 | \n", - "\n", - " |
| 3 | \n", - "BEST | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "1304 | \n", - "6000 | \n", - "\n", - " |
| 4 | \n", - "T2 | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "1304 | \n", - "6000 | \n", - "\n", - " |
| 5 | \n", - "T1 | \n", - "0.921375 | \n", - "0.984229 | \n", - "1.827774 | \n", - "0.0 | \n", - "1751 | \n", - "4000 | \n", - "\n", - " |
| 6 | \n", - "T3 | \n", - "0.868708 | \n", - "0.900073 | \n", - "1.670708 | \n", - "0.0 | \n", - "1869 | \n", - "3000 | \n", - "\n", - " |
| 7 | \n", - "U2 | \n", - "0.781167 | \n", - "0.794625 | \n", - "2.717651 | \n", - "0.0 | \n", - "296 | \n", - "300 | \n", - "\n", - " |
| 8 | \n", - "U3 | \n", - "0.748125 | \n", - "0.753708 | \n", - "1.927702 | \n", - "0.0 | \n", - "237 | \n", - "250 | \n", - "\n", - " |
| 9 | \n", - "U1 | \n", - "0.743917 | \n", - "0.751885 | \n", - "2.664018 | \n", - "0.0 | \n", - "147 | \n", - "150 | \n", - "\n", - " |
| \n", + " | class_id | \n", + "rows | \n", + "alpha_non_nan | \n", + "
|---|---|---|---|
| 0 | \n", + "0 | \n", + "0 | \n", + "0 | \n", + "
| 1 | \n", + "1 | \n", + "0 | \n", + "0 | \n", + "
| 2 | \n", + "2 | \n", + "0 | \n", + "0 | \n", + "
| 3 | \n", + "3 | \n", + "0 | \n", + "0 | \n", + "
| 4 | \n", + "4 | \n", + "0 | \n", + "0 | \n", + "
| 5 | \n", + "5 | \n", + "0 | \n", + "0 | \n", + "
| 6 | \n", + "6 | \n", + "0 | \n", + "0 | \n", + "
| \n", - " | tag | \n", - "test_acc | \n", - "train_acc | \n", - "alpha | \n", - "rand_num_spikes | \n", - "multiclass_mode | \n", - "ww_note | \n", - "
|---|---|---|---|---|---|---|---|
| 0 | \n", - "R11_O1 | \n", - "0.932625 | \n", - "0.986656 | \n", - "1.741002 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 1 | \n", - "O1 | \n", - "0.932083 | \n", - "0.986667 | \n", - "2.415733 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 2 | \n", - "O3 | \n", - "0.931750 | \n", - "0.986354 | \n", - "1.708937 | \n", - "3.0 | \n", - "stack | \n", - "\n", - " |
| 3 | \n", - "R22_O3 | \n", - "0.931375 | \n", - "0.986531 | \n", - "2.459738 | \n", - "9.0 | \n", - "stack | \n", - "\n", - " |
| 4 | \n", - "R21_O3 | \n", - "0.931292 | \n", - "0.986302 | \n", - "2.443492 | \n", - "54.0 | \n", - "stack | \n", - "\n", - " |
| 5 | \n", - "R12_O1 | \n", - "0.930792 | \n", - "0.986615 | \n", - "1.698639 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 6 | \n", - "R31_O2 | \n", - "0.930667 | \n", - "0.986146 | \n", - "2.603029 | \n", - "43.0 | \n", - "stack | \n", - "\n", - " |
| 7 | \n", - "O2 | \n", - "0.930583 | \n", - "0.986219 | \n", - "2.389655 | \n", - "34.0 | \n", - "stack | \n", - "\n", - " |
| 8 | \n", - "R32_O2 | \n", - "0.930583 | \n", - "0.986219 | \n", - "2.389655 | \n", - "37.0 | \n", - "stack | \n", - "\n", - " |
| 9 | \n", - "BEST | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 10 | \n", - "T2 | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 11 | \n", - "T1 | \n", - "0.921375 | \n", - "0.984229 | \n", - "1.827774 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 12 | \n", - "T3 | \n", - "0.868708 | \n", - "0.900073 | \n", - "1.670708 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 13 | \n", - "U2 | \n", - "0.781167 | \n", - "0.794625 | \n", - "2.717651 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 14 | \n", - "U3 | \n", - "0.748125 | \n", - "0.753708 | \n", - "1.927702 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 15 | \n", - "U1 | \n", - "0.743917 | \n", - "0.751885 | \n", - "2.664018 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| \n", + " | tag | \n", + "num_boost_round_requested | \n", + "best_iteration | \n", + "train_acc | \n", + "test_acc | \n", + "train_bal_acc | \n", + "test_bal_acc | \n", + "test_macro_f1 | \n", + "train_mlogloss | \n", + "test_mlogloss | \n", + "... | \n", + "hp_max_depth | \n", + "hp_eta | \n", + "hp_num_boost_round | \n", + "hp_early_stopping_rounds | \n", + "hp_min_child_weight | \n", + "hp_gamma | \n", + "hp_subsample | \n", + "hp_colsample_bytree | \n", + "hp_reg_lambda | \n", + "hp_reg_alpha | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "BEST | \n", + "6000 | \n", + "1304 | \n", + "0.985479 | \n", + "0.927125 | \n", + "0.971639 | \n", + "0.870948 | \n", + "0.890698 | \n", + "0.063135 | \n", + "0.192197 | \n", + "... | \n", + "10 | \n", + "0.05 | \n", + "6000 | \n", + "150 | \n", + "1 | \n", + "0 | \n", + "0.9 | \n", + "0.9 | \n", + "5 | \n", + "0.5 | \n", + "
1 rows × 25 columns
\n", + "| \n", - " | tag | \n", - "alpha_mean_per_class | \n", - "alpha_median_per_class | \n", - "layers_analyzed | \n", - "
|---|---|---|---|---|
| 0 | \n", - "U1 | \n", - "NaN | \n", - "NaN | \n", - "0 | \n", - "
| 1 | \n", - "T2 | \n", - "NaN | \n", - "NaN | \n", - "0 | \n", - "
| 2 | \n", - "O1 | \n", - "NaN | \n", - "NaN | \n", - "0 | \n", - "
| \n", + " | tag | \n", + "test_acc | \n", + "train_acc | \n", + "alpha | \n", + "rand_num_spikes | \n", + "best_iteration | \n", + "num_boost_round_requested | \n", + "ww_note | \n", + "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "O1 | \n", + "0.932083 | \n", + "0.986667 | \n", + "2.415733 | \n", + "0.0 | \n", + "347 | \n", + "8000 | \n", + "\n", + " |
| 1 | \n", + "O3 | \n", + "0.931750 | \n", + "0.986354 | \n", + "1.708937 | \n", + "3.0 | \n", + "374 | \n", + "7000 | \n", + "\n", + " |
| 2 | \n", + "O2 | \n", + "0.930583 | \n", + "0.986219 | \n", + "2.389655 | \n", + "34.0 | \n", + "271 | \n", + "6000 | \n", + "\n", + " |
| 3 | \n", + "BEST | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "1304 | \n", + "6000 | \n", + "\n", + " |
| 4 | \n", + "T2 | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "1304 | \n", + "6000 | \n", + "\n", + " |
| 5 | \n", + "T1 | \n", + "0.921375 | \n", + "0.984229 | \n", + "1.827774 | \n", + "0.0 | \n", + "1751 | \n", + "4000 | \n", + "\n", + " |
| 6 | \n", + "T3 | \n", + "0.868708 | \n", + "0.900073 | \n", + "1.670708 | \n", + "0.0 | \n", + "1869 | \n", + "3000 | \n", + "\n", + " |
| 7 | \n", + "U2 | \n", + "0.781167 | \n", + "0.794625 | \n", + "2.717651 | \n", + "0.0 | \n", + "296 | \n", + "300 | \n", + "\n", + " |
| 8 | \n", + "U3 | \n", + "0.748125 | \n", + "0.753708 | \n", + "1.927702 | \n", + "0.0 | \n", + "237 | \n", + "250 | \n", + "\n", + " |
| 9 | \n", + "U1 | \n", + "0.743917 | \n", + "0.751885 | \n", + "2.664018 | \n", + "0.0 | \n", + "147 | \n", + "150 | \n", + "\n", + " |
| \n", + " | tag | \n", + "test_acc | \n", + "train_acc | \n", + "alpha | \n", + "rand_num_spikes | \n", + "multiclass_mode | \n", + "ww_note | \n", + "
|---|---|---|---|---|---|---|---|
| 0 | \n", + "R11_O1 | \n", + "0.932625 | \n", + "0.986656 | \n", + "1.741002 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 1 | \n", + "O1 | \n", + "0.932083 | \n", + "0.986667 | \n", + "2.415733 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 2 | \n", + "O3 | \n", + "0.931750 | \n", + "0.986354 | \n", + "1.708937 | \n", + "3.0 | \n", + "stack | \n", + "\n", + " |
| 3 | \n", + "R22_O3 | \n", + "0.931375 | \n", + "0.986531 | \n", + "2.459738 | \n", + "9.0 | \n", + "stack | \n", + "\n", + " |
| 4 | \n", + "R21_O3 | \n", + "0.931292 | \n", + "0.986302 | \n", + "2.443492 | \n", + "54.0 | \n", + "stack | \n", + "\n", + " |
| 5 | \n", + "R12_O1 | \n", + "0.930792 | \n", + "0.986615 | \n", + "1.698639 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 6 | \n", + "R31_O2 | \n", + "0.930667 | \n", + "0.986146 | \n", + "2.603029 | \n", + "43.0 | \n", + "stack | \n", + "\n", + " |
| 7 | \n", + "O2 | \n", + "0.930583 | \n", + "0.986219 | \n", + "2.389655 | \n", + "34.0 | \n", + "stack | \n", + "\n", + " |
| 8 | \n", + "R32_O2 | \n", + "0.930583 | \n", + "0.986219 | \n", + "2.389655 | \n", + "37.0 | \n", + "stack | \n", + "\n", + " |
| 9 | \n", + "BEST | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 10 | \n", + "T2 | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 11 | \n", + "T1 | \n", + "0.921375 | \n", + "0.984229 | \n", + "1.827774 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 12 | \n", + "T3 | \n", + "0.868708 | \n", + "0.900073 | \n", + "1.670708 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 13 | \n", + "U2 | \n", + "0.781167 | \n", + "0.794625 | \n", + "2.717651 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 14 | \n", + "U3 | \n", + "0.748125 | \n", + "0.753708 | \n", + "1.927702 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 15 | \n", + "U1 | \n", + "0.743917 | \n", + "0.751885 | \n", + "2.664018 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| \n", - " | tag | \n", - "test_acc | \n", - "train_acc | \n", - "alpha | \n", - "rand_num_spikes | \n", - "multiclass_mode | \n", - "ww_note | \n", - "
|---|---|---|---|---|---|---|---|
| 0 | \n", - "R11_O1 | \n", - "0.932625 | \n", - "0.986656 | \n", - "1.741002 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 1 | \n", - "O1 | \n", - "0.932083 | \n", - "0.986667 | \n", - "2.415733 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 2 | \n", - "O3 | \n", - "0.931750 | \n", - "0.986354 | \n", - "1.708937 | \n", - "3.0 | \n", - "stack | \n", - "\n", - " |
| 3 | \n", - "R22_O3 | \n", - "0.931375 | \n", - "0.986531 | \n", - "2.459738 | \n", - "9.0 | \n", - "stack | \n", - "\n", - " |
| 4 | \n", - "R21_O3 | \n", - "0.931292 | \n", - "0.986302 | \n", - "2.443492 | \n", - "54.0 | \n", - "stack | \n", - "\n", - " |
| 5 | \n", - "R12_O1 | \n", - "0.930792 | \n", - "0.986615 | \n", - "1.698639 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 6 | \n", - "R31_O2 | \n", - "0.930667 | \n", - "0.986146 | \n", - "2.603029 | \n", - "43.0 | \n", - "stack | \n", - "\n", - " |
| 7 | \n", - "O2 | \n", - "0.930583 | \n", - "0.986219 | \n", - "2.389655 | \n", - "34.0 | \n", - "stack | \n", - "\n", - " |
| 8 | \n", - "R32_O2 | \n", - "0.930583 | \n", - "0.986219 | \n", - "2.389655 | \n", - "37.0 | \n", - "stack | \n", - "\n", - " |
| 9 | \n", - "BEST | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 10 | \n", - "T2 | \n", - "0.927125 | \n", - "0.985479 | \n", - "1.921460 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 11 | \n", - "T1 | \n", - "0.921375 | \n", - "0.984229 | \n", - "1.827774 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 12 | \n", - "T3 | \n", - "0.868708 | \n", - "0.900073 | \n", - "1.670708 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 13 | \n", - "U2 | \n", - "0.781167 | \n", - "0.794625 | \n", - "2.717651 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 14 | \n", - "U3 | \n", - "0.748125 | \n", - "0.753708 | \n", - "1.927702 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| 15 | \n", - "U1 | \n", - "0.743917 | \n", - "0.751885 | \n", - "2.664018 | \n", - "0.0 | \n", - "stack | \n", - "\n", - " |
| \n", + " | tag | \n", + "alpha_mean_per_class | \n", + "alpha_median_per_class | \n", + "layers_analyzed | \n", + "
|---|---|---|---|---|
| 0 | \n", + "U1 | \n", + "NaN | \n", + "NaN | \n", + "0 | \n", + "
| 1 | \n", + "T2 | \n", + "NaN | \n", + "NaN | \n", + "0 | \n", + "
| 2 | \n", + "O1 | \n", + "NaN | \n", + "NaN | \n", + "0 | \n", + "
| \n", + " | tag | \n", + "test_acc | \n", + "train_acc | \n", + "alpha | \n", + "rand_num_spikes | \n", + "multiclass_mode | \n", + "ww_note | \n", + "
|---|---|---|---|---|---|---|---|
| 0 | \n", + "R11_O1 | \n", + "0.932625 | \n", + "0.986656 | \n", + "1.741002 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 1 | \n", + "O1 | \n", + "0.932083 | \n", + "0.986667 | \n", + "2.415733 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 2 | \n", + "O3 | \n", + "0.931750 | \n", + "0.986354 | \n", + "1.708937 | \n", + "3.0 | \n", + "stack | \n", + "\n", + " |
| 3 | \n", + "R22_O3 | \n", + "0.931375 | \n", + "0.986531 | \n", + "2.459738 | \n", + "9.0 | \n", + "stack | \n", + "\n", + " |
| 4 | \n", + "R21_O3 | \n", + "0.931292 | \n", + "0.986302 | \n", + "2.443492 | \n", + "54.0 | \n", + "stack | \n", + "\n", + " |
| 5 | \n", + "R12_O1 | \n", + "0.930792 | \n", + "0.986615 | \n", + "1.698639 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 6 | \n", + "R31_O2 | \n", + "0.930667 | \n", + "0.986146 | \n", + "2.603029 | \n", + "43.0 | \n", + "stack | \n", + "\n", + " |
| 7 | \n", + "O2 | \n", + "0.930583 | \n", + "0.986219 | \n", + "2.389655 | \n", + "34.0 | \n", + "stack | \n", + "\n", + " |
| 8 | \n", + "R32_O2 | \n", + "0.930583 | \n", + "0.986219 | \n", + "2.389655 | \n", + "37.0 | \n", + "stack | \n", + "\n", + " |
| 9 | \n", + "BEST | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 10 | \n", + "T2 | \n", + "0.927125 | \n", + "0.985479 | \n", + "1.921460 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 11 | \n", + "T1 | \n", + "0.921375 | \n", + "0.984229 | \n", + "1.827774 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 12 | \n", + "T3 | \n", + "0.868708 | \n", + "0.900073 | \n", + "1.670708 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 13 | \n", + "U2 | \n", + "0.781167 | \n", + "0.794625 | \n", + "2.717651 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 14 | \n", + "U3 | \n", + "0.748125 | \n", + "0.753708 | \n", + "1.927702 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |
| 15 | \n", + "U1 | \n", + "0.743917 | \n", + "0.751885 | \n", + "2.664018 | \n", + "0.0 | \n", + "stack | \n", + "\n", + " |