Product / Deep Analysis
A real AutoML search, not a wrapper around a chatbot
Bayesian hyperparameter search across up to five model families, cross-validated feature selection, out-of-fold stacking, and multi-seed variance reduction, run entirely on the machine you already have, while it is idle.
| family | CV R² |
|---|---|
| stack (champion) | 0.412 |
| lightgbm | 0.399 |
| xgboost | 0.391 |
| random forest | 0.360 |
What it does
Deep Analysis is CLAIR's compute-heavy AutoML tier. A single run is a staged, checkpointed pipeline that can span multiple idle windows or nights and survive a restart: it prepares seeded cross-validation folds, ranks and selects features, optionally engineers new stateless candidate features, runs a Bayesian hyperparameter search per model family, builds leak-free out-of-fold predictions, and stacks them into a meta-learner that only becomes champion if it beats every single family on its own. With more than one seed, the whole pipeline repeats and averages to reduce the "lucky seed" effect.
The champion is persisted. When your data changes, CLAIR re-fits the known-best pipeline in minutes instead of searching from scratch, and an occasional fresh search can challenge the incumbent, only replacing it when the gain clears both a fixed margin and the noise in both scores.
How it works
Three tiers trade time for search depth. Automatic feature selection runs in every tier and is never a knob you have to touch.
| Tier | Folds | Families | Trials/family | Seeds | Budget |
|---|---|---|---|---|---|
| Quick | 5 | XGBoost only | 30 | 1 | minutes |
| Standard | 10 | XGBoost, LightGBM, Random Forest | 100 | 3 | 20 to 40 min |
| Exhaustive | 10 | every available family (5, incl. CatBoost, for the alpha) | 300 | 5 | hours / overnight |
Explainability runs on the champion: SHAP global and local importances, and partial-dependence and ICE curves. A per-column drift check (PSI plus a KS or chi-square test) compares one dataset version against another. Trained models compile to ONNX and run through ONNX Runtime with hardware acceleration chosen for your machine, falling back to CPU.
Every feature-importance panel in CLAIR, Quick-look, the SHAP chart, Deep Analysis's own champion importance, and the Regression tab, runs the same shared leakage checks and shows the same caution sentence when one fires: a row-number or surrogate-key column standing in for a real predictor, a column that is really just a rebinned copy of the target, or a threshold-derived flag that leaks the target's own cutoff. A panel with no caution means the check ran and found nothing, not that it never ran. Separately, a champion is checked against the obvious guess: a classifier is compared to its own majority-class baseline, a regressor to predicting the mean, and a champion that does not clearly beat that baseline is marked weak with a plain caution everywhere its result appears.
Honest limits
- The Exhaustive tier is opt-in and clearly labelled as heavy: it can run for hours and is meant for an idle machine, not an interactive wait.
- A stacked champion (several model families combined) can be scored, including What-if and batch prediction, but CLAIR deliberately refuses to generate a SHAP explanation for it: a stack's prediction is a weighted combination in the meta-learner's own space, not the original feature space SHAP values live in, and CLAIR will not invent a per-column attribution it cannot stand behind. A single-family model can be explained; Quick-look always trains one, and a Deep search sometimes finishes on one too.
- CatBoost is only searched in the Exhaustive tier, and only when it is enabled for your build.
- ONNX export applies to single-XGBoost champions; a stacked or scikit-learn champion is kept in its native format instead.
Related
Chat over your data with cited, verified answers.
A model for one outcome column, with a range, not just a number.
Background jobs that make chat and analysis sharper after upload.
Your files stay files. CLAIR tracks and refreshes them.
HTML, PowerPoint, PDF, and publication-ready figures.
Boards of charts and pinned findings, exported as one PDF.