Behind every alert, a transaction history.
We explored it to rank the likelihood of escalation.
Synthetic data.
Real experiments.
About 5 minutes to explore.
One alert, many transactions, one outcome. Our unit of prediction is the alert—not the individual transaction.
14,000 training alerts · Jan 2025–Dec 2026
were escalated
Predicting “dismissed” for everyone would miss every escalation. We care about ranking, not a majority-class shortcut.
Share of transaction rows, not alert counts
Cards and bank transfers dominate. International transactions are rare, making their per-alert summaries less stable.
No single field cleanly separates the outcomes. Explore what changes—and what mostly overlaps.
Escalated alerts had slightly more activity on average, but the distributions overlap heavily.
Transactions per alert.
Each class is normalized separately. Lines join histogram-bin centers; they are not fitted probability curves.
Daily transaction activity relative to the signal date
Per-alert means; each point is one calendar day. Day −1: 40.10 transactions per dismissed alert, 41.66 per escalated alert. Days 0–7 include eight calendar dates.
Cumulative distribution of transactions per alert
Millions of transactions, but only 14,000 labeled examples. We first needed an alert-level representation.
Join each alert to its transactions. Summarize the history before the alert date, rather than treating every transaction as an independent training example.
Measure size distributions inside each transaction type and direction. Add activity, variability, recent changes and sequence summaries. Then test which groups actually help.
Fit on four folds and score the fifth. Imputation and scaling learn only from each training fold. Repeat across three split seeds, then fit the selected model on all training alerts.
The size field is a standardized indicator—not currency. Negative values are valid; sums of this indicator are not financial turnover.
We removed feature groups to see what the model depended on. Category-specific size distributions made the biggest difference.
Compact logistic model · C = 0.01 · same five folds, split seed 67
OOF ROC-AUC · zoomed axis starts at 0.54. Baseline: 0.63022.
Remove category-conditioned size summaries and compare the ranking.
Groups overlap: removing composition also removes some recent-share changes. These are group diagnostics, not isolated causal effects.
The focused alternative: 57 size-focused features + regularized logistic regression. It reached 0.64100 AUC on split 67. LightGBM adds complementary information in a blend.
| Representation / model | Features | OOF AUC |
|---|
Same five folds · split seed 67.
Size-focused logistic regression does most of the work. A smaller LightGBM contribution improves the ranking.
A weighted average of probabilities. Both components are finally fitted on all 14,000 labeled alerts.
Bars show improvement above chance (AUC 0.50); the displayed range ends at 0.67.
AUC ≈ 0.645 means an escalated alert ranks above a dismissed one in about 64.5% of random pairs. It is not classification accuracy.
| Split seed | Original blend | Final ensemble | Gain |
|---|
The reference is the original 50/50 logistic / histogram-boosting configuration, recomputed with model seed 67 on matching folds.
On split 31415, the fixed 75/25 ensemble gained 0.01069 AUC. A 500-resample paired bootstrap gave a 95% interval of [0.00448, 0.01634].
This interval is conditional on fitted predictions and does not account for the full model search. Repeated splits reuse the same labels; they are stability checks, not independent holdout datasets.
Logistic: median imputation, missingness indicators, scaling, C=0.1. LightGBM: 350 trees, 7 leaves, learning rate 0.025, L2=10. Model and bootstrap seed 67. Timestamp ties use a canonical ordering.
Scores and curves use the executed notebook with split seeds 67, 2026 and 31415. Small numerical differences are possible on reruns.
Direction- and type-specific size distributions were more useful than a single overall description of activity.
Adding recent changes and more behavioral summaries did not automatically help. Feature removal improved the linear model.
The ensemble gained 0.011–0.012 AUC over the original baseline across the three fold assignments.