Research note · RN-002

How we hunt financial crime — including the experiments that failed.

Our detection strategy in five decisions: think in networks, layer detectors that cover each other's blind spots, validate on ground truth we didn't build, chase the representation instead of the model — and let the analyst's time be a budget, not a firehose. With the numbers, including the ones that refused to move.

Pierre Lague11 min read

Financial crime is a network phenomenon. A mule ring is unremarkable one account at a time: modest balances, plausible transfers, nothing a per-row rule would stop. The crime only becomes visible as shape— money fanning into a collector, value hopping through intermediaries faster than commerce moves, a cluster of “strangers” sharing structure no strangers share. Meanwhile, European deployments carry a second, non-negotiable constraint: under the incoming AML package and the AI Act, a score nobody can explain is a product nobody can deploy. Our strategy follows from taking both facts seriously at once.

Decision one: the unit of analysis is the network

Before any model runs, the engine resolves accounts, devices, identifiers, and counterparties into one graph, because that is the level at which laundering exists. The classic typologies — fan-in collection, fan-out scatter, gather-scatter layering, cycles — are graph signatures, and we maintain them as an explicit, versioned library that mirrors the playbooks investigators already use. A typology match is evidence an analyst recognises on sight, which is worth more in a case file than any probability.

Decision two: five layers, one explained verdict

No single detector survives contact with a live adversary, so we don't field one. We field five, chosen so that each covers the others' blind spots:

LayerCatchesBlind spot it leaves
Deterministic rulesthe known and the mandatory — hard blocks, high precisionanything not yet codified
Supervised modelmany weak signals fused into a risk scoreschemes with no labels yet
Anomaly (one-class)novel behaviour — no label requiredknows 'unusual', not 'fraud'
Graph typologiescollusion: rings, mules, synthetic clusterscrimes without structure
Ensemble + human reviewfuses the above; routes the uncertain tail to experts
The detection fabric. One shared feature store feeds every layer — a regulator, an analyst, and the model all read the same numbers with the same definitions.

The fusion step is where most ensembles quietly become black boxes, so ours is deliberately boring: fixed weights, then a corroboration policy — the score is boosted when three or more layers agree and damped when a lone weak signal fires. Cases route to human review not just on score, but on epistemics: when detectors disagree, when a case sits near a threshold, or when the anomaly layer flags behaviour that matches no rule and no known typology — which is exactly what an emerging scheme looks like, and gets surfaced as one rather than silently discarded. The review queue is ordered by information value, so analyst attention lands where it teaches the system most.

Decision three: validate on ground truth we didn't build

Every vendor's engine works on the vendor's demo data. So beyond our internal benchmark we validate on the IBM AMLSim corpus (Altman et al., NeurIPS 2023) — hundreds of thousands of accounts of realistic transaction flow with labelled laundering, built by people who have never seen our code. First honest answer: scoring whole money-flow clusters, the way our console groups alerts, the engine managed 0.70 AUROC — real signal, far from acceptable, and the published graph-neural-network results on this data (~0.95) told us how much was left on the table.

Decision four: chase the representation, not the model

What followed is the most instructive stretch of this project. We attacked that 0.70 three times, and three times the number refused to move:

HypothesisInterventionResult
“It needs graph features”4 structural features (fan-out, hubs, cycles, reciprocity)0.700 → 0.699
“It needs amount features”per-transaction dispersion & size signalsunchanged
“It needs a better model”logistic → random forest0.693 vs 0.691 — a tie
Three falsified hypotheses on the cluster-level task, IBM AMLSim 1M-row slice. The lesson wasn't in any single failure but in their pattern.

Three controlled failures form a diagnosis: the bottleneck was the representation. Cluster-level candidates carry diluted labels (a forty-account neighbourhood counts as illicit if one member is) and aggregate away the per-account structure the signal lives in. No model outruns its labels. So we changed the unit: score every account individually, with clean per-account labels and features describing its own behaviour and graph context. Same dataset, same nine interpretable features, same zero-dependency forest — 0.70 became 0.910. The single largest gain in the project came from changing what the model looks at, not the model.

This note adds the next chapter. The 0.910 vector still had two blind spots: the account's temporal shape (burstiness, active lifespan, sustained rate) and its counterparties' behaviour beyond a single averaged degree. We added four temporal features and one round of direction-aware neighbourhood aggregation— for every base feature, its mean over the account's senders and receivers separately. That is the computation a one-layer graph neural network performs, hand-rolled, inspectable, and still free of any dependency. As a control, we also re-ran the old features with a much larger forest:

ConfigurationForest AUROCtop 0.1% precision / recall
9 base features (previous published)0.9100.20 / 0.11
9 base features + 2× forest capacity0.9180.24 / 0.13
+ temporal & 1-hop context features0.9480.80 / 0.45
context features + capacity (final)0.9560.81 / 0.45
Ablation on IBM AMLSim (1M rows, 423,684 accounts, 754 illicit, 5-fold out-of-fold). Representation moves the number; capacity barely does.

The representation change delivers roughly five times the gain of doubling model capacity — the fourth controlled experiment to land on the same conclusion — and stacking the two lands at 0.956: an interpretable, dependency-free forest matching the published graph-neural-network results on this dataset, with every score still decomposing into named, human-readable evidence. The feature importances tell the story plainly: the account's active lifespan and sustained transaction rate— signals the model previously could not see — jump straight to the top, exactly where AML intuition says disposable conduit accounts should give themselves away, with the counterparties' currency mix (a neighbourhood aggregate) right behind them.

As with our agentic-detection work, we publish the method and the measurements, not the calibration: exact normalisations, thresholds, and fusion weights are the product, and they are fitted per deployment anyway.

Decision five: analyst time is a budget, not a firehose

An AUROC is a ranking, and a ranking becomes operational the moment you fix a budget. At the new operating curve, a team reviewing the top 0.1% of accounts — about 420 on this dataset — works at 81% precision and captures 45%of all laundering accounts, at a base rate of 0.18%. Widen to the top 0.5% and recall reaches ~60%; the top 5% captures three quarters. The budget is chosen with the client, per team size — and the scores behind it are calibrated probabilities, not raw model outputs, so “80% likely” means what it says.

The loop then closes the way the whole fabric is designed to: every analyst verdict is a durable label that retrains the supervised layer; every decision exports as a provenance dossier — the evidence chain in W3C PROV format, integrity hashed — so the answer to “why was this account flagged?” survives the analyst, the quarter, and the audit.

Limitations
IBM AMLSim is a synthetic benchmark — the best public option precisely because real labelled laundering data cannot circulate, but synthetic nonetheless; results on it are evidence of method, not a production guarantee. Numbers above are account-level; entity-level scoring (one actor, many accounts) remains measurably harder (0.849) and is where our current work points. Cross-corpus robustness on the dataset's other variants is queued, not done. And the engine's identity features (devices, IPs, onboarding) are idle on this flow-only corpus — deployments with identity signal have more to work with, not less.

Fighting the same adversary?

If your team works fraud, mule networks, or AML cases and wants detection it can explain to a regulator, we should talk.

Start a conversation