AI has transformed supervision, with 94% of financial services firms reporting that they are using—or planning to use—AI-based detection capabilities.
As described in the Financial Industry Regulatory Authority’s report on AI Applications in the Securities Industry, AI technology offers firms the ability to capture and surveil large amounts of structured and unstructured data in various forms (e.g., text, speech, voice, image, and video) from both internal and external sources in order to identify patterns and anomalies. This enables firms to holistically surveil and monitor functions across the enterprise, as well as monitor conduct in a more efficient, effective, and risk-based manner.
A key benefit of the tools is the significant reduction of false positives—freeing up compliance and supervisory staff time to conduct more thorough reviews of the remaining alerts. It is therefore no surprise that compliance vendors are racing to claim the highest false positive reduction rates. But firms should proceed with caution.
In this two-part series, Rohit Jain, Distinguished Engineer at Theta Lake, draws on more than 20 years of experience working with multiple generations of machine learning models and methods to explain why.
Crying Wolf: False Positives in Machine Learning
In the world of Machine Learning, particularly in domains like surveillance and compliance monitoring for behaviors like insider trading, collusion, or workplace misconduct, we are often looking to find a needle in a huge haystack of digital communications. One of the most persistent and expensive challenges is when the machine claims it has found a needle, but it is actually just a piece of straw: the False Positive (FP).
What is a False Positive?
In the context of Machine Learning (ML), a False Positive occurs when a model incorrectly predicts the positive class (bad behavior we are looking for) when it is actually negative (normal behavior). The system flags an event as “suspicious” or “guilty” when it is actually benign.
For text-based sentence classifiers in compliance, FPs often stem from linguistic ambiguity, sarcasm, or industry-specific jargon.
The High Cost for Firms of False Positives
It is tempting to tune a system to be “better safe than sorry”—flagging everything that might be risky. However, excessive False Positives create systemic risks. They’re not just a minor annoyance; they become operational bottlenecks. Below are a few examples of the costly impacts of excessive FP rates.
- Resource Drain: Every flagged item requires human review. If an analyst spends 10 minutes investigating a false alarm, and the system generates 100 FPs a day, that is nearly 17 hours of wasted labor daily. It is not an efficient use of time if a compliance officer spends 90% of their day reviewing benign emails incorrectly flagged as “insider trading.”
- The “Boy Who Cried Wolf” Effect: This is the most dangerous consequence. When humans review hundreds of false alarms, they become desensitized. They begin to scan faster and less critically. Boredom and repetition lead to rapid, careless clicking. In this state, a human reviewer is statistically more likely to dismiss a True Positive (actual misconduct) because they have been conditioned to expect false alarms. A reviewer, bored and tired from dismissing 500 incorrect alerts, might inadvertently click “Dismiss” on the 501st alert—which happened to be actual evidence of fraud.
The Base Rate Problem: Needles in Haystacks
To understand why FPs are so difficult to eliminate, we must understand Base Rates. The base rate is the natural occurrence of the event in the wild. In corporate communications, the base rate of actual bad behavior is infinitesimally low, perhaps 0.01% of all emails.
- The Math of Inevitability: Even a model with 99% accuracy will generate an overwhelming number of FPs when the base rate is low. If you scan 1,000,000 emails and only 10 are fraud, a 1% error rate means you will have 10,000 False Positives for those 10 actual cases.
- Accuracy is easy!: If your communications haystack has 99% hay and 1% needles, just classify the whole haystack as ‘hay’ and you’re 99% accurate even though you mis-classified the 1% that was needles as ‘hay’. This misclassification does not solve the practical problem of detecting and investigating “needles,” but allows organizations to claim 99% accuracy. Frame of reference is important.
- Bias-Variance Tradeoff: You cannot simply “tune out” all FPs.
- To reduce FPs, you make the model stricter (High Bias). But then you risk missing actual fraud (Low Recall).
- To catch all fraud, you make the model looser (High Variance). This inevitably lets in more noise (High FPs).
- There is no “perfect” setting; only a tradeoff that matches your business risk appetite.
Why the Accuracy Metric is a Trap
Firms should never trust “Accuracy” as a metric where there is a large mismatch between the number of positive vs negative samples. A model that predicts “No Fraud” for every single email will be 99.999% accurate, but it is 0% useful because it catches nothing. This is commonly referred to as a “low base-rate” problem. For cases where we are looking for the very few bad (positive) samples in a massive set of normal (negative) samples, accuracy can be very high because it is easy to be accurate on a large number of negatives by calling everything negative. Accuracy measured solely on the uninteresting negative samples masks the inaccuracy on the positive samples that are truly of interest. This is why “Accuracy” is a vanity metric in these kinds of detections. It hides the model’s inability to find the rare “needle” inside the massive “haystack.”
When a vendor claims its AI has “99% accuracy,” firms must interrogate their validation methodology. Reported metrics can vary wildly depending on the data used for evaluation in terms of the base rates of positives and negatives in that data. In part 2 we will look at the techniques to reduce false positives and the questions firms need to ask vendors.
