In part 1 of this blog series, ‘AI-Enabled Supervision: The Truth Behind False Positive Reduction Claims’ we explained what false positives are, why they are difficult to eliminate and why vendor claims of 99% or 100% rate reduction are a myth. In this article we provide a technical deep dive into the techniques that data scientists use to reduce false positives, the metrics that actually matter, and the questions firms can ask vendors to verify reduction rate claims.
Metrics that Matter: Precision vs. Recall
As discussed previously, the base rate of actual misconduct in corporate communications is extremely low (often less than 0.01%) which makes “accuracy” a misleading metric in these environments. Let’s assume a fraud rate of 0.01%. A model that predicts “No Fraud” for every email could achieve 99.99% accuracy — yet detect none of the fraudulent emails that we wanted to find. In low base-rate problems where there are large numbers of negative or ‘normal’ data and very few positives, or ‘bad’ data, accuracy on the large number of negatives can be easily achieved by even the most naive classifier. For example, labelling every communication negative means that the accuracy stat can be boosted as the natural occurrence of fraud is very low. This approach masks the failure of detecting the positive data that we need to find. What matters more than accuracy is how well a model finds the rare positives, or how it distinguishes the rare “needle” from the overwhelming “haystack.”
Since accuracy can be misleading, we must use metrics that account for imbalance. Below we describe three metrics that are crucial for better understanding the performance and effectiveness of risk detection models: precision, recall, and F-1 scores.
Precision – The “Trust” Metric
- What it answers: “Of all the emails the model claimed were fraud, what percentage actually were fraud?”
- High Precision means fewer False Positives.
Recall – The “Safety” Metric
Recall = True PositivesTrue Positives + False Negatives
- What it answers: “Of all the actual fraud emails that exist, what percentage did the model find?”
- High Recall means less Missed Fraud.
The F1-Score
The F1-score is the harmonic mean of Precision and Recall. A harmonic mean is an average of the reciprocal of values and is better suited for averaging rates or ratios. It gives equal weight to both the precision and recall values even if they are very different in magnitude and is more sensitive to small values in either input. If either precision or recall are low then the harmonic mean will drop significantly, while the arithmetic mean will show a high average. The F1 ratio provides a balanced view of the classifier’s performance, preventing a vendor from gaming the system by optimizing only for either precision or recall.
Put simply, if a single number is what you want to look at then the F1-score is a better indicator of the usefulness of the risk detection. One can also use weighted F1 scores with different weights for precision vs. recall depending on the business impact of one vs the other.
The Confusion Matrix
Even better than a single number, is the confusion matrix, which provides a more complete picture of performance and is the basis for calculating nearly all performance metrics. The confusion matrix indicates not just how many errors are made, but also the types of those errors. For binary classification (positive vs negative), it is a 2×2 matrix with predicted values on one side and actual values on the other. To illustrate, below is an example of a confusion matrix for a version of Theta Lake’s market manipulation risk detection.
The four quadrants show numbers for each possibility and provide a more detailed view of a classifier’s performance.
Techniques to Reduce False Positives
Developing ML models for extremely skewed distributions (where negatives vastly outnumber positives) is notoriously difficult. Standard algorithms often prioritize the majority class (the large number of un-interesting negatives) to minimize overall error, effectively ignoring the more important minority class (the tiny fraction of important positives).
The Right Metrics: Precision and Recall
Instead of accuracy, we must use metrics like precision and recall across multiple detection thresholds and tune those thresholds along with the metrics. There is always a trade-off. Increasing Recall (catching more bad guys) usually lowers Precision (creating more false alarms), and vice versa.
Data scientists use specific techniques to handle low base-rate problems without drowning in FPs:
- Contextual Embeddings (LLMs): Moving away from keyword matching (which triggers on “kill”) to semantic understanding (which understands “killed it” means “did well”).
- Cost-Sensitive Learning: Programming the algorithm to penalize False Negatives (missing a fraud) more heavily than False Positives during training, or vice versa, depending on the business goal.
- Ensemble Methods: Combining multiple machine learning models can materially reduce FPs by compensating for individual model weaknesses. At Theta Lake, we combine machine learning techniques in principled ways so as to maximize overall precision and recall while keeping our models robust to variations. We also combine elements like lexicons, fuzzy matching, and rules to add another layer of ensembling that further improves precision, recall, robustness, and efficiency.
- Threshold Tuning: Instead of a binary “Yes/No,” models output a probability score (e.g., 85% chance of collusion). By raising the threshold (e.g., only flag if >95%), you reduce FPs but risk missing some TPs (true positives). At Theta Lake we tune thresholds on large amounts of test data to ensure the best tradeoff between precision and recall on real world data.
- Post-Processing Filters: Adding rules-based logic after the ML model. (e.g., “If the ML flags ‘shoot’, but the context contains ’email’ or ‘message’, suppress the alert”).
- Resampling:
- Undersampling: Randomly removing “innocent” examples from the training data to balance the scales.
- Oversampling (e.g., SMOTE): Synthetically generating new “guilty” examples to help the model learn the features of misconduct better.
Verifying Vendor’s Accuracy Claims
When a vendor claims their AI has “99% accuracy” or “Detects 99% of Fraud” you must dig deeper into their validation methodology. Reported metrics can vary wildly depending on the data used for evaluation.
- The Balanced Set Trap: A vendor might test their model on a balanced dataset that is 50% fraud and 50% benign. The model will appear to perform exceptionally well.
- The Real World Failure: When that same model is deployed on real-world data (0.01% fraud), the False Positive rate will skyrocket, rendering the tool unusable.
How to verify:
- Ask for performance metrics on a stratified test set that mimics the actual distribution of your live data, not a balanced training set.
- Request a Confusion Matrix or ask for Precision/Recall numbers. This will show how many FPs and False Negatives the model generated.
- Definition of a “Hit”: Are they counting individual sentences or whole threads? (Metrics vary greatly depending on the unit of analysis).
- Out-of-Domain Testing: Did they test the model on data it has never seen before?
Continuous Monitoring and Improvement
A classifier is not a “set it and forget it” tool. Language evolves (e.g., new slang for insider trading, new ways to commit fraud), and data distribution drifts. Best practice includes:
- Feedback Loops: You must have a mechanism where human reviewers can flag FPs. This data must be fed back into the training set.
Theta Lake provides multiple mechanisms to incorporate user feedback such as the Audit Report, Bug Reports, and easy to use Thumbs Up/Down flags.
- Drift Detection: Monitor the rate of positives over time. If the “Fraud” rate suddenly jumps from 0.1% to 5% without a business reason, your model may be drifting or reacting to a new benign email template.
How Theta Lake approaches this
Theta Lake actively monitors hit rates for all our classifiers across all data centers. We ensure that all our built-in classifiers are updated frequently and monitor metrics before and after each update to ensure continual improvement in precision and recall as data distributions change over time. Our cloud based platform and uniform set of classifiers prevent “bit rot” where a classifier is modified once and then decays over time. This is frequently the case with other vendors who customize classifiers for each customer since it’s impossible for them to monitor, maintain and update all those custom classifiers over time.
Practical Takeaway: Questions to Ask Compliance Vendors
Undoubtedly AI detections can significantly reduce the number of false positives, which in turn, free up compliance and supervisory staff time to conduct more thorough reviews of higher-quality alerts. But with vendors racing to claim the highest false positive reduction rates, and the use of misleading accuracy rates, firms need to proceed with caution to ensure that AI systems are detecting risks, and continue to do so over time.
If you are evaluating a solution for compliance or surveillance, ask these specific questions:
- Data Distribution: “Was the model evaluated on a dataset with a realistic base rate (e.g., <1% positives) or a balanced dataset (50/50)?”
- Metrics: “Please provide the Precision, Recall, and F1-score. Do not just give me Accuracy. What does the Precision-Recall curve look like?”
- Testing: “How does the model perform on ‘adversarial’ examples (benign texts that look like bad ones)?”
- Volume: “Based on our daily volume of 100k emails, how many alerts per day should we realistically expect?”
- FP Reduction: “What specific post-processing or linguistic filters do you use to prevent common benign idioms (like ‘killing the competition’) from triggering alerts?”
- Feedback Loops: “How does the system learn from our analysts’ dismissals of False Positives?”
