Anomaly Detection with Classical Machine Learning
Anomaly detection is about finding rare, unexpected, or suspicious data points. Before deep learning, classical ML methods dominated this space — and they still work well for many real‑world cases.
This post summarizes the most common classical algorithms and compares their strengths and weaknesses.
Common algorithms
1) Z‑Score / Gaussian thresholding
Assumes the data is normally distributed and flags points beyond a threshold (e.g., 3σ).
- Best for: simple univariate metrics.
- Weakness: breaks under non‑Gaussian data.
2) IQR (Interquartile Range)
Uses quartiles and flags points outside Q1 − 1.5×IQR or Q3 + 1.5×IQR.
- Best for: robust outlier detection in skewed distributions.
- Weakness: univariate; doesn’t capture interactions.
3) One‑Class SVM
Learns a boundary around “normal” data in feature space.
- Best for: complex but low‑dimensional data.
- Weakness: slow at scale, sensitive to kernel choice.
4) Isolation Forest
Randomly partitions data; anomalies require fewer splits to isolate.
- Best for: medium‑large datasets with mixed features.
- Weakness: less effective on very high‑dimensional sparse data.
5) Local Outlier Factor (LOF)
Compares density of a point to its neighbors.
- Best for: local anomalies where density changes.
- Weakness: sensitive to neighborhood size; struggles with global outliers.
6) DBSCAN / density clustering
Labels sparse regions as noise.
- Best for: clusterable data with noise.
- Weakness: parameter sensitive; hard in high dimensions.
Comparison table
| Algorithm | Strengths | Weaknesses | Best for |
|---|---|---|---|
| Z‑Score | Simple, fast | Assumes Gaussian | Univariate metrics |
| IQR | Robust to skew | Univariate only | Skewed distributions |
| One‑Class SVM | Flexible boundary | Slow, sensitive | Low‑dimensional complex data |
| Isolation Forest | Scales well | Less precise on very sparse data | Mixed feature datasets |
| LOF | Finds local anomalies | Parameter sensitive | Local density shifts |
| DBSCAN | Finds clusters + noise | Hard in high‑D | Clustered data with outliers |
How to choose quickly
- Start with IQR or Z‑Score for single‑metric monitoring.
- Use Isolation Forest for general‑purpose anomaly detection at scale.
- Use LOF when anomalies are local to neighborhoods.
- Use DBSCAN when clusters and noise are clearly separated.
Summary
Classical anomaly detection is still powerful. The right algorithm depends on data shape, dimensionality, and whether you care about local or global outliers. In practice, teams often run multiple detectors and combine their scores.