AI/ML Gotchas: Overfitting, Bias-Variance, and Precision vs Recall

Most AI/ML bugs aren't about exotic math — they're about misunderstanding what a metric actually means, leaking test data into training, or trusting a model that's been overfit on a flawed dataset. These are the gotchas that produce models that look great in development and fail in production.

1. Accuracy on imbalanced data is meaningless

`` Dataset: 1000 emails, 990 not-spam, 10 spam Model: predicts "not spam" for everything Accuracy: 99% ``

99% accuracy sounds great. But the model catches zero spam. On imbalanced datasets, accuracy is misleading — you need precision and recall (and ideally F1 or AUC).

2. Precision vs recall — pick the right one for the problem

The classic memorization: precision = how PRECISE my positive predictions are. Recall = of all the positives that exist, how many did I RECALL.

3. Overfitting vs underfitting

• Underfitting: model too simple, fails on training AND test data. Bias is high.
• Overfitting: model too complex, memorizes training data, fails on test data. Variance is high.

The signal:

• Training accuracy 60%, test accuracy 58% → underfit. Get a more complex model or better features.
• Training accuracy 99%, test accuracy 65% → overfit. Get more data, simplify the model, add regularization.
• Training accuracy 95%, test accuracy 93% → well-fit (probably).

4. The bias-variance tradeoff

Total error = bias² + variance + irreducible noise

• Increasing model complexity → bias decreases, variance increases
• Decreasing model complexity → variance decreases, bias increases

You can't have both zero — the goal is finding the sweet spot. Cross-validation and regularization techniques (L1, L2, dropout, early stopping) are how you control this in practice.

5. Data leakage — the worst ML bug

Data leakage happens when information from the test set influences training. Classic forms:

• Random train/test split on time-series data: future data leaks into training. Always use a temporal split.
• Normalizing the whole dataset before splitting: the test set's mean/std influenced the normalization, leaking info.
• Feature engineering on the full dataset: same problem.
• Target leakage: using a feature that's only available AFTER you'd know the answer. E.g., predicting "will user churn?" using "did user cancel in the last 30 days" as a feature.

Symptom: model performs perfectly in evaluation but flops in production. If your model is "too good," check for leakage.

6. Cross-validation for small datasets, holdout for big ones

The mistake: using random K-fold on time-series data leaks future info into training.

7. ROC AUC is misleading on imbalanced data

ROC curves plot TPR vs FPR. AUC measures area under it. On imbalanced data, FPR stays low even for bad models because there are so few actual negatives.

For imbalanced classes, use PR-AUC (precision-recall AUC) instead. It's much harsher and reflects what you actually care about.

8. scikit-learn defaults that surprise

• LogisticRegression defaults to regularization with C=1.0 (not "no regularization")
• RandomForestClassifier defaults to 100 trees (in older versions, 10 — read your version)
• train_test_split defaults to test_size=0.25
• accuracy_score is the default scoring metric for most classifiers — but defaults aren't always what you want for imbalanced data

Always pass explicit parameters in production code. Defaults can change between library versions.

9. One-hot encoding explodes dimensionality

``python # Country with 200 unique values → 200 binary columns df_encoded = pd.get_dummies(df, columns=['country']) ``

For high-cardinality categoricals (zip codes, user IDs, product SKUs), one-hot encoding creates sparse high-dim data that hurts model performance and memory. Use target encoding, embeddings, or feature hashing instead.

10. Train/validate/test (NOT just train/test)

The right split:

• Training set (60-80%): model learns from this
• Validation set (10-20%): tune hyperparameters on this
• Test set (10-20%): final evaluation, touched ONCE at the end

If you tune hyperparameters using the test set, you've effectively trained on it — test scores are now optimistic.

11. "Random" forests are deterministic — until they're not

``python from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier() # uses random initial state ``

Without setting random_state, results vary between runs. Always set it for reproducibility:

``python RandomForestClassifier(random_state=42) ``

But beware: setting the same random_state for everything means you're testing one specific configuration. Run experiments with multiple seeds to estimate variance.

12. fit modifies the object in-place

```python model.fit(X_train, y_train) model.score(X_test, y_test) # OK

model.fit(X_test, y_test) # ← retrained on test data! model.score(X_test, y_test) # now meaningless ```

If you accidentally call .fit() on test data (e.g., copy-paste error), you've leaked. Your scores are now invalid.

13. Gradient descent gotchas

• Learning rate too high: loss diverges (goes to infinity/NaN)
• Learning rate too low: model takes forever to converge or stops at a local minimum
• No normalization: features with large scales dominate gradients
• Batch size too large: faster per epoch but worse generalization
• Batch size too small: noisy gradients, slow convergence

Modern optimizers (Adam, RMSprop) reduce learning-rate sensitivity but don't eliminate these issues.

14. Confusion matrix terminology

• Memorize:
• Precision = TP / (TP + FP) — accuracy of positive predictions
• Recall (sensitivity) = TP / (TP + FN) — coverage of actual positives
• Specificity = TN / (TN + FP) — coverage of actual negatives

15. Modern LLM gotchas (bonus)

If you're working with LLMs in production:

• Prompt injection: user inputs can override your system prompt. Always sanitize and use structured outputs.
• Hallucination: LLMs confidently invent facts. Use retrieval-augmented generation (RAG) or function calling for ground truth.
• Token costs scale with context: long conversation histories cost real money at scale.
• Temperature 0 isn't deterministic: token sampling at temp 0 is mostly deterministic but not guaranteed across model versions.
• Context windows have hard limits: silently truncating long inputs causes baffling behavior.

Need help with an ML model in production?

We've debugged production ML systems that were silently giving wrong answers for months. If your model worked in development but doesn't in production, or your accuracy dropped after a data update, we can audit it.