Overfitting, underfitting and the bias–variance trade-off

A 99% training score with 70% on test is classic overfitting: the model has memorised the training data, including its noise, instead of learning patterns that generalise.

Bias–variance trade-off. For squared error, expected test error = bias² + variance + irreducible noise.

• Bias is error from wrong assumptions, such as fitting a straight line to a curve.
• Variance is sensitivity to the particular training sample: retrain on slightly different data and the predictions change a lot.
• Increasing model complexity usually lowers bias and raises variance. The aim is the point where validation error is lowest. Learning curves (error against training-set size) and validation curves (error against a complexity setting) show where you are.

Fixing overfitting (reduce variance)

• Get more training data, or use data augmentation.
• Regularise: L2 (ridge) or L1 (lasso) penalties, dropout, weight decay.
• Use a simpler model: fewer features, shallower trees (max_depth), pruning.
• Early stopping on validation loss.
• Ensembles: bagging and random forests average away variance.
• Use cross-validation to choose hyperparameters honestly.

Fixing underfitting (reduce bias)

• Use a more expressive model or add better features (interactions, polynomial terms).
• Reduce the regularisation strength.
• Train longer, or tune the learning rate.

Also check for leakage. A suspiciously high test score usually means information about the target slipped into the features, or duplicate rows appear in both train and test.