TK
Menu

Data Science / Machine Learning

Loan Approval Prediction with an Assurance Review

Four classifiers predicting loan approval on 3,584 applications from a commercial bank, and a review showing why 87% accuracy only matched the always-approve baseline: the model never predicted a rejection.

Problem

Loan officers decide on thousands of applications by hand. A model that predicts approval from the application form could speed up the easy cases and flag risky ones. The question: which applicant characteristics predict approval, and how well can a model predict it?

Approach

  • Exploratory analysis of 3,584 historical applications from a Zimbabwean commercial bank: income, guarantees, loan amount, term, profession, location and credit history.
  • Mode and median imputation, a log transform for skewed loan amounts, engineered features (total income, instalment, balance income) and one-hot encoding (401 features).
  • Logistic regression, decision tree, random forest and XGBoost (tuned with grid search), each with stratified 5-fold cross-validation and a hidden test set.

Result

ModelMean CV accuracyHidden test accuracy
Logistic regression0.8670.785
Random forest0.8670.778
XGBoost (tuned)0.8670.778
Decision tree0.7980.646

Assurance review

Confusion matrix of the logistic regression model

  • 86.7% of applications were approved. A rule that approves everyone scores 0.867, exactly the cross-validated accuracy of three models.
  • The model predicted approve for all 1,076 validation applications: recall on rejections is 0.00, so it never flags a loan that should be declined.
  • The root cause was a wrong assumption that the classes were balanced. With an AUC of 0.72 there is real signal; the fix is class weights, threshold tuning and metrics that see the minority class (balanced accuracy, PR-AUC).

What this shows

Building the full modelling pipeline, and then doing what an AI assurance review should: checking a headline metric against a baseline, finding that it says nothing, and naming the fix. The bank and loan account numbers are anonymised in the published notebook.