
Problem
Loan officers decide on thousands of applications by hand. A model that predicts approval from the application form could speed up the easy cases and flag risky ones. The question: which applicant characteristics predict approval, and how well can a model predict it?
Approach
- Exploratory analysis of 3,584 historical applications from a Zimbabwean commercial bank: income, guarantees, loan amount, term, profession, location and credit history.
- Mode and median imputation, a log transform for skewed loan amounts, engineered features (total income, instalment, balance income) and one-hot encoding (401 features).
- Logistic regression, decision tree, random forest and XGBoost (tuned with grid search), each with stratified 5-fold cross-validation and a hidden test set.
Result
| Model | Mean CV accuracy | Hidden test accuracy |
|---|---|---|
| Logistic regression | 0.867 | 0.785 |
| Random forest | 0.867 | 0.778 |
| XGBoost (tuned) | 0.867 | 0.778 |
| Decision tree | 0.798 | 0.646 |
Assurance review

- 86.7% of applications were approved. A rule that approves everyone scores 0.867, exactly the cross-validated accuracy of three models.
- The model predicted approve for all 1,076 validation applications: recall on rejections is 0.00, so it never flags a loan that should be declined.
- The root cause was a wrong assumption that the classes were balanced. With an AUC of 0.72 there is real signal; the fix is class weights, threshold tuning and metrics that see the minority class (balanced accuracy, PR-AUC).
What this shows
Building the full modelling pipeline, and then doing what an AI assurance review should: checking a headline metric against a baseline, finding that it says nothing, and naming the fix. The bank and loan account numbers are anonymised in the published notebook.