Home | Portfolio | Terms and Conditions | E-mail me | LinkedIn
Multilevel Modelling of U.S. Home Loan Data
The housing market has undergone quite a change in the past decade, with more stringent lending criteria for housing having been enforced.
A key objective of financial institutions is to minimise the risk of mortgage lending by ensuring that the debtor is ultimately able to repay the loan.
In this example, multilevel modelling techniques are used to analyse data from the Federal Home Loan Bank System to determine the main influencing factors on loan-to-value ratios (LTVs) across the United States.
As a disclaimer - the below is not intended as any form of financial advice and is merely used as an example to illustrate the workings of the relevant models.
Feature Selection - ExtraTreesClassifier
The dataset in question is 55990 rows × 82 columns, and the first task is to conduct feature selection to ensure that the most relevant explanatory variables are selected for further analysis.
To this end, the ExtraTreesClassifier is run in Python - the script is available in the GitHub repository.
The ExtraTrees Classifier identified Income (x9), UPB or Unpaid Principal Balance (x12), Loan Amount (x50), Back-End Ratio (x52), and PMI or percent of original loan balance covered by mortgage insurance (x55) as the variables with the highest score as ranked by the classifier:
A linear regression was then generated in R to analyse the impact of the above variables in LTV.
Due to the VIF detecting multicollinearity across the variables for PMI and Back-End ratio, these two variables were dropped from the model.
Given that LTV is expressed in percentage format, the log of Income and UPB was used as the independent variables in the regression model.
The following results were generated:
> # Linear Regression > reg1 <- lm(LTV ~ log(Income) + log(UPB), train) > summary(reg1) Call: lm(formula = LTV ~ log(Income) + log(UPB), data = train) Residuals: Min 1Q Median 3Q Max -0.78943 -0.06494 0.02240 0.08982 0.42595 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.916374 0.015363 59.65 <2e-16 *** log(Income) -0.061727 0.001654 -37.32 <2e-16 *** log(UPB) 0.045761 0.001569 29.17 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.1536 on 44789 degrees of freedom Multiple R-squared: 0.03078, Adjusted R-squared: 0.03074 F-statistic: 711.3 on 2 and 44789 DF, p-value: < 2.2e-16
From the above, the selected variables show high significance at the 5% level. However, the R-Squared of 15% is quite low. This indicates that while these variables are of importance in explaning the variation in LTV, there are either some important variables that are missing in this regard, or LTV itself is subject to a high degree of random variation.
Given that this is the case, the eventual multilevel modelling will not be used for predicting LTV per se - rather to analyse whether significant differences exist across this variable by state.
The Breusch-Pagan test for heteroscedasticity was run, and the VIF test for multicollinearity was run once again:
> library(lmtest) > bptest(reg1) studentized Breusch-Pagan test data: reg1 BP = 1229.1, df = 2, p-value < 2.2e-16 > library(car) > vif(reg1) log(Income) log(UPB) 1.828859 1.828859
From the above, a VIF < 5 indicates that multicollinearity is not present in the model. However, a p-value of below 0.05 for the Breusch-Pagan test indicates that heteroscedasticity is present in the model. This is an issue across this dataset, given that different states with varying incomes are being represented. This results in a situation where states with higher incomes and higher housing prices are being given more weight by the model, skewing the results.
Multilevel Modelling with lmer
In order to mitigate this issue, a better solution is to analyse LTV variation by State, specifically filtering by FIPSStateCode as illustrated in the dataset.
In this regard, random effects are firstly being tested for. It is hypothesised that the effect of Income, UPB and Loan Amount will vary across different states. Therefore, random slopes and intercepts are being generated in this regard.
> # Random effects > r1 <- lmer(LTV ~ 1 + (1 + log(Income) |FIPSStateCode), data = train, REML=FALSE) Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.0179241 (tol = 0.002, component 1) > r2 <- lmer(LTV ~ 1 + (1 + log(UPB) |FIPSStateCode), data = train, REML=FALSE) > r3 <- lmer(LTV ~ 1 + (1 + log(Amount) |FIPSStateCode), data = train, REML=FALSE) Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.0318744 (tol = 0.002, component 1) > anova(r1,r2,r3) Data: train Models: r1: LTV ~ 1 + (1 + log(Income) | FIPSStateCode) r2: LTV ~ 1 + (1 + log(UPB) | FIPSStateCode) r3: LTV ~ 1 + (1 + log(Amount) | FIPSStateCode) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) r1 5 -43118 -43075 21564 -43128 r2 5 -43906 -43862 21958 -43916 787.748 0 < 2.2e-16 *** r3 5 -43929 -43886 21970 -43939 23.235 0 < 2.2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
For models r1 and r3, a “Model failed to converge” error was obtained. Moreover, when testing for ANOVA, we see that the fit improvement between r2 and r3 is marginal. Therefore, r2 is used to represent the random effect for the model.
Now, an attempt is made to model the fixed effects between Income and LTV.
> # Fixed effects > m1=lmer(LTV ~ log(Income) + (1 + log(UPB) |FIPSStateCode), data = train, REML = FALSE) Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.264243 (tol = 0.002, component 1) > summary(m1) Linear mixed model fit by maximum likelihood ['lmerMod'] Formula: LTV ~ log(Income) + (1 + log(UPB) | FIPSStateCode) Data: train AIC BIC logLik deviance df.resid -45886.6 -45834.4 22949.3 -45898.6 44786 Scaled residuals: Min 1Q Median 3Q Max -5.5590 -0.4516 0.1039 0.6687 3.1559 Random effects: Groups Name Variance Std.Dev. Corr FIPSStateCode (Intercept) 1.395926 1.1815 log(UPB) 0.008612 0.0928 -1.00 Residual 0.020841 0.1444 Number of obs: 44792, groups: FIPSStateCode, 53 Fixed effects: Estimate Std. Error t value (Intercept) 1.642752 0.019925 82.45 log(Income) -0.071496 0.001586 -45.09 Correlation of Fixed Effects: (Intr) log(Income) -0.935 convergence code: 0 Model failed to converge with max|grad| = 0.264243 (tol = 0.002, component 1)
However, a “Model failed to converge” error is obtained once again. In this regard, the random effects portion of the model generated earlier will be used to model this problem.
Here are the summary coefficients for the model:
> summary(r2)$coefficients Estimate Std. Error t value (Intercept) 0.8032787 0.007135987 112.5673
Here is a breakdown of the intercept and slope across states:
> coef(r2) $FIPSStateCode log(UPB) (Intercept) 1 -0.048725940 1.424839137 2 0.071810557 -0.129437630 4 0.048194732 0.144809114 5 0.009605831 0.686029830 6 0.079868545 -0.373252028 8 0.066271417 -0.058015008 9 0.002805582 0.825132568 10 0.041174114 0.313942756 11 -0.021533357 1.059306143 12 0.048449155 0.193163918 13 0.029536299 0.427508846 15 0.033478378 0.274425498 16 0.083027128 -0.230813954 17 -0.037538462 1.267887084 18 0.062788077 0.002831210 19 0.012487451 0.632244118 20 0.042927859 0.273238843 21 0.084679686 -0.278537734 22 0.013490445 0.638005090 23 0.196958251 -1.714481216 24 -0.064853734 1.666322971 25 0.092872449 -0.434507155 26 0.063390995 -0.008880011 27 0.010592391 0.649027798 28 -0.021592652 1.134230484 29 0.046409632 0.220361661 30 -0.015760240 1.003635842 31 0.030503923 0.412809350 32 0.128775010 -0.842500079 33 0.042681993 0.239718731 34 0.079823518 -0.273370374 35 0.064508955 -0.037793158 36 -0.026011440 1.070115946 37 -0.017808095 1.028135790 38 -0.028361889 1.180258974 39 0.059861306 0.032634645 40 0.084900070 -0.235779484 41 0.134953722 -0.918560447 42 0.009586766 0.690380057 44 0.018198628 0.597527446 45 -0.084074305 1.892673690 46 0.024345909 0.479053971 47 0.090823001 -0.344979523 48 -0.003117153 0.856093955 49 0.084008316 -0.283965571 50 0.084902254 -0.165635531 51 -0.006251239 0.886143183 53 0.041150673 0.227720105 54 0.019188765 0.541915533 55 0.005422513 0.753196009 56 0.085633968 -0.296857819 66 0.118784559 -0.560868823 72 -0.055580831 1.595271936 attr(,"class")  "coef.mer"
The data is then aggregated (i.e. the average UPB, LTV, Income, and Loan Amount is calculated), and the following data frame is generated along with the random slope and intercept across states:
When analysing the data frame as a whole, it is quite interesting to observe that there is generally an inverse relationship between the Intercept and Slope, i.e. the lower the intercept, the higher the Unpaid Principal Balance across states, and vice versa.
Here are some visualization plots:
UPB vs LTV
LTV vs Income
UPB vs Income
What is particularly helpful to analyse this type of data is a bubble chart, which is developed using plotly. Here is a plot of average income versus average UPB by state, with the ratio of UPB over income determining the size of the bubble.
Here is the bubble chart, illustrating the plot of income versus UPB. States (represented by bubbles) that are smaller in size demonstrate a lower ratio of UPB to income, and these states therefore represent lower risk for a lender. We can also see a few small bubbles where the average income is greater than $120k, and therefore an institution may be more inclined to target high-income individuals with little risk from a UPB standpoint.
Here is an interactive web-based chart: https://plot.ly/~michaeljgrogan/15/#/
From the plots, we can see that:
- There is a negative correlation between UPB and LTV, i.e. as the average LTV falls, the average UPB rises.
- There is a negative correlation between LTV and Income, i.e. as the average income rises, average LTV falls.
- There is a strong positive correlation between UPB and Income, i.e. as the average income rises, average UPB rises.
These findings are interesting in that they illustrate higher incomes are not necessarily the main predictor of risk when extending home loans. In the case of states with higher incomes, we have seen that the Unpaid Principal Balance rises quite significantly, reflecting a combination of 1) higher house prices in that state, and 2) the propensity of higher-income earners to purchase homes with significantly higher prices than that of lower-income earners.
Therefore, while states with higher incomes had slightly lower LTVs, the effect was not overly pronounced.
In this regard, one possible implication of this is that an institution can choose to set parameters as to their lending criteria, depending on how conservative the institution wishes to be.
For instance, an institution could choose to focus its marketing efforts on states with a UPB of less than $300,000 and an LTV of lower than 0.75.
Or, if the institution wanted to take some risk in terms of the loan size, another criteria set could be a UPB of up to $600,000 but an LTV of lower than 0.70 to compensate for the added risk.
In this example, we have seen how multilevel modelling is used to conduct regression analysis across different segments in a dataset, and specifically how this has been applied in assessing mortgage risk.