Open pricing demo: GLM vs GBM on French motor data

A reproducible, public-data walkthrough of a frequency pricing model, from GLM baseline to GBM and SHAP, using the freMTPL2 dataset.

This project shows my modelling method using only public data, so anyone can run it and check it.

Dataset

freMTPL2freq is roughly 680k French motor third-party liability policies with exposure and claim counts. It’s available through the CASdatasets R package and on OpenML.

Plan

Data prep. Cap exposure at 1, band vehicle and driver age, and group sparse regions.
GLM baseline. Poisson with log link and log(exposure) offset.
GBM challenger. LightGBM with a Poisson objective and the same offset (passed as init_score).
Compare. Poisson deviance on a held-out set, lift charts, and double-lift of GBM vs GLM.
Explain. SHAP summary and dependence plots, which show the GBM interactions the GLM doesn’t capture.

import numpy as np
import statsmodels.api as sm
import statsmodels.formula.api as smf

glm = smf.glm(
    "ClaimNb ~ C(VehAgeBand) + C(DrivAgeBand) + C(Area) + np.log(Density) + C(VehBrand)",
    data=train,
    family=sm.families.Poisson(),
    offset=np.log(train["Exposure"]),
).fit()

Status

Notebook in progress.