Hidden Markov models with variational inference in marketing science
Abstract
Hidden Markov Models (HMMs) are a well known type of model for many varieties of sequen-
tial data. There exist several algorithms for learning HMMs: a variant of an expectation-
maximization (EM) algorithm known as the Baum Welch method, Markov Chain Monte
Carlo (MCMC), and Variational Inference (VI). This third method is less frequently used, yet
it has interesting properties with regard to convergence, sparsity, and interpretation that are
worth further exploration. HMMs are used as explanatory models in the field of marketing
science, where one of the goals is to interpret the model structure to understand customer
behavior. This thesis will explore the use of HMMs trained with VI to build an interpretable
classification model for customer churn on a dataset consisting of call data records from a
mobile telecommunications company.
In this thesis we first provide an introduction to VI for HMMs and then derive a mixture
of HMMs (mHMMs) using VI. A mHMMs is then shown to be quite capable of performing
unsupervised clustering of sequential data. Next, we present the design and interface of a new
open source library for training HMMs and mHMMs with VI and EM. We show that this
library achieves excellent performance while still providing an intuitive interface in the Python
programming language. We then examine the performance of classifiers using HMMs trained
with VI and EM on several classification datasets. The results from these experiments are then
used to build and test several simple classification models to predict churn for the provided
dataset. As these models are shown to have poor performance, we train a more traditional
machine learning model based on gradient boosted trees and evaluate the interpretability,
stability and performance of this model over a subsequent 18 months of data.
Type
Thesis, DEng Doctor of Engineering
Rights
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
Collections
Except where otherwise noted within the work, this item's licence for re-use is described as Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.