A Simple and General Exponential Family Model for Partial Membership and Factor Analysis

Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani

Description

This code provides implementations of unsupervised model with latent/hidden variables for the analysis of non-Gaussian data. Implementations of two models are provided: the Bayesian Partial Membership Model (BPM), and Exponential Family Factor Analysis/PCA (EXFA). These models allow for a wide range of statistical and modelling tasks, including:

Visualisation of high-dimensional data,
Dimensionality reduction,
Missing data imputation,
Analysis of heterogeneous (mixed) data sets.

Applications of these methods include: image denoising, collaborative filtering and recommender systems, ideal point analysis in political science, and data visualisation, amongst others. We use a general purpose sampling method known as Hybrid Monte Carlo, and provide code and examples for its use. The papers below provide other application areas. This, and related pages provide demos on the use of this code.

Setup
Demo examples

Experiments in chapter
Key papers related this code:
Support Packages
Miscellaneous

Matab code for Hybrid (Hamiltonian) Monte Carlo Sampling for exponential family PCA and Bayesian Partial Membership (see list of related papers below).

Download the Code

This code was written and tested using Matlab 2012a (7.14).

25 February 2013: Initial release. Download EFLVM.zip

Setup

Add all paths to Matlab and specify the directories to be used.

% Start matlab in the main directory and add:
addpath(genpath(pwd));

% On inital use change the directories that are to be used in the file:
disp('change folders in setup.dir')
edit setupDir.m
disp('paused .')
pause

Demo examples

1. Example of use of HMC for Bayesian logistic regression. Use link below for details and results.

demo_BayesLogReg;

2. Examples on synthetic data for EFA and BPM. Code provides analysis of Binary data only at present. Extensions to other cases follow similarly to the binary case. Use link below for details and results.

demo_EFA;
demo_BPM;

Experiments in chapter

All experiments and figures appearing in the chapter can be reproduced.

plotEFLVM('synth-efa');
plotEFLVM('synth-bpm');
plotEFLVM('votes-bpm');
plotEFLVM('votes-efa');
plotEFLVM('fuzzykm');

Key papers related this code:

Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani, A Simple and General Exponential Family Model for Partial Membership and Factor Analysis, chapter in Handbook of Mixed Membership Models. 2013.
Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani. Bayesian Exponential Family PCA, in NIPS 2008.
Katherine A. Heller, Sinead A. Williamson and Zoubin Ghahramani, Statistical Models for Partial Membership, in ICML 2008.

Support Packages

We make use of Matlab and the statistics toolbox, the lightspeed package and other utilities. These additional files are included in the folder support/.

Miscellaneous

This code may have bugs. If you find any errors, please let us know.

Shakir