ENH improve obsmodels

The current state of obsModels is really, really ugly.

Time and again, they seem to block progress on what should be "easy" fronts. The current code is hard to follow, with both ObsModel objects (that track suff stats, ELBOs, and interact with the HModel) and Distr objects (that do some of the core computations). Logic is further divided depending on EM and VB, with EM computations happening, for example, either in ObsModel or in a Distr object like GaussDistr.py, while VB stuff mostly happens in the conjugate prior class, like GaussWishDistr.py.

Here's a list of what's tricky about all this: * MAP inference for EM is an "obvious" want-to-have, but currently the code for creating the prior is trapped in a Distr object that isn't needed for anything else in EM * isolation of posterior update logic into Distr objects makes things hard to vectorize * distributed logic all over the place is really hard to follow * the list structure for obsModel.comp can be annoying to deal with. Reshuffling the order of components is handled differently than the allocModels, etc. * the obsModel.comp field is confusing: it can store either point estimates or conjugate prior/posterior parameters, and the user needs to figure it out based on the infer algorithm / context * lots of the ELBO computation is hard-to-debug and harder to get a "big picture" for whats going on, how to streamline, what might cancel, etc. * the path for initialization is very confusing

So, I've started to think about a redesign, where * ALL logic (EM, VB, soVB, Gibbs updates, etc) happens in ONE class (either GaussObsModel.py, or MultObsModel.py, or DiagGaussObsModel.py, etc.). No more Distr objects.

Instead of a list of comps, the obsModel object has two (maybe empty) ParamBags: EstParams and Post. EstParams hold the point estimates (delivered by EM, for example), while Post holds the conjugate posterior parameters. This makes things more standard, and we can use the ParamBag methods like insertComp or deleteComp to resize as needed.
Initialization is very standard. You can call either setEstParams or setPost, and hand over either (1) a data object and an LP, so we do classic M step updates, (2) an existing Post or EstParams bag, which is used to initialize directly, or (3) custom values for the params/hyperparams.
AbstractObsModel has just bare-bones code for interfacing with HModel, and some cool general-purpose caching code, so we can save the results of expensive computations (like choleskies or log determinants) in a way that is tested once and available worldwide, and doesn't need to be reimplemented for each ObsModel.

Comments (3)