Automatic structured variational inference

[cscherrer]

https://arxiv.org/abs/2002.00643
https://twitter.com/LucaAmb/status/1359561091278381056

[cscherrer]

@LucaAmbrogioni it seems like this should be a "model transform". So the output is a new model. Does that sound right?

I mean, as opposed to outputting a log-density function, or some other code.

[LucaAmbrogioni]

Yes indeed. It takes a model as input and it output a new trinable model.

[cscherrer]

Ok IIUC it's something like

1. Find the MLE, set it aside
2. Transform the model to its prior
3. Turn each distributional argument into a convex combination of the prior and the MLE

And I guess there's a different variational parameter for each, none are shared?

[LucaAmbrogioni]

No you do not need to find the MLE independently (although that is an interesting research idea, the problem is that the relevant likelihood is given by all the downstream observed nodes and it involves latent variables). You just set it as a free parameter.

Then you can train both the convex coefficients and the MLE parameter jointly.

[cscherrer]

Ah right, that makes more sense