beanmachine.ppl.inference.vi.discrepancy module

Csiszar f-functions in log-space.

beanmachine.ppl.inference.vi.discrepancy.kl_forward(logu: torch.Tensor) → torch.Tensor

Log-space Csiszar function for forward KL-divergence D_f(p,q) = KL(p||q).

Also known as the inclusive KL-divergence, minimizing results in zero-avoiding / mass-covering behavior.

Parameters

logu (torch.Tensor) – ``p.log_prob``s evaluated at samples from q.

beanmachine.ppl.inference.vi.discrepancy.kl_reverse(logu: torch.Tensor) → torch.Tensor

Log-space Csiszar function for reverse KL-divergence D_f(p,q) = KL(q||p).

Also known as the exclusive KL-divergence and negative ELBO, minimizing results in zero-forcing / mode-seeking behavior.

Parameters

logu (torch.Tensor) – ``p.log_prob``s evaluated at samples from q.