Derivation of GMM

May 5, 2017 - 2 minute read -

Derivation of one single-variable gaussian distribution

Considering a generative model, let $P$ denote the generated probability distribution, ${x}_{i=1}^n$ denote the data set, and $\mu$ and $\sigma$ denotes the parameters of a generated Gaussian Distribution. We will go over a max-likelihood process to find the parameters of the generated distribution.

Independence Assumption: $x_i\perp x_j\ \forall i,j \ \mbox{s.j} \ 1\le i,j\le n$

Result of one multi-variable gaussian distribution

Multi-variate Gaussian Distribution Probability Function

Result of EM algorithm

Gaussian Mixture

Latent Variable

Result

Ref