Gaussian Mixture Models

The slides introduce Gaussian Mixture Models (GMMs) and extend to mixtures of Bernoulli distributions. They begin with the formulation of GMMs as weighted sums of Gaussian components, describing latent variables, prior and conditional distributions, and posterior responsibilities. Next, they explain parameter estimation for means, covariances, and mixing coefficients using maximum likelihood, showing how these correspond to weighted averages based on responsibilities. The Expectation-Maximization (EM) algorithm is then detailed, with E-step (responsibility computation) and M-step (parameter re-estimation), iterating until convergence. Finally, the slides discuss mixtures of Bernoulli distributions, covering their likelihood, latent representation, complete-data log-likelihood, responsibilities, and EM updates—laying a foundation for discrete latent variable models like HMMs.