This seminar course will expand on the concepts introduced in CSC 535. The primary aim of this course is to explore advanced techniques in probabilistic graphical models (PGMs) and statistical machine learning (ML) more broadly. Students will develop the ability to apply these techniques to their own research. Students will learn to perform statistical inference and reasoning in complex probabilistic statistical models. The course will survey stateoftheart ML research including: variational inference, advanced Markov chain Monte Carlo sampling, Bayesian nonparametrics, Bayesian optimization, and Bayesian Deep Learning. Upon conclusion of this course students will be capable of developing new methods and advancing the stateoftheart in ML and PGM research.
D2L: https://d2l.arizona.edu/d2l/home/1205997
Piazza: https://piazza.com/arizona/fall2022/csc696h1
Instructor: Jason Pacheco, GS 724, Email: pachecoj@cs.arizona.edu
Office Hours (Zoom): Tuesdays 34:30pm, Thursdays 9:00am10:30am
Instructor Homepage: http://www.pachecoj.com
Date  Topic  Readings  Presenter / Slides 

8/22  Introduction + Course Overview  (slides)  
8/24  Probability and Statistics : A Review 
PRML
: Sec. 1.2.11.2.4 Optional: Why Isn't Everyone a Bayesian? Efron, B. 1986 Objections to Bayesian Statistics Gelman, A. 2008 Reference: PRML : Sec. 2.12.3 
(slides) 
8/29  Probability and Statistics : Graphical Models 
PRML
: Sec. 8.18.3 Optional: WJ : Sec. 2.1 and 2.2 
(slides) 
8/31  Probability and Statistics : Message Passing Inference 
PRML
: Sec. 8.4 Optional: Factor Graphs and the SumProduct Algorithm Kschischang, et al. 2001 Reference: Example factortovariable message update (Jupyter Notebook) 
(slides) 
9/05  Labor Day : No Classes  
9/07  Probability and Statistics : Message Passing Inference (Cont'd)  (slides)  
9/12  Probability and Statistics : The Exponential Family 
PRML
: Sec. 2.4 Optional: WJ : Sec. 3.13.3 
(slides) 
9/14  Variational Inference 
Variational Inference: A Review for Statisticians Blei, D., et al., J. Am. Stat. Assoc. 2017 Optional: PRML : Sec. 10.110.4 
Eric Duong (slides) 
9/19  Variational Inference : Mean Field Example 
Latent Dirichlet Allocation Blei, D. M., et al. JMLR, 2003 
Yang Hong (slides) 
9/21  Variational Inference : Stochastic Mean Field 
Stochastic Variational Inference Hoffman, M. D. et al. JMLR, 2013 
Amir Mohammad Esmaieeli Sikaroudi (slides) 
9/26  Variational Inference : Stochastic Mean Field (continued) 
Project Proposal (slides) 

9/28  Variational Inference : Stein Variational 
Stein Variational Gradient Descent Liu, Q. and Wang, D., NeurIPS. 2016 
Alex Loomis (slides) 
10/03  Monte Carlo Methods 
Introduction to Monte Carlo Methods MacKay, D. J. C . Learning in Graphical Models. Springer, 1998 
Jason (slides) 
10/05  Monte Carlo Methods (continued) 
Jason (slides) 

10/10  Monte Carlo Methods : Hamiltonian Monte Carlo 
MCMC Using Hamiltonian Dynamics Neal, R. M., From: "The Handbook of MCMC.", Chapman & Hall / CRC Press, 2011 Read Sec. 14 (inclusive) 
Maryam Eskandari 
10/12  Early Project Status (Phase I)  
10/17  Monte Carlo Methods : No UTurn Sampler 
The NoUTurn sampler: Adaptively Setting Path Lengths in HMC Hoffman, M. D. and Gelman, A. JMLR, 2014 

10/19  Early Project Status (Phase II)  
10/24  Implicit Models 
Markov chain Monte Carlo Without Likelihoods Marjoram, P. et al. PNAS, 2003 
Jason 
10/26  Implicit Models : Approximate Bayesian Computation 
Approximate Bayesian Computation (ABC) Sunnaker, M. et al. PLoS Computational Biology, 2013 
Md. Moyeen Uddin 
10/31  Implicit Models : Neural Likelihood Free Inference 
Sequential Neural Likelihood: Fast LikelihoodFree Inference with Autoregressive Flows Papamakarios, G. et al. AISTATS, 2019 
Shanrui Zhang 
11/02  Bayesian Deep Learning  Jason  
11/07  Bayesian Deep Learning : Variational Autoencoder 
Autoencoding Variational Bayes Kingma, D. P. and Welling, M. arXiv, 2013 
Tuan Nguyen 
11/09  Bayesian Deep Learning : Dropout Monte Carlo 
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Gal, Y. and Ghahramani, Z. ICML, 2016 
Sammi Abida Salma 
11/14  Bayesian Deep Learning : TBD  
11/16  
11/21  Gaussian Processes 
CH 2  2.1.3 and 2.1.5 (inclusive) Gaussian Processes for Machine Learning Rasmussen, C. MIT Press, 2006 

11/23  Bayesian Optimization 
Taking the Human Out of the Loop: A Review of Bayesian Optimization Shahriari, B. et al. Proceedings of the IEEE, 2015 

11/28  Bayesian Optimization (Continued)  TBD  
11/30  Course Wrapup  
12/05  Project Presentations  
12/07  Project Presentations 