Computer Science 287/487

Sampling Algorithms

Fall 2020

Instructor: Daniel Štefankovič
Class: Monday, Wednesday 2:00pm-3:15pm, ONLINE (Zoom).
Instructor office hours, ONLINE (Zoom): Tuesday 9pm-10pm, Thursday 8am-9am.


Textbooks: there is no required textbook; see each section for the recommended reading (mostly online; I will also supply handouts).

Other resources/recommended reading:

• [MR95] Randomized Algorithms, R. Motwani, P. Raghavan, 1995.
• [H02] Finite Markov chains and algorithmic applications, O. Haggstrom, 2002.
• [J03] Counting, sampling and integrating: algorithms and complexity, M. Jerrum, 2003.
• [H16] Perfect Simulation, M. Huber, 2016.
• [MU17] Probability and Computing: Randomized Algorithms and Probabilistic Analysis (2nd ed), M. Mitzenmacher, E. Upfal, 2017.

Schedule

TOPIC 1: Introduction (1 class)

Introduction to counting and sampling.
Exact counting, approximate counting (FPRAS), approximate sampling.
Basic graphical models (Ising model, Potts model, colorings, hard-core model).
Survey of results and open problems.


TOPIC 2: Exact counting (2-3 classes)

Spanning trees (determinant).
#P-completeness.
Planar matchings (pfaffian).

Reading:

• [J03] - Chapter 1 and 2

TOPIC 3: Approximate counting (2-3 classes)

Monte Carlo (Markov, Chebyshev, Chernoff, squared coefficient of variation).
Rejection sampling, counting DNF assignments.
Network unreliability.
AP-reductions.

Reading:

• [J03] - Section 3.1 and 3.2
• D. Karger. A randomized fully polynomial time approximation scheme for the all-terminal network problem.
• M. Dyer, L. Goldberg, C. Greenhill, M. Jerrum. The Relative Complexity of Approximate Counting Problems.

TOPIC 4: Markov chain basics (2 classes)

Markov chain, ergodicity. Stationary distribution.
Ergodicity. Total variation distance. Mixing time.
Reversibility. Detailed balance condition.
Relaxation time.

Reading:

• [J03] - Section 3.3
• D. Gillman. A Chernoff bound for random walks on expander graphs.
  

TOPIC 5: Coupling (3-4 classes)

Path coupling.
Colorings. Linear extensions. Spanning trees.
Censoring. Coupling for Ising model.

Reading:

• [J03] - Chapter 4
• R. Bubley, M. Dyer. Faster Random Generation of Linear Extensions.
• D. Wilson. Mixing times of lozenge tiling and card shuffling Markov chains.
• Y. Peres, P. Winkler. Can Extra Updates Delay Mixing?
• E. Mossel, A. Sly. Exact Thresholds for Ising-Gibbs Samplers on General Graphs.

TOPIC 6: Perfect sampling (2-3 classes)

Coupling from the past.
LLL based perfect sampling. Network reliability.

Reading:

• [H02] - Chapters 10 and 11.
• R. Moser, G. Tardos. A constructive proof of the general Lovász Local Lemma.
• H. Guo, M. Jerrum, J. Liu. Uniform Sampling Through the Lovász Local Lemma.
• H. Guo, K. He. Tight bounds for popping algorithms.

TOPIC 7: Canonical paths (4-5 classes)

Conductance, canonical paths, lower and upper bounds on mixing time.
Matchings.
High temperature expansion for Ising model.
Permanent. Annealing.

Reading:

• [J03] - Chapter 5
• M. Jerrum, A. Sinclair. Polynomial-Time Approximation Algorithms for the Ising Model.
• M. Jerrum, A. Sinclair, E. Vigoda. A Polynomial-Time Approximation Algorithm for the Permanent of a Matrix with Nonnegative Entries.
• I. Bezáková, D. Štefankovič, V. Vazirani, E.Vigoda. Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems.
• D. Štefankovič, S. Vempala, E.Vigoda. Adaptive Simulated Annealing: A Near-Optimal Connection between Sampling and Counting.
• M. Huber. Approximation algorithms for the normalizing constant of Gibbs distributions.

TOPIC 8: Tight results (2 classes)

Correlation decay technique for the hard-core model.
Tight hardness result for the hard-core model.

Reading:

• D. Weitz. Counting Independent Sets up to the Tree Threshold.
• A. Sinclair, P. Srivastava, M. Thurley. Approximation Algorithms for Two-State Anti-Ferromagnetic Spin Systems on Bounded Degree Graphs.
• L. Li, P. Lu, Y. Yin. Correlation Decay up to Uniqueness in Spin Systems.
• N. Sun, A. Sly. Counting In Two-Spin Models on d-regular Graphs.
• A. Galanis, D. Štefankovič, E.Vigoda. Inapproximability for Antiferromagnetic Spin Systems in the Tree Nonuniqueness Region.

TOPIC 9: Recent results (4-5 classes)

Sampling Bases of a Matroid.
Influence matrix.

Reading:

• N. Anari, S. Oveis Gharan, C. Vinzant. Log-Concave Polynomials I: Entropy and a Deterministic Approximation Algorithm for Counting Bases of Matroids.
• N. Anari, K. Liu, S. Oveis Gharan, C. Vinzant. Log-Concave Polynomials II: High-Dimensional Walks and an FPRAS for Counting Bases of a Matroid.
• N. Anari, K. Liu, S. Oveis Gharan, C. Vinzant. Log-Concave Polynomials III: Mason's Ultra-Log-Concavity Conjecture for Independent Sets of Matroids.
• N. Anari, K. Liu, S. Oveis Gharan, C. Vinzant. Log-Concave Polynomials IV: Exchange Properties, Tight Mixing Times, and Faster Sampling of Spanning Trees.
• M. Cryan, H. Guo, G. Mousa. Modified log-Sobolev inequalities for strongly log-concave distributions.
• V. Alev, L. Lau. Improved Analysis of Higher Order Random Walks and Applications.
• N. Anari, K. Liu, S. Oveis Gharan. Spectral Independence in High-Dimensional Expanders and Applications to the Hardcore Model.
• Z. Chen, K. Liu, E. Vigoda. Rapid Mixing of Glauber Dynamics up to Uniqueness via Contraction.
• Z. Chen, A. Galanis, D. Štefankovič, E. Vigoda. Rapid Mixing for Colorings via Spectral Independence.
• W. Feng, H. Guo, Y. Yin, C.Zhang. Rapid mixing from spectral independence beyond the Boolean domain.
• H. Guo, M. Jerrum. Random Cluster Dynamics for the Ising Model is Rapidly Mixing.

Grading

The grade for the course is based on the homework (theoretical and applied; there will be 5-7 problem sets) and a small project; there are no exams.

Homework Rules

• Theoretical homework should be typeset (LaTeX preferred) and emailed to the instructor by the deadline (specific for each assignment). You may work with other people on the theoretical homework, but you must each write up your solutions separately (without any written aid). If you work with other people, indicate who you worked with on your solution.
  
  
• No late homework will be accepted without instructor's permission (a permission must be requested at least 24 hours before the due date).

This course follows The University Policy on Academic Honesty .