Course Content
This page includes all the class content for the course thus far. We will update this page with lecture materials, lab materials, and readings as the class goes on.
The class is roughly split into two parts. The “classical ML” part will concern foundational techniques and ideas that underlie the subfield of machine learning from a unifying statistical learning framework. Sam will be the lecturer in this part. The “modern ML” part will concern deep learning and techniques that have gained traction in the past decade because of the proliferation of data and computational power. Nick will be the lecturer in this part.
- Lecture slides can be found by clicking on the lecture title for the appropriate day.
- All the materials and reading on the right column is optional, but reading (a subset of) these materials before each lecture might help digesting the content during lecture.
- ESL refers to The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman.
- UML refers to Understanding Machine Learning: From Theory to Algorithms by Shalev-Shwartz and Ben-David.
- ISL refers to An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani.
- PPA refers to Patterns, Predictions, and Actions by Hardt and Recht.
This is a tentative schedule and is subject to change.
Lecture Recordings. Recordings of the lectures are available to watch on Brightspace. Just navigate to the Brightspace page and then click Zoom.
Math Review. As stated in the prerequisites, students are expected to have a solid understanding of linear algebra, probability and statistics, and multivariable calculus for this course. In addition to the resources listed on the homepage of the site, here are several shorter notes to get you started if you need to review material:
- Mathematics for Machine Learning (includes all the topics so you can pick and choose)
- Probability Review
- Linear Algebra Review
Sam also designed a course the past two summers at Columbia meant to give students a deeper understanding of these prerequisites (given that they have already taken them and would like to progress to graduate-level machine learning). The entire course could be found at this page: Math for ML, and lecture videos can be found in “Video Recordings” on this page.
Math Review Videos. We will also occasionally post some Math Review Videos after that week’s lab depending on lab feedback on what students found confusing about the material in that week. Familiarity with all the prerequisites in this class can definitely feel daunting at times, so the purpose of these review videos are to re-introduce some prereq concepts or lecture derivations at a slower pace. These are a totally optional extra resource. Hopefully this helps you digest or page in you may have been rusty on!
Part 1: “Classical” ML (Weeks 1–11)
Week 1: Introduction & Basic Statistical Learning Framework
- Jan 20
- Lecture Lecture 1: Intro & Supervised Learning Framework
- UML 2.1-2.2; ISL 2.1, 3.1 - 3.4; notes on conditional expectation (highly recommended if this feels unfamiliar)
- (marked up lecture slides)
- Jan 22
- Lab Lab 1: Statistical Learning Review & Intro to Gradient Descent
- Math Review
- (1) Math Review: Joint, Conditional, and Marginal Distributions (notes); (2) Math Review: Conditional Expectation (notes); (3) Math Review: Bayes Hypothesis (notes)
Week 2: Optimization & Gradient Descent
Week 3: Regularization & Loss Functions
- Feb 3
- Lecture Lecture 3: Regularization & Loss Functions
- ESL Ch. 3 or ISL Ch. 6.1-6.2 for a lighter intro; PPA pg. 92-98, PPA pg. 100-106 (for a more “modern” treatment of regularization), PPA pg. 122-125 (if you’re really interested in “modern treatment” of regularization)
- Feb 5
- Lab
Week 4: Linear Classification, Convex Optimization, & SVM
- Feb 10
- Lecture
- Feb 12
- Lab
Week 5: President's Day (Class/Lab Cancelled)
- Feb 17
- No class this week!
- Feb 19
- No lab this week!
Week 6: Features & Kernels
- Feb 24
- Lecture
- Feb 26
- Lab
Week 7: MLE & Conditional Probability Models
- Mar 3
- Lecture
- Mar 5
- Lab
Week 8: Midterm
- Mar 10
- Midterm
- Mar 12
- No lab this week!
Week 9: Spring Break (Mar 16-20)
- Mar 17
- No class this week!
- Mar 19
- No lab this week!
Week 10: Decision Trees, Bagging, & Random Forests
- Mar 24
- Lecture
- Mar 26
- Lab
Week 11: Boosting
- Mar 31
- Lecture
- Apr 2
- Lab
Part 2: Modern ML (Weeks 12–16)
Week 12: Learnable Features → Deep Neural Networks
- Apr 7
- Lecture
- Apr 9
- Lab
Week 13: Structured Neural Networks
- Apr 14
- Lecture
- Apr 16
- Lab
Week 14: Generative Models
- Apr 21
- Lecture
- Apr 23
- Lab
Week 15: Reinforcement Learning
- Apr 28
- Lecture
- Apr 30
- Lab
Week 16: Language Models
- May 5
- Lecture
- May 7
- End of semester, no lab this week!