Statistical learning (STATS 790)

Last updated: 10 Apr 2023

Materials on this web page are under construction and subject to change

Course meetings

3:30-5:30 Mon (Toronto time), IWC (Ivor Wynne Centre) 223 (floor plan)
9:30-10:30 Weds, Hamilton Hall 102
For Fields attendees: by zoom

Misc info

Textbooks:
- T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, 2nd ed. https://web.stanford.edu/~hastie/Papers/ESLII.pdf
- Cosma Shalizi. Advanced Data Analysis from an Elementary Point of View. https://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/
Software: R, Julia, Quarto, TeX, (recommended) VS Code: see software page
Platforms: Avenue to Learn, course web page (see below)
Full course bibliography
Expected preparation: While there are no formal prerequisites, to succeed in this course you will need to be comfortable with (1) multivariable calculus; (2) linear algebra (focusing on numerical computation, e.g. at the level of MATH 3NA3); (3) basics of linear regression (e.g. at the level of STATS 3A03); (4) Some programming language; R is recommended, but knowledge of Python/Matlab/Java should be transferable; (5) basic knowledge of data science methods (e.g. lasso/ridge regression, random forests, boosting), at the level of STATS/CSE 780 (recommended; to catch up in this area, try James et al. (2013) chapters 2, 5-8).
Notes on plagiarism/collaboration/attribution

Where to find stuff

(to be completed)

SpringerLink

Communication

instructor: Ben Bolker
- General questions (anything you don’t consider private) are best asked in the appropriate channel on Teams
- Otherwise e-mail to bolker@mcmaster.ca with “STATS 790” in the subject line; include your MacID in your message (if applicable) if not e-mailing from your McMaster e-mail account
- office hours TBA
teaching assistant: TBA
- weekly office hours, TBA
- please contact …
- will answer as soon as possible, but will be more readily available during normal working hours (9 AM to 5 PM Toronto time)
- responses are not guaranteed within 24 hours of an assignment deadline; please plan ahead!

Schedule

(subject to revision!)

Week	Start date	Topic	readings	Assignments	notes
1	09 Jan 2023	Overview: big picture, software, bias-variance tradeoff	ESL ch. 1-2; ADA ch. 1; Breiman (2001) and one of {Miller, Foti, and Fox (2021); Gelman (2021); Raper (2020)}	software setup	esl chap 1 scribbles; ada chap 1 scribbles; intro; chap1; intro R code; intro Julia code
2	16 Jan 2023	Regression methods (OLS/lasso/ridge)	ESL ch. 3 (up to section 3.4.4/p. 91)	HW 1 (Friday)	esl chap 2 scribbles; chap2 notes
3	23 Jan 2023	Regression with non-Gaussian outcomes	ESL 4.4		ridge/pcr code; notes; myglm.R
4	30 Jan 2023	Basis expansion and regularization: GAMs	ESL 5	notes; South Africa ex
5	06 Feb 2023	Model assessment and selection		HW 2 (13 Feb)
6	13 Feb 2023	Catch-up
	20 Feb 2023	Midterm recess
7	27 Feb 2023	Assessing uncertainty		notes	HW 3 (3 Mar)
8	06 Mar 2023	Tree-based methods (CART/MARS/etc.)		notes
9	13 Mar 2023	Random forests, boosting		HW 4
10	20 Mar 2023	Trees continued; model interpretation/var importance		Project proposal	HW 4 (5 Apr)
11	27 Mar 2023	Kernel-based methods (SVM/GP)		notes
12	03 Apr 2023	Data with dependence (temporal/spatial)
13	10 Apr 2023	Wrap up		notes	Project draft,
	30 Apr 2023			Project final version

References

Breiman, Leo. 2001. “Statistical Modeling: The Two Cultures.” Statistical Science 16 (3): 199–215. http://www.jstor.org/stable/2676681.

Gelman, Andrew. 2021. “Reflections on Breiman’s Two Cultures of Statistical Modeling.” Observational Studies 7 (1): 95–98. https://doi.org/10.1353/obs.2021.0025.

James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning. Vol. 112. Springer.

Miller, Andrew C., Nicholas J. Foti, and Emily B. Fox. 2021. “Breiman’s Two Cultures: You Don’t Have to Choose Sides.” arXiv:2104.12219 [Cs, Stat], April. http://arxiv.org/abs/2104.12219.

Raper, Simon. 2020. “Leo Breiman’s "Two Cultures".” Significance 17 (1): 34–37. https://doi.org/10.1111/j.1740-9713.2020.01357.x.