Resources
A collection of textbooks and resources that have been recommended to me or that I’ve found useful. I clearly have not gone through every resource listed here.
I don’t maintain this page actively. Please contact me at schitrak@ualberta.ca if there’s a link issue or a resource you’d like added!
One day I will write a road-map on how I would read through this list if I wanted to learn a certain topic.
Linear Algebra
- Introduction to Applied Linear Algebra by Boyd & Vandenberghe (2018) — Accompanying resources
- YouTube: Essence of Linear Algebra by 3Blue1Brown (2016) — Essential intuition of linear algebra
- Introduction to Linear Algebra by Gilbert Strang (2023) — MIT 18.06 videos (2010)
- Linear Algebra for Data Science by Kang & Cho (2025)
- Stanford CS229 Linear Algebra Review (2015)
- Matrix Calculus for Machine Learning and Beyond by Edelman & Johnson (2025) — MIT Courseware (2025), MIT videos (2025), Summary notes, golden reference
- For introduction see: Matrix Calculus Introduction ~1 hour (2020)
- The Matrix Calculus You Need for Deep Learning by Parr & Howard (2018) — Really good paper
- The Matrix Cookbook by Petersen & Pedersen (2012) — How to study?, Matrix Forensics (2021), essential reference
- Matrix Computations by Golub & Van Loan (2013) — Algorithmic linear algebra, has nice matrix computation tricks
- Numerical Linear Algebra by Trefethen & Bau (2022)
- Mathematics for Machine Learning by Deisenroth, Faisal & Ong (2020) — Videos (San Diego Book Club 2024)
- Linear Algebra and Optimization for Machine Learning by Aggarwal (2020)
- Matrix Differential Calculus by Magnus & Neudecker (2019)
- Matrix Analysis by Horn & Johnson (2013)
- Linear Algebra Done Right by Axler (2024) — Accompanying videos, essential but avoids determinants
- Linear Algebra by Berberian (2014)
- Linear Algebra in Action by Dym (2013) — Connects theory to applications
- Linear Algebra by Hoffman & Kunze (1971) — Classic rigorous book that covers almost everything
- Finite Dimensional Vector Spaces by Halmos (1958) — Problem Book (1995), supplementary not a core book, bit out of convention today as notation is different, Axler book was clearly inspired by this
Mathematical Analysis
- Book of Proof by Hammack (2018) — Excellent starting point to learn proof writing
- How to Prove It by Velleman (2019)
- How to Think Like a Mathematician by Houston (2009)
- Proofs: A Long-Form Mathematics Textbook by Cummings (2021) — Accompanied videos
- Discrete Mathematics with Applications by Epp (2019) — Gentle introduction/beginners, good for CS
- Discrete Mathematics and Its Applications by Rosen (2018) — Also popular for beginners
- Concrete Mathematics by Graham, Knuth & Patashnik (1994) — Graduate level
- Understanding Analysis by Abbott (2015) — Solutions, Recommend for self-study, more readable than Rudin
- Analysis I & II by Tao (2016) — Also good for self study!
- Mathematical Analysis by Apostol (1974) — Really good second book!
- Baby Rudin: Principles of Mathematical Analysis by Rudin (1976) — Study guide, classic but terse, not great for self-study but excellent as second read + exercises
- Note: Rudin’s multivariable integration uses differential forms (differential geometry background). His functional analysis book is also considered outdated (field moved towards C* algebras and Banach spaces).
- Real Mathematical Analysis by Pugh (2015) — Visual/geometric approach, 500+ exercises
- Real Analysis Game Lean 4 (2025) — Follow-along lecture notes, interactive formal proofs in Lean
- Counterexamples in Analysis by Gelbaum & Olmsted (2003) — Very nice book!
Note: Unless you specifically need measure theory, and you’re only curious about probability theory, get a measure-theoretic probability book instead (e.g., Williams, Billingsley, Capiński) as pure measure theory can be dry.
- Measures, Integrals and Martingales by Schilling (2017) — Also: Counterexamples in Measure and Integration I like this! Tailored to probability theory!
- Note: Doesn’t cover Vitali coverings and absolutely continuous functions which should be standard.
- Real Analysis: Measure Theory, Integration, and Hilbert Spaces by Stein & Shakarchi (2005) — Great first book!
- Measure, Integral and Probability by Capiński & Kopp (2004) — Accompanied video lectures, focuses on measure theory then connects to probability with finance applications
- An Epsilon of Room by Tao (2010) — Read after an introduction to measure theory
- Measure Theory and Fine Properties of Functions by Evans & Gariepy (2015) — Read after Epsilon of Room and before functional analysis
- Measure, Integration & Real Analysis by Axler (2020) — Readable as well
- An Introduction to Measure Theory by Tao (2011) — Bit harder, but teaches very good problem-solving approaches.
- Real Analysis by Folland (1999) — Reference book, elegant presentation, has probability theory chapters
- Real Analysis by Royden (2024) — Another classic reference book
- Measure Theory by Cohn (2015) — Accompanied lectures, Has cool topics like prob. theory, compact group measures
- Measure Theory by Bogachev (2006) — 2-volume, extremely advanced and comprehensive
- Introductory Functional Analysis with Applications by Kreyszig (1991) — Best introduction, no measure theory
- A Course in Functional Analysis by Conway (2007) — Standard text, good after Kreyszig
- Functional Analysis, Calculus of Variations and Optimal Control by Clarke (2013) — Really good for optimization
- Functional Analysis for Probability and Stochastic Processes by Bobrowski (2005)
- Functional Analysis, Sobolev Spaces and PDEs by Brezis (2011) — Standard text for PDE applications
- Probability and Measure Theory by Ash (1999) — Has a chapter on functional analysis
- Asymptotic Statistics by Van der Vaart (2000) — Advanced statistics with functional analysis topics
- Fourier Analysis: An Introduction by Stein & Shakarchi (2003) — Beginner introduction
- Fourier Analysis by Duoandikoetxea (2001) — Better advanced book. Really well written!
- Classical Fourier Analysis by Grafakos (2014)
- Singular Integrals and Differentiability Properties of Functions by Stein (1970)
- Lectures on Harmonic Analysis by Wolff (2003)
Note: For topology read up on separability, metrization, Urysohn metrization theorem, Tychonoff theorem, and especially Baire category theorem. For the most part, topology chapters in analysis textbooks is more than enough!
- Topology Without Tears by Morris (2024) — intro with no prerequisites
- Introduction to Topology by Mendelson (1975) — Dover book: introduction, well-written and approachable
- Topology by Munkres (2000) — Classic standard topology book
- Introduction to Topology by Gamelin & Greene (1983) — Good reference book
- General Topology by Willard (1970) — Comprehensive graduate-level text
- Counterexamples in Topology by Steen & Seebach (1978)
- Metric Spaces by Magnus (2022) — Companion to analysis
- Measure and Category by Oxtoby (1980) — Advanced: Connects topological and measure spaces
Topology chapters in analysis books:
- Real Analysis and Probability by Dudley — Chapters 1-2, Appendix A/E
- Real Analysis by Folland — Chapter 4
- Principles of Mathematical Analysis by Rudin — Chapter 2
- Real Analysis by Royden — Has good chapters
- Measure Theory by Bogachev — Advanced and topologically flavoured
Note: This is for curiosity.
- Naive Set Theory by Halmos (1974) — Classic book on basic set theory
- Elements of Set Theory by Enderton (1977) — Rigorous but accessible undergraduate text
- Introduction to Set Theory by Hrbacek & Jech (1999) — Good intro and bridge to advance topics
- Set Theory by Jech (2003) — Advanced set theory, contains a lot
Probability Theory
- Probability with Martingales by Williams (1991) — Excellent first book. Then, follow up with more rigour
- A User’s Guide to Measure Theoretic Probability by Pollard (2001) — Really like it! Good for non-mathematicians
- Probability: Theory and Examples by Durrett (2019) — Good starting books
- Probability and Measure by Billingsley (1995) — Classic comprehensive grad school book
- Note: Avoid the 2012 Anniversary Edition (3rd ed retyped with many errors). Get the 2nd edition.
- Real Analysis and Probability by Dudley (2002) — Cassic book, dense. More for reference?
- Probability and Measure Theory by Ash (1999) — Has a chapter on functional analysis
- Probability and Stochastics by Çınlar (2011) — Explains sigma-algebra as notion of information very well
- Probability and Random Processes by Grimmett & Stirzaker (2020) — Companion: 1000 Exercises in Probability, goes beyond a course (martingales, random walks, Markov chains)
- Stanford CS229 Probability Theory Review
Inequalities:
- The Cauchy-Schwarz Master Class by Steele (2004) — Beautiful fun book on mathematical inequalities
- Inequalities by Hardy, Littlewood & Pólya (1988) — Classic, notation slightly outdated
Concentration Inequalities:
- Concentration Inequalities by Boucheron, Lugosi & Massart (2013) — Detailed! Misses some martingale topics
- An Introduction to Matrix Concentration Inequalities by Tropp (2015) — essential for ML
- Concentration of Measure for the Analysis of Randomised Algorithms by Dubhashi & Panconesi (2009)
- Concentration Inequalities by Clément Canonne (2022) gives the general scope in learning theory.
- The Concentration of Measure Phenomenon by Ledoux (2001) — Hard for first read. Reference book
- Concentration by McDiarmid (1998) — Survey
- Superconcentration and Related Topics by Chatterjee (2014)
- Concentration Inequalities by Gopalan & Tyagi — YouTube playlist
- Concentration of Measure by Wasserman
Note: Imperative to understand low-dimensional statistics and matrix inequalities first. This is a heavy topic.
- High Dimensional Probability by Vershynin (2018) — Free draft PDF, Video course (41 lectures), essential for modern data science. 2nd edition coming Summer 2025.
- High-Dimensional Statistics by Wainwright (2019) — Non-asymptotic viewpoint, rigorous
- Probability in High Dimension by van Handel (2016) — Free PDF, Princeton lecture notes
- High-Dimensional Data Analysis with Low-Dimensional Models by Wright & Ma (2022) — Free pre-production PDF
- Foundations of Data Science by Blum, Hopcroft & Kannan (2020) — Free PDF
- Upper and Lower Bounds for Stochastic Processes by Talagrand (2021) — 2nd edition, also fits concentration inequalities
- Probability in High Dimensions by Tropp (2023) — Lecture notes, Caltech ACM 217
- High-Dimensional Statistics by Giraud (2024) — Free PDF, Paris-Saclay lecture notes
- Lecture Notes on High-Dimensional Data by Wegner (2024) — Accessible introduction
- Bandit Algorithms by Lattimore & Szepesvári (2020) — Free PDF, Chapters 2,3,5,7,26 for probability
- A Second Course in Probability by Ross & Peköz — Chapters 1,3,4,5 recommended
Statistics
Introductory Statistics/Probability:
- Introduction to Probability by Blitzstein & Hwang (2019) — beautiful intro to probability
- Mathematical Statistics with Applications by Wackerly, Mendenhall & Scheaffer (2014) — Standard undergraduate text
- All of Statistics by Wasserman (2004) — Really good
Statistical Inference:
- Statistical Inference by Casella & Berger (2002) — Classic Book
- Computer Age Statistical Inference by Efron & Hastie (2016)
- Essential Statistical Inference by Boos & Stefanski (2013) — Some consider better than Casella
Mathematical Statistics:
- Advanced Calculus with Applications in Statistics by Khuri (2003) — Easier book on how math relates to stats
- Mathematical Statistics by Shao (2003) — Rigorous follow-up to Casella
- Testing Statistical Hypotheses by Lehmann & Romano (2022) — Standard PhD text
- Theory of Point Estimation by Lehmann & Casella (1998) — Companion to Testing Statistical Hypotheses
- Theory of Statistics by Schervish (1995)
- Mathematical Statistics: A Decision Theoretic Approach by Ferguson (1967) — Decision theory foundations
- Mathematical Statistics: Asymptotic Minimax Theory by Korostelev & Korosteleva (2011)
Courses:
Numerical Analysis:
- Numerical Linear Algebra by Trefethen & Bau (1997) — 40 self-contained lectures, essential for computational mathematics
- Numerical Analysis for Statisticians by Lange (2010) — Tailored to statisticians: eigenvalue decomposition, Newton-Raphson, EM/MM algorithms, splines
- Scientific Computing: An Introductory Survey by Heath (2018) — Lecture slides, very accessible
- Numerical Methods of Statistics by Monahan (2011) — Floating-point standards, RNG, sorting, FFT
- An Introduction to Numerical Analysis by Süli & Mayers (2003) — Oxford course, excellent balance of rigor and accessibility
- Numerical Mathematics by Quarteroni, Sacco & Saleri (2007) — MATLAB code, comprehensive
- Introduction to Numerical Analysis by Stoer & Bulirsch (2002) — Classic rigorous European-style text
- Numerical Recipes by Press et al. (2007) — Free 2nd edition online, practical cookbook with code
- MIT 18.335: Introduction to Numerical Methods (2019) — GitHub (active), rigorous MIT course
- fast.ai: Computational Linear Algebra (2017) — YouTube videos, top-down teaching with Python
Statistical Computing & Monte Carlo:
- Monte Carlo Statistical Methods by Robert & Casella (2004) — Slides, R programs, partial solutions, the definitive MCMC reference
- Non-Uniform Random Variate Generation by Devroye (1986) — Free PDF, rigorous treatment of random variable generation
- Statistical Computing with R by Rizzo (2019) — GitHub code, comprehensive R-based treatment
- Computational Statistics by Givens & Hoeting (2013) — Datasets, R code, lecture slides, balanced theory and practice
- Monte Carlo: Concepts, Algorithms and Applications by Fishman (1996) — Variance reduction, discrete event simulation, PRNGs
- Handbook of Monte Carlo Methods by Kroese, Taimre & Botev (2011) — MATLAB code, 772-page comprehensive reference
- Bayesian Data Analysis by Gelman et al. (2013) — Free PDF, standard Bayesian computation text
- Statistical Rethinking by McElreath (2020) — YouTube lectures (2023), R package, accessible Bayesian approach
- Modern Data Science with R by Baumer, Kaplan & Horton (2021) — Free online, practical R-based data science
- Duke STA663: Computational Statistics in Python — Complete course notes, MCMC, GPU computing
- Aalto BDA Course by Vehtari — Free BDA3 PDF, R demos, Python demos, excellent practical Bayesian course
Optimization:
- Convex Optimization by Boyd & Vandenberghe (2004) — Free PDF, Lecture slides, Stanford EE364A videos, THE standard reference
- Numerical Optimization by Nocedal & Wright (2006) — Line search, trust-region, conjugate gradient, quasi-Newton (BFGS), essential for research
- Algorithms for Optimization by Kochenderfer & Wheeler (2019) — Free PDF, Julia code, modern and practical
- Introduction to Stochastic Search and Optimization by Spall (2003) — SGD, simulated annealing, genetic algorithms
- Introductory Lectures on Convex Optimization by Nesterov (2004) — Essential for understanding accelerated methods
- Nonlinear Programming by Bertsekas (2016) — Deep theoretical treatment, 880 pages
- Optimization for Data Analysis by Wright & Recht (2022) — Modern treatment focused on ML
- Convex Optimization: Algorithms and Complexity by Bubeck (2015) — Free monograph, theoretical complexity focus
- CMU 10-725: Convex Optimization by Tibshirani (2019) — Slides, scribed notes, homework, ML-focused
- UC Berkeley EE227C: Convex Optimization and Approximation (2018) — Course notes, Julia/Python, theory-focused
- Asymptotic Statistics by Van der Vaart (2000) — Advanced statistics with functional analysis topics
- All of Statistics by Wasserman (2004) — Comprehensive overview
Learning
Note: I don’t have any posted resources on Large Language Models (LLM) since the field is changing very fast.
- ISL: Introduction to Statistical Learning by James et al. (2023) — Online course/Labs, Best intro to statistical ML
- ESL: Elements of Statistical Learning by Hastie, Tibshirani & Friedman (2017) — More advanced to ISL
- Hands-On ML with Scikit-Learn, Keras & TensorFlow by Géron (2022) — GitHub notebooks, practical coding focus
- Probabilistic ML: Introduction and Advanced Topics by Murphy (2023) — More methods than proofs, CS-oriented
- Pattern Recognition and Machine Learning by Bishop (2006) — A bit math oriented but not much
- Bayesian Reasoning and Machine Learning by Barber (2012) — good exercises
- Artificial Intelligence: A Modern Approach by Russell & Norvig (2021) — Broad AI reference
- Mathematics for Machine Learning by Deisenroth, Faisal & Ong (2020) — Videos (San Diego Book Club 2024)
Courses: There are many courses on Machine Learning. I found interesting based on the researchers I connect with!
- CMU 10-202: Introduction to Modern AI by Zico Kolter(2026)
- Machine Learning Mastery — Practical coding tutorials
- Learning Theory from First Principles by Bach (2024) — Blog Course/Schedule Written with rigour!
- Mathematical Methods in Data Science by Roch (2025) — Accompanying UChicago course and videos
- Understanding Machine Learning by Shalev-Shwartz & Ben-David (2014) — Videos, Solutions, Beautiful Videos
- Patterns, Predictions, and Actions by Hardt & Recht (2022) — Free PDF, Problem sets, Course and Blog (2025)
- Mathematical Analysis of ML Algorithms by Zhang (2023) — Slides, Complementary Csaba Szepesvari videos
- Foundations of Machine Learning by Mohri et al. (2018)
- Statistical Learning with Sparsity by Hastie, Tibshirani & Wainwright (2015) — Lasso and sparsity focus
- Mathematical Foundations of ML by Nowak (2022)
Courses: There are many courses, here are some I found intersting based on my interest.
- Advanced Topics in Statistical ML by Tibshirani (2024) — Lecture notes
- UChicago Mathematical Foundations of ML (2025) — Really good lecture videos! Based on Roch (2025)
- Reproducing Kernel Hilbert Spaces in ML by Gretton (2025) — UCL/Gatsby course, comprehensive RKHS coverage
Note: Better resources may exist, but Sutton book is my go to.
- Reinforcement Learning: An Introduction by Sutton & Barto (2018)
- Reinforcement Learning: An Overview by Kevin Murphy (2025)
- Markov Decision Processes by Puterman (2025)
- Algorithms for Reinforcement Learning by Szepesvári (2010)
Courses:
- David Silver’s RL Course (2015) — 10 lectures, complement to Sutton & Barto
- UC Berkeley CS285: Deep RL by Levine (2023) — 23 lectures on deep RL
- Stanford CS234: Reinforcement Learning by Brunskill
- Reinforcement Learning from Human Feedback by Nathan Lambert (2025) — RLHF!!
- Spinning Up in Deep RL by OpenAI (2018) — Practical implementations (VPG, TRPO, PPO, DDPG, TD3, SAC)
- Hugging Face Deep RL Course (2022)
- CleanRL — Implementations of RL algorithms
- Learning Deep Representations of Data (2025) — Really good second book!
- Understanding Deep Learning by Prince (2023) — One of the best books
- The Little Book of Deep Learning by Fleuret (2024) — Perfect for phone reading
- Deep Learning by Goodfellow, Bengio & Courville (2016) — Classic, a bit outdated (no transformers, RLHF, diffusion, etc.)
- Deep Learning: Foundations and Concepts by Bishop & Bishop (2024) — Updated, includes transformers
- Deep Learning with PyTorch by Stevens, Antiga & Viehmann (2020) — Really good hands-on book
Practical Deep Learning:
- PyTorch Documentation — Best resource
- Learn PyTorch — Practical tutorials
- Dive into Deep Learning by Zhang et al. (2023) — Interactive book with code, updated with latest works
Courses and Videos:
- 3Blue1Brown Neural Networks — Visual intro to neural networks
- Neural Networks: Zero to Hero by Karpathy — Build neural networks from scratch
- MIT 6.S191: Introduction to Deep Learning (2026) — Light after reading about DL
- UIUC Deep Learning Theory Lecture Notes by Matus Telgarsky (2023)
- Waterloo Deep Learning Theory Lecture Videos by Ali Ghodsi (2023) — really good!
- The Principles of Deep Learning Theory by Roberts, Yaida & Hanin (2022) — Cambridge University Press
- Simons Institute: Foundations of Deep Learning (2019) — Lecture series
Discrete Mathematics
- Introduction to the Theory of Computation by Sipser (2012) — The standard textbook, clear exposition, excellent problems
- Graph Theory by Diestel (2025) — Free online, 6th edition, graduate level, covers infinite graphs
- Introduction to Graph Theory by West (2001) — Comprehensive, widely used for graduate courses
- Graph Theory by Bondy & Murty (2008) — Rigorous, excellent exercises
- Enumerative Combinatorics Vol 1 & 2 by Stanley (2023) — Supplementary materials, advanced, encyclopedic reference
- A Course in Combinatorics by van Lint & Wilson (2001) — Classic graduate text
- Combinatorial Problems and Exercises by Lovász (2007) — Excellent problem collection