Mathematics for Machine Learning
(Ulrike von Luxburg, Winter term 2020/21)
Quick links

Youtube channel for the videos

moodle to hand in assignments
Exam
We now finally received the allocation for our exams: Confirmed: March 10, 9:3011:30, N7, Morgenstelle
 Confirmed: April 12, 12:30  15:00, N10, Morgenstelle
The exam covers all the material up to and inlcuding the week of Feburary 8 (that is, up to video S.8). Assignment 13 below is a test exam, so you know what type of questions we are going to ask.
Exam results
You can find the results for the first exam here, for which this grading key was used.You can find the results for the second exam here, for which this grading key was used.
Material and assignments
Lecture notes: Lecture notes by myself, handwritten, publically
available. As I've been teaching this course for the first
time, the notes contain lots of errors and typos. If you
find some, please let me know by email, I'll fix it
(please indicate the file and the page number).

Linear algebra (A): pdf

Calculus (C): pdf
 Probability theory (P):
pdf
 Statistics (S):
pdf
 Mixed materials (H):
pdf
 Lecture notes by Matthias Hein (password protected, not for public use)
Lectures (public on youtube): Please watch the indicated lectures for each week. They contain the material that you need to solve the assignments.
 Week starting Nov 2: Videos A.1  A.7
 Week starting Nov 9: Videos A.8  A.13
 Week starting Nov 16: Videos A.14  A.19
 Week starting Nov 23: Videos A.20  A.24
 Week starting Nov 30: Videos A.25  A.31; Voluntary addon: A.32 and A.33
 Week starting Dec 7: Videos C.1  C.3 and C.7+8. Voluntary recap: C.4C.6
 Week starting Dec 14: Videos C.9C.11; C.14C.19. Voluntary addon: C.12 and C.13
 Week starting Jan 11: Videos P.1 and P.4  P.12
 Week starting Jan 18: Videos P.2, P.3, and P.13  P.16
 Week starting Jan 25: Videos P.17 P.21
 Week starting Feb 1: Videos S.1  S.4
 Week starting Feb 8: Videos S.5  S.8
 Week starting Feb 15: Videos P 17a (just the theorem; proof optional), S.9  S.11
 Week starting Feb 22: Videos on Optimization: Convex optimization 1 Convex optimization 2 Video on highdimensional geometry: H 1
 Assignment 1, due Nov 9. Solutions for Assignment 1
 Assignment 2, due Nov 16. Solutions for Assignment 2
 Assignment 3, due Nov 23. Solutions for Assignment 3
 Assignment 4, due Nov 30. Solutions for Assignment 4
 Assignment 5, due Dec 07. Solutions for Assignment 5
 Assignment 6, due Dec 14. Solutions for Assignment 6
 Assignment 7, due Dec 21. Solutions for Assignment 7
 Assignment 8, due Jan 18. Solutions for Assignment 8
 Assignment 9, due Jan 25. Solutions for Assignment 9
 Assignment 10, due Feb 01. Solutions for Assignment 10
 Assignment 11, due Feb 08. Solutions for Assignment 11
 Assignment 12, questions.tex, due Feb 15. The questions provided by the students can be found here. Solutions for Assignment 12
 Assignment 13, due Feb 22. Solutions for Assignment 13
Background information
This course is intended for master students who plan to dive further in machine learning. Depending on your background, much of the material might be a recap  or not. Contents of the course are Linear algebra, Mulitvariate analysis, Probability Theory, Statistics, Optimization.
Lectures
Lectures are being held by Ulrike von Luxburg and will be provided on youtube. For each week, we will publish a list that tells you which videos you are supposed to watch. If necessary, we might also offer inverted lectures via zoom, in which you can ask questions. These lectures would take place Thursdays 8:309:30. We will inform all registered participants by email about the dates and links.
Tutorials
We will have weekly tutorial sessions in small groups of about 2030 students, where you can ask questions and interact with other students. You will be able to enter your preferences regarding the time when you register for the tutorials. The teaching assitants are: Luca Rendsburg
 Sebastian Bordt (group 1)
 Solveig Klepper (group 2)
 Alexander Conzelmann (group 3)
Assignments
You will get weekly assignments that you have to solve in groups of two students. Achieving half of the possible points is a formal requirement for being admitted to the exam.Exams
The current plan is as follows (with uncertainty, as we need to wait what the Covid situation and the university regulations will allow us to do): The final exams will take place onsite in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. You can choose which exam to take. However, please note that in case you miss the exams, you cannot simply take an oral exam instead, you will have to wait until next year’s exams take place.The general mode for exams is: You are not allowed to bring any material (books, slides, etc) except for what we call the controlled cheat sheet: one side (A4, one side only) of handwritten (!) notes, made by yourself. This cheat sheet has to be handed in together with the exam (but will not be graded of course).
Literature:
General: Lecture notes by Matthias Hein, who has taught the course last year. Similar to what I will do this year, not completely identical.
 Deisenroth, Faisal, Ong: Mathematics for Machine Learning, 2019. Not as deep as what we do in this class, but a good start.
 For linear algebra, I recommend: Sheldon Axler: Linear Algebra Done Right. Third edition, 2015. There are also online videos by the author if you want to get longer explanations than the ones I will provide.
 Calculus (Integration, Measures, Metric spaces and their topology): Sheldon Axler: Measure, Integration & Real Analysis. 2019
 Calculus (Differential calculus in R^n): Here I haven't found
my favorite english textbook yet. Below are some references, but the
first one is slightly to recipelike, the other too abstract. Still
watching out for a good compromise...
 Books with many figures, but partly informal or recipelike. Might be good as a start if you need to get the intuition before diving deeper:
Stanley Miklavcic: An Illustrative Guide to Multivariable and Vector Calculus.
Charles Pugh: Real Mathematical Analysis  Mathematically rigorous, but not easy to read:
Terence Tao, Analysis 1 and 2. (just discovered it, love it!).
Rudin: Principles of Mathematical Analysis. (A classic, sometimes called the BabyRudin).  Calculus, a german book I like: Walter: Analysis 1 and Analysis 2. The second one covers everything that we have been discussing.
 Books with many figures, but partly informal or recipelike. Might be good as a start if you need to get the intuition before diving deeper:
 Probability theory: Jacod, Protter: Probability essentials. Short and to the point, tries to avoid measure theory whereever possible, yet is rigorous. Good compromise.
 Statistics:
 For a very short overview over all the topics we cover: Wasserman: All of statistics, a concise course in statisticial inferece.
 A bit more details: Casella/Berger, Statistical Inference.
 Testing, rigorously: Lehmann/Romano: Testing statistical hypotheses.
 For highdimensional probability and statistics there are several good books, but they go
much deeper than our lecture:
 Wainwritght: Highdimensional statistics
 Vershynin: Highdimensional probability
 Bühlmann, van de Geer: Statistics for Highdimensional data (this is from the more traditional statitics point of view)