Mathematics for Machine Learning
(Ulrike von Luxburg, Winter term 2020/21)

Quick links

Material and assignments

Lecture notes:
  • Linear algebra (A): pdf
  • Calculus (C): pdf
  • Probability theory (P): pdf
  • Statistics (S): pdf
  • Mixed materials (H): pdf


Lectures (public on youtube): Please watch the indicated lectures for each week. They contain the material that you need to solve the assignments.
  • Week starting Nov 2: Videos A.1 - A.7
  • Week starting Nov 9: Videos A.8 - A.13
  • Week starting Nov 16: Videos A.14 - A.19
  • Week starting Nov 23: Videos A.20 - A.24
  • Week starting Nov 30: Videos A.25 - A.31; Voluntary add-on: A.32 and A.33
  • Week starting Dec 7: Videos C.1 - C.3 and C.7+8. Voluntary recap: C.4-C.6
  • Week starting Dec 14: Videos C.9-C.11; C.14-C.19. Voluntary add-on: C.12 and C.13
  • Week starting Jan 11: Videos P.1 and P.4 - P.12
  • Week starting Jan 18: Videos P.2, P.3, and P.13 - P.16
  • Week starting Jan 25: Videos P.17- P.21
  • Week starting Feb 1: Videos S.1 - S.4
  • Week starting Feb 8: Videos S.5 - S.8
  • Week starting Feb 15: Videos P 17a (just the theorem; proof optional), S.9 - S.11
  • Week starting Feb 22: Videos on Optimization: Convex optimization 1 Convex optimization 2 Video on high-dimensional geometry: H 1
Assignments):

Background information

This course is intended for master students who plan to dive further in machine learning. Depending on your background, much of the material might be a recap - or not. Contents of the course are Linear algebra, Mulitvariate analysis, Probability Theory, Statistics, Optimization.

Lectures

Lectures are being held by Ulrike von Luxburg and will be provided on youtube. For each week, we will publish a list that tells you which videos you are supposed to watch. If necessary, we might also offer inverted lectures via zoom, in which you can ask questions. These lectures would take place Thursdays 8:30-9:30. We will inform all registered participants by email about the dates and links.


Tutorials

We will have weekly tutorial sessions in small groups of about 20-30 students, where you can ask questions and interact with other students. You will be able to enter your preferences regarding the time when you register for the tutorials. The teaching assitants are:

Assignments

You will get weekly assignments that you have to solve in groups of two students. Achieving half of the possible points is a formal requirement for being admitted to the exam.

Exams

The current plan is as follows (with uncertainty, as we need to wait what the Covid situation and the university regulations will allow us to do): The final exams will take place on-site in Tuebingen, and you need to be physically present. There is going to be one exam at the beginning of the semester break and one at the end of the semester break. You can choose which exam to take. However, please note that in case you miss the exams, you cannot simply take an oral exam instead, you will have to wait until next year’s exams take place.

The general mode for exams is: You are not allowed to bring any material (books, slides, etc) except for what we call the controlled cheat sheet: one side (A4, one side only) of handwritten (!) notes, made by yourself. This cheat sheet has to be handed in together with the exam (but will not be graded of course).

  • Information sheet about the lecture.

    • Literature:

    General:
    • Lecture notes by Matthias Hein, who has taught the course last year. Similar to what I will do this year, not completely identical.
    • Deisenroth, Faisal, Ong: Mathematics for Machine Learning, 2019. Not as deep as what we do in this class, but a good start.
    More specific:
    • For linear algebra, I recommend: Sheldon Axler: Linear Algebra Done Right. Third edition, 2015. There are also online videos by the author if you want to get longer explanations than the ones I will provide.
    • Calculus (Integration, Measures, Metric spaces and their topology): Sheldon Axler: Measure, Integration & Real Analysis. 2019
    • Calculus (Differential calculus in R^n): Here I haven't found my favorite english textbook yet. Below are some references, but the first one is slightly to recipe-like, the other too abstract. Still watching out for a good compromise...
      • Books with many figures, but partly informal or recipe-like. Might be good as a start if you need to get the intuition before diving deeper:
        Stanley Miklavcic: An Illustrative Guide to Multivariable and Vector Calculus.
        Charles Pugh: Real Mathematical Analysis
      • Mathematically rigorous, but not easy to read:
        Terence Tao, Analysis 1 and 2. (just discovered it, love it!).
        Rudin: Principles of Mathematical Analysis. (A classic, sometimes called the Baby-Rudin).
      • Calculus, a german book I like: Walter: Analysis 1 and Analysis 2. The second one covers everything that we have been discussing.
    • Probability theory: Jacod, Protter: Probability essentials. Short and to the point, tries to avoid measure theory whereever possible, yet is rigorous. Good compromise.
    • Statistics:
      • For a very short overview over all the topics we cover: Wasserman: All of statistics, a concise course in statisticial inferece.
      • A bit more details: Casella/Berger, Statistical Inference.
      • Testing, rigorously: Lehmann/Romano: Testing statistical hypotheses.
    • For high-dimensional probability and statistics there are several good books, but they go much deeper than our lecture:
      • Wainwritght: High-dimensional statistics
      • Vershynin: High-dimensional probability
      • Bühlmann, van de Geer: Statistics for High-dimensional data (this is from the more traditional statitics point of view)

    Online feedback form

    We want to know what you like / do not like about the lecture! You can tell us anonymously with the following feedback form. The more concrete and constructive the feedback, the higher the likelihood that we can adapt to it.