Cheatsheet: Mathematics for Machine Learning 3: Probability and Statistics for Data-Driven Models
A lookup card for the whole course. Every formula is tied to a concrete dataset question.
Module 1: Probability Foundations for Data
1.1 Axioms, Conditional Probability and Independence
The three Kolmogorov axioms (event space $\Omega$, probability measure $P$):
| # | Axiom | Statement |
|---|---|---|
| 1 | Non-negativity | $P(A) \ge 0$ for every event $A$ |
| 2 | Normalization | $P(\Omega) = 1$ |
| 3 | Additivity | If $A, B$ disjoint, $P(A \cup B) = P(A) + P(B)$ |
Consequences you use constantly:
- $P(A^c) = 1 - P(A)$
- $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
- $P(\emptyset) = 0$
Conditional probability: $$ P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0 $$
Multiplication rule: $P(A \cap B) = P(A \mid B),P(B) = P(B \mid A),P(A)$
Independence: $A \perp B$ iff $P(A \cap B) = P(A)P(B)$, equivalently $P(A \mid B) = P(A)$.
Warning: Disjoint (mutually exclusive) is NOT independent. If $P(A)>0$ and $P(B)>0$ and they are disjoint, then $P(A \mid B) = 0 \ne P(A)$, so they are dependent.
Law of Total Probability (partition $B_1, \dots, B_k$ of $\Omega$): $$ P(A) = \sum_{i=1}^{k} P(A \mid B_i),P(B_i) $$
Dataset question: In sessions.csv, what fraction of sessions convert given the user arrived from a paid ad? → condition on the channel column.
1.2 Expectation and Variance on a Real Dataset
Random variable: a function from outcomes to numbers (e.g. order_value per session).
| Quantity | Discrete | Continuous |
|---|---|---|
| Expectation | $E[X] = \sum_x x,P(X{=}x)$ | $E[X] = \int x,f(x),dx$ |
| Variance | $\mathrm{Var}(X) = E[(X - \mu)^2]$ | same definition |
Shortcut form: $\mathrm{Var}(X) = E[X^2] - (E[X])^2$
Linearity of expectation (always holds, no independence needed): $$ E[aX + bY + c] = aE[X] + bE[Y] + c $$
This cheatsheet is premium content. Sign in and purchase the course to view the full content. Sign in