Axioms, Conditional Probability and Independence
Every later lesson in this course (expectation, the named distributions, Bayes' rule, confidence intervals) is assembled from three rules thatKolmogorov stated them in 1933, and they still carry the whole course. Get these right and later formulas look inevitable; skip them and everything looks like magic.
By the end of this lesson you can:
- Turn a plain-language dataset question into a sample space and named events
- Check the three axioms on a two-way count table
- Compute a conditional probability and explain why the denominator changes
- Test independence with the multiplication rule instead of eyeballing percentages
- Spot the two traps that break real analyses: flipping the condition, and calling disjoint events independent
Our running dataset is sessions.csv: 10,000 rows, one row per website session, with columns channel, device, converted, order_value. The question we want to answer: do mobile visitors convert at a lower rate than desktop visitors? Every idea below is a tool for answering exactly that.
From a dataset question to a sample space
An outcome is one row of the file. The sample space $\Omega$ is the set of all 10,000 sessions. An event is a subset of $\Omega$: $A = {\text{mobile sessions}}$, $B = {\text{converted sessions}}$.
Probability is then just a ratio of counts, $P(A) = |A| / |\Omega|$, which is why the axioms are so easy to verify on a table. One caution: the sample space is a modeling choice. "All sessions in this file" is not "all visitors who will ever arrive", and a probability statement is only as precise as the space it lives in.
The three axioms, and what they buy you
For events $A$ and $B$ in $\Omega$:
- Non-negativity: $P(A) \ge 0$
- Normalization: $P(\Omega) = 1$
- Additivity: if $A \cap B = \emptyset$, then $P(A \cup B) = P(A) + P(B)$
Almost everything else is derived. The complement rule comes from axioms 2 and 3 applied to $A$ and $A^c$: $P(A^c) = 1 - P(A)$. The general addition rule comes from splitting $A \cup B$ into disjoint pieces:
$$ P(A \cup B) = P(A) + P(B) - P(A \cap B) $$
The converted column is a 0/1 variable, so $P(\text{converted}) = 600/10000 = 0.06$, and $P(\text{not converted}) = 0.94$ by the complement rule, no counting required. Here is the full table:
| Converted | Not converted | Total | |
|---|---|---|---|
| Mobile | 180 | 3,820 | 4,000 |
| Desktop | 420 | 5,580 | 6,000 |
| Total | 600 | 9,400 | 10,000 |
Worked example (union). $P(\text{mobile} \cup \text{converted}) = 0.40 + 0.06 - 0.018 = 0.442$. Check by counting: $4000 + 600 - 180 = 4420$, and $4420/10000 = 0.442$. Subtracting the intersection is the step people skip, and it is the difference between a correct answer and a probability above 0.46 for no reason.
Conditional probability: shrinking the sample space
The question "do mobile visitors convert less?" is not $P(\text{converted})$. It is $P(\text{converted} \mid \text{mobile})$: among mobile sessions only, what fraction converted?
$$ P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0 $$
The intuition is mechanical: conditioning on $B$ throws away every row outside $B$ and renormalizes, so the denominator becomes the size of $B$. From the table, $P(\text{converted} \mid \text{mobile}) = 180/4000 = 0.045$ and $P(\text{converted} \mid \text{desktop}) = 420/6000 = 0.07$. Mobile converts at 4.5%, desktop at 7%, a relative lift of about 0.64. Question answered.
Rearranging the definition gives the multiplication rule, $P(A \cap B) = P(B),P(A \mid B)$, which chains through funnels. If 25% of sessions add to cart and 12% of those sessions purchase, then $P(\text{cart} \cap \text{purchase}) = 0.25 \times 0.12 = 0.03$. Extending the chain to three steps, $P(V \cap C \cap P) = P(V),P(C \mid V),P(P \mid V \cap C)$, is how conversion funnels are actually estimated.
Independence: when conditioning changes nothing
Events $A$ and $B$ are independent when
$$ P(A \cap B) = P(A),P(B) $$
which, when $P(B) > 0$, is the same as $P(A \mid B) = P(A)$. Knowing $B$ occurred tells you nothing about $A$.
Worked example (dependent). Device and conversion: $P(\text{mobile}),P(\text{converted}) = 0.40 \times 0.06 = 0.024$, but the joint probability is $0.018$. The product rule fails, so the two are dependent, consistent with 4.5% versus 7%.
Worked example (independent). Suppose 800 of 4,000 mobile sessions and 1,200 of 6,000 desktop sessions signed up for the newsletter. Both rates are 0.20, the overall rate is $2000/10000 = 0.20$, and $P(\text{mobile} \cap \text{signup}) = 0.08 = 0.40 \times 0.20$. Device and signup are independent, so pooling them is safe for that question and not for conversion.
Independence also powers repeated trials: if sessions convert independently, three specific sessions all converting has probability $0.06^3 \approx 0.000216$. That multiplication is the seed of the binomial distribution in Module 2.
Warning: Disjoint and independent are close to opposites. If $A$ and $B$ are disjoint and both have positive probability, then $P(A \cap B) = 0$ while $P(A)P(B) > 0$, so they are dependent, and strongly so: seeing $A$ guarantees $B$ did not happen.
Fill in the blanks before moving on
The axioms require P(A) _____, P(Ω) _____, and P(A ∪ B) = P(A) + P(B) whenever A and B are _____. The complement rule states that P(A^c) = _____. Conditional probability is defined as P(A | B) = _____. Events A and B are independent exactly when P(A ∩ B) = _____.
Traps that quietly corrupt a real analysis
- Flipping the condition. 4.5% of mobile sessions convert, but 30% of conversions come from mobile ($180/600$). Same table, different questions. This is the confusion Bayes' rule resolves in section 2.3.
- Assuming independence because the rates look similar. Two decimals can hide a real gap. Compute the product of the marginals and compare it to the joint count.
- Moving the wrong number. Conditioning on $B$ changes the denominator only. The numerator stays the joint count.
- Forgetting to subtract the overlap in $P(A \cup B)$, which produces probabilities that are too large.
- Symbols before words. Most wrong answers come from a mis-specified event ("converted mobile" versus "converted, given mobile"), not from arithmetic. Write the event in a sentence, then translate it.
What to carry forward
- The sample space is the denominator you are implicitly dividing by; conditioning chooses a smaller one.
- Three axioms give you the complement rule, the addition rule, and everything downstream.
- Independence is a testable claim, $P(A \cap B) = P(A)P(B)$, not an assumption.
- A conditional probability answers a sharper question than a marginal one, and dataset questions are almost always conditional.
Front: What is the sample space of an experiment? Back: The set of all possible outcomes; every event is a subset of it.
Front: What is an event? Back: A subset of the sample space whose outcomes satisfy some condition.
Front: State the three probability axioms. Back: P(A) ≥ 0; P(Ω) = 1; P(A ∪ B) = P(A) + P(B) when A and B are disjoint.
Front: Complement rule Back: P(A^c) = 1 − P(A), derived from normalization plus additivity on disjoint A and A^c.
Front: Addition rule for two overlapping events Back: P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
Front: Definition of conditional probability Back: P(A | B) = P(A ∩ B) / P(B), defined when P(B) > 0.
Front: What changes when you condition on B? Back: The sample space shrinks to B, so every probability is divided by P(B).
Front: Multiplication rule Back: P(A ∩ B) = P(A) P(B | A) = P(B) P(A | B).
Front: When are A and B independent? Back: When P(A ∩ B) = P(A) P(B), equivalently P(A | B) = P(A) if P(B) > 0.
Front: In the sessions table, P(converted) = 0.06 and P(converted | mobile) = 0.045. Independent? Back: No. Conditioning changed the probability, so conversion depends on device.
Front: Why are disjoint events with positive probability dependent? Back: P(A ∩ B) = 0 but P(A)P(B) > 0, so the product rule fails; observing A rules out B.
Front: How do P(mobile | converted) and P(converted | mobile) differ on this table? Back: 0.30 versus 0.045; same joint count, different denominators.
Front: How do you get P(A | B) from a two-way count table? Back: Divide the joint count by the row or column total for B, not by the grand total.
Next up: Expectation and Variance on a Real Dataset, where the same sessions.csv table becomes a single summary number for order_value and a second number for how far it typically strays.