# 🧩 Axioms, Conditional Probability and Independence

By Better Wise

Every later lesson in this course (expectation, the named distributions, Bayes' rule, confidence intervals) is assembled from three rules thatKolmogorov stated them in 1933, and they still carry the whole course. Get these right and later formulas look inevitable; skip them and everything looks like magic.

By the end of this lesson you can:

- Turn a plain-language dataset question into a sample space and named events
- Check the three axioms on a two-way count table
- Compute a conditional probability and explain why the denominator changes
- Test independence with the multiplication rule instead of eyeballing percentages
- Spot the two traps that break real analyses: flipping the condition, and calling disjoint events independent

Our running dataset is `sessions.csv`: 10,000 rows, one row per website session, with columns `channel`, `device`, `converted`, `order_value`. The question we want to answer: **do mobile visitors convert at a lower rate than desktop visitors?** Every idea below is a tool for answering exactly that.

## From a dataset question to a sample space

An **outcome** is one row of the file. The **sample space** $\Omega$ is the set of all 10,000 sessions. An **event** is a subset of $\Omega$: $A = \{\text{mobile sessions}\}$, $B = \{\text{converted sessions}\}$.

Probability is then just a ratio of counts, $P(A) = |A| / |\Omega|$, which is why the axioms are so easy to verify on a table. One caution: the sample space is a modeling choice. "All sessions in this file" is not "all visitors who will ever arrive", and a probability statement is only as precise as the space it lives in.

## The three axioms, and what they buy you

For events $A$ and $B$ in $\Omega$:

1. **Non-negativity:** $P(A) \ge 0$
2. **Normalization:** $P(\Omega) = 1$
3. **Additivity:** if $A \cap B = \emptyset$, then $P(A \cup B) = P(A) + P(B)$

Almost everything else is derived. The complement rule comes from axioms 2 and 3 applied to $A$ and $A^c$: $P(A^c) = 1 - P(A)$. The general addition rule comes from splitting $A \cup B$ into disjoint pieces:

$$
P(A \cup B) = P(A) + P(B) - P(A \cap B)
$$

The `converted` column is a 0/1 variable, so $P(\text{converted}) = 600/10000 = 0.06$, and $P(\text{not converted}) = 0.94$ by the complement rule, no counting required. Here is the full table:

| | Converted | Not converted | Total |
| --- | --- | --- | --- |
| Mobile | 180 | 3,820 | 4,000 |
| Desktop | 420 | 5,580 | 6,000 |
| Total | 600 | 9,400 | 10,000 |

**Worked example (union).** $P(\text{mobile} \cup \text{converted}) = 0.40 + 0.06 - 0.018 = 0.442$. Check by counting: $4000 + 600 - 180 = 4420$, and $4420/10000 = 0.442$. Subtracting the intersection is the step people skip, and it is the difference between a correct answer and a probability above 0.46 for no reason.

## Conditional probability: shrinking the sample space

The question "do mobile visitors convert less?" is not $P(\text{converted})$. It is $P(\text{converted} \mid \text{mobile})$: among mobile sessions only, what fraction converted?

$$
P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0
$$

The intuition is mechanical: conditioning on $B$ throws away every row outside $B$ and renormalizes, so the denominator becomes the size of $B$. From the table, $P(\text{converted} \mid \text{mobile}) = 180/4000 = 0.045$ and $P(\text{converted} \mid \text{desktop}) = 420/6000 = 0.07$. Mobile converts at 4.5%, desktop at 7%, a relative lift of about 0.64. Question answered.

Rearranging the definition gives the **multiplication rule**, $P(A \cap B) = P(B)\,P(A \mid B)$, which chains through funnels. If 25% of sessions add to cart and 12% of those sessions purchase, then $P(\text{cart} \cap \text{purchase}) = 0.25 \times 0.12 = 0.03$. Extending the chain to three steps, $P(V \cap C \cap P) = P(V)\,P(C \mid V)\,P(P \mid V \cap C)$, is how conversion funnels are actually estimated.

## Independence: when conditioning changes nothing

Events $A$ and $B$ are **independent** when

$$
P(A \cap B) = P(A)\,P(B)
$$

which, when $P(B) > 0$, is the same as $P(A \mid B) = P(A)$. Knowing $B$ occurred tells you nothing about $A$.

**Worked example (dependent).** Device and conversion: $P(\text{mobile})\,P(\text{converted}) = 0.40 \times 0.06 = 0.024$, but the joint probability is $0.018$. The product rule fails, so the two are dependent, consistent with 4.5% versus 7%.

**Worked example (independent).** Suppose 800 of 4,000 mobile sessions and 1,200 of 6,000 desktop sessions signed up for the newsletter. Both rates are 0.20, the overall rate is $2000/10000 = 0.20$, and $P(\text{mobile} \cap \text{signup}) = 0.08 = 0.40 \times 0.20$. Device and signup are independent, so pooling them is safe for that question and not for conversion.

Independence also powers repeated trials: if sessions convert independently, three specific sessions all converting has probability $0.06^3 \approx 0.000216$. That multiplication is the seed of the binomial distribution in Module 2.

> **Warning:** 
> **Disjoint and independent are close to opposites.** If $A$ and $B$ are disjoint and both have positive probability, then $P(A \cap B) = 0$ while $P(A)P(B) > 0$, so they are dependent, and strongly so: seeing $A$ guarantees $B$ did not happen.

## Fill in the blanks before moving on

The axioms require P(A) _____, P(Ω) _____, and P(A ∪ B) = P(A) + P(B) whenever A and B are _____.
The complement rule states that P(A^c) = _____.
Conditional probability is defined as P(A | B) = _____.
Events A and B are independent exactly when P(A ∩ B) = _____.

## Traps that quietly corrupt a real analysis

- **Flipping the condition.** 4.5% of mobile sessions convert, but 30% of conversions come from mobile ($180/600$). Same table, different questions. This is the confusion Bayes' rule resolves in section 2.3.
- **Assuming independence because the rates look similar.** Two decimals can hide a real gap. Compute the product of the marginals and compare it to the joint count.
- **Moving the wrong number.** Conditioning on $B$ changes the denominator only. The numerator stays the joint count.
- **Forgetting to subtract the overlap** in $P(A \cup B)$, which produces probabilities that are too large.
- **Symbols before words.** Most wrong answers come from a mis-specified event ("converted mobile" versus "converted, given mobile"), not from arithmetic. Write the event in a sentence, then translate it.

## What to carry forward

- The sample space is the denominator you are implicitly dividing by; conditioning chooses a smaller one.
- Three axioms give you the complement rule, the addition rule, and everything downstream.
- Independence is a testable claim, $P(A \cap B) = P(A)P(B)$, not an assumption.
- A conditional probability answers a sharper question than a marginal one, and dataset questions are almost always conditional.

Front: What is the sample space of an experiment?
Back: The set of all possible outcomes; every event is a subset of it.

Front: What is an event?
Back: A subset of the sample space whose outcomes satisfy some condition.

Front: State the three probability axioms.
Back: P(A) ≥ 0; P(Ω) = 1; P(A ∪ B) = P(A) + P(B) when A and B are disjoint.

Front: Complement rule
Back: P(A^c) = 1 − P(A), derived from normalization plus additivity on disjoint A and A^c.

Front: Addition rule for two overlapping events
Back: P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Front: Definition of conditional probability
Back: P(A | B) = P(A ∩ B) / P(B), defined when P(B) > 0.

Front: What changes when you condition on B?
Back: The sample space shrinks to B, so every probability is divided by P(B).

Front: Multiplication rule
Back: P(A ∩ B) = P(A) P(B | A) = P(B) P(A | B).

Front: When are A and B independent?
Back: When P(A ∩ B) = P(A) P(B), equivalently P(A | B) = P(A) if P(B) > 0.

Front: In the sessions table, P(converted) = 0.06 and P(converted | mobile) = 0.045. Independent?
Back: No. Conditioning changed the probability, so conversion depends on device.

Front: Why are disjoint events with positive probability dependent?
Back: P(A ∩ B) = 0 but P(A)P(B) > 0, so the product rule fails; observing A rules out B.

Front: How do P(mobile | converted) and P(converted | mobile) differ on this table?
Back: 0.30 versus 0.045; same joint count, different denominators.

Front: How do you get P(A | B) from a two-way count table?
Back: Divide the joint count by the row or column total for B, not by the grand total.

Next up: **Expectation and Variance on a Real Dataset**, where the same `sessions.csv` table becomes a single summary number for `order_value` and a second number for how far it typically strays.

---

## Contents

Part of [🎲 Mathematics for Machine Learning 3: Probability and Statistics for Data-Driven Models](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data.md)

- **🎲 Probability Foundations for Data**
  - [🧩 Axioms, Conditional Probability and Independence](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/probability-axioms-conditional-independence.md) — this lesson
  - [📊 Expectation and Variance on a Real Dataset](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/expectation-variance-worked-dataset.md) (sign-in required)
  - [🔗 Covariance, Correlation and Dependence](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/covariance-correlation-dependence.md)
  - [✅ Checkpoint: Probability Foundations](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/checkpoint-probability-foundations.md)
- **📈 Distributions and Bayes' Rule**
  - [🔢 Bernoulli, Binomial and Poisson Counts](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/bernoulli-binomial-poisson-count-data.md)
  - [🔔 Gaussian and Exponential Models](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/gaussian-exponential-continuous-data.md)
  - [🧠 Bayes' Rule and Base Rates](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/bayes-rule-base-rates.md)
  - [🧪 Checkpoint: Distributions and Bayes](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/checkpoint-distributions-and-bayes.md)
- **🎯 Estimation and Inference**
  - [⚙️ Maximum Likelihood and MAP Estimation](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/maximum-likelihood-map-estimation.md)
  - [⚖️ Bias-Variance Tradeoff in Practice](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/bias-variance-tradeoff-practice.md)
  - [📐 Sampling, Confidence Intervals and Hypothesis Tests](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/sampling-confidence-intervals-hypothesis-tests.md) (premium content)
  - [🏁 Checkpoint: Estimation and Inference](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/checkpoint-estimation-and-inference.md) (premium content)
- [Course cheatsheet](https://everwise.study/courses/mathematics-for-machine-learning-3-probability-and-statistics-for-data/cheatsheet.md) (premium content)