📐 Sampling, Confidence Intervals and Hypothesis Tests

By Better Wise

The previous lesson split prediction error into bias and variance. This lesson moves from model error to estimate error: you have one sample, one estimate, and a decision to make. We will use a delivery-time dataset question to build a confidence interval and run a one-sample test, then pin down what a p-value does and does not mean.

What you will be able to do

  • Explain why a sample mean varies and estimate its standard error.
  • Build and interpret a 95% confidence interval for a mean.
  • Run a one-sample t-test and state the p-value correctly.
  • Separate statistical significance from practical importance.

A Delivery-Time Dataset Question

A regional delivery team records 64 completed orders. The sample mean delivery time is 31 minutes, and the sample standard deviation is 8 minutes. The business question is simple: is the average delivery time above the 28-minute service target?

The sample mean is one draw from a sampling distribution, not the population mean itself. If we repeated the study with new random samples of 64 orders, the mean would move. The standard error of the sample mean estimates that movement:

$\mathrm{SE} = s / \sqrt{n} = 8 / \sqrt{64} = 1$ minute.

With independent orders and finite variance, the central limit theorem makes the sampling distribution of the mean approximately normal for $n = 64$, even if individual delivery times are skewed.

A 95% Confidence Interval for Mean Delivery Time

Because $\sigma$ is unknown, use the t critical value with $n - 1 = 63$ degrees of freedom. The 95% interval is:

$$ \bar{x} \pm t^* \frac{s}{\sqrt{n}} $$

With $t^*$ about 2.00, the interval is 31 plus or minus 2.00(1), or (29.0, 33.0) minutes.

The correct reading: if we repeated this sampling and interval construction many times, about 95% of the intervals would contain the true mean delivery time. The interval does not contain 95% of individual deliveries, and the computed interval is not a 95% probability statement about a fixed unknown mean.

In this lesson

  • A Smaller Sample Widens the Interval
  • One-Sample Test: Is the Mean Above 28 Minutes?
  • What a p-value Actually Says
  • Review Deck: Sampling and Inference
  • What to Keep and Where We Go Next

About this course

Part of Mathematics for Machine Learning 3: Probability and Statistics for Data-Driven Models — Gain the probability and statistics fluency models rely on: reason about distributions, apply Bayes and maximum likelihood, and quantify uncertainty in real data.

Level
Intermediate
Goal
Fundamentals
Language
English
Focus
probability distributions Bayes' rule maximum likelihood statistical inference sampling and uncertainty
Prior knowledge
Basic calculus (derivatives, integrals), Vectors and matrices, Some Python for small simulations
Practice
In-depth examples
Time
~5.3h
Voice
a data scientist who grounds every distribution in a real dataset question before showing the formula
Not covered
Measure-theoretic probability, Bayesian deep learning, Causal inference

This lesson is premium content. Sign in and purchase the course to view the full content. Sign in