Norms and Distances: Measuring Vector Size

By Better Wise

In the last lesson a vector was an arrow, a point, and a NumPy array. Arrows have length, and that length is what we now put a number on. Everything in this course that compares vectors later (cosine similarity, projections, least squares) quietly relies on the idea of size we build here.

By the end of this lesson you can:

draw what the L1, L2 and max norms actually measure;
compute all three with NumPy;
compute the distance between two vectors;
pick the norm that fits a task.

Three ways to measure an arrow

Picture the vector $v = (3, 4)$ as an arrow from the origin to the point $(3, 4)$. How long is it? The honest answer is "it depends on how you are allowed to walk".

flowchart LR
    Origin["🏠<br/>Origin (0,0)"] --> L1["🧱<br/>L1: walk the grid<br/>3 + 4 = 7"]
    Origin --> L2["📏<br/>L2: fly straight<br/>sqrt(9+16) = 5"]
    Origin --> Max["⬛<br/>Max: widest side<br/>max(3,4) = 4"]

Three different rulers, three different numbers, one arrow. None of them is wrong; each answers a different question.

L2 norm: straight-line length

The L2 norm (also called the Euclidean norm) is the ruler you grew up with. Square each component, add, take the square root:

$$ |v|_2 = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} $$

For $(3,4)$ that is $\sqrt{9+16} = 5$. This is Pythagoras applied twice for higher dimensions. In NumPy:

import numpy as np

v = np.array([3.0, 4.0])
np.linalg.norm(v)                 # L2 by default -> 5.0
np.sqrt(np.sum(v**2))             # same thing, spelled out -> 5.0

A useful mental picture: the set of all vectors with L2 norm equal to 1 is a circle. That circle is what the L2 norm is "shaped like".

L1 norm: the city-block walk

The L1 norm (Manhattan norm) adds the absolute values of the components:

$$ |v|_1 = |v_1| + |v_2| + \cdots + |v_n| $$

For $(3,4)$ that is $7$. This is how far a taxi drives on a grid of streets. Its unit ball is a diamond, which is why L1 keeps pushing values to exactly zero: moving along an axis is as cheap as moving diagonally.

np.linalg.norm(v, ord=1)          # 7.0
np.sum(np.abs(v))                 # same

Max norm: the widest component

The max norm (also called the infinity norm, $L_\infty$) cares only about the largest absolute component:

$$ |v|_\infty = \max_i |v_i| $$

For $(3,4)$ that is $4$. Its unit ball is a square. Use it when the cost of a system is set by its single worst dimension, for example the maximum deviation of any one sensor.

np.linalg.norm(v, ord=np.inf)     # 4.0
np.max(np.abs(v))                 # same

Norm	Formula for $(3,4,-1)$	NumPy call	Unit shape
L1	$3+4+1 = 8$	`np.linalg.norm(v, ord=1)`	diamond
L2	$\sqrt{9+16+1} \approx 5.10$	`np.linalg.norm(v)`	circle
Max	$\max(3,4,1) = 4$	`np.linalg.norm(v, ord=np.inf)`	square

Note: All three are norms in the technical sense: they are zero only for the zero vector, they scale linearly ($|cv| = |c|,|v|$), and they satisfy the triangle inequality. That is what makes them trustworthy as "size".

Try it. For $u = (1, -2, 2)$ compute all three norms by hand, then check in NumPy.

Click to reveal the answer

L1 = 1 + 2 + 2 = 5. L2 = sqrt(1 + 4 + 4) = 3. Max = 2.

u = np.array([1.0, -2.0, 2.0])
np.linalg.norm(u, ord=1)          # 5.0
np.linalg.norm(u)                 # 3.0
np.linalg.norm(u, ord=np.inf)     # 2.0

Distance: the norm of a difference

Two vectors are two arrows. How far apart are their tips? Subtract them, then measure the resulting arrow.

$$ d(u, v) = |u - v| $$

a = np.array([1.0, 2.0])
b = np.array([4.0, 6.0])

diff = b - a                       # [3.0, 4.0]
np.linalg.norm(diff)               # L2 distance -> 5.0
np.linalg.norm(diff, ord=1)        # L1 distance -> 7.0
np.max(np.abs(diff))               # max distance -> 4.0

Note that the order does not matter, $|u - v| = |v - u|$, because absolute values swallow the sign. Visualise it: the arrows point in opposite directions from each other, and both have the same length.

Distance is where norms earn their keep. If two customers are represented as vectors of purchase counts, the L2 distance says "how different overall", the L1 distance says "how many total units of difference", and the max distance says "biggest single disagreement". Same data, three honest readings.

Tip: np.linalg.norm(a - b) is the one-liner you will use most. There is also scipy.spatial.distance.euclidean, but for a course on fundamentals NumPy keeps the geometry visible.

Try it. Let $p = (0, 0)$ and $q = (3, -4)$. Compute all three distances between them.

Click to reveal the answer

Because $p$ is the origin, the distance equals the norm of $q$. L2 = 5, L1 = 7, max = 4. In NumPy:

q = np.array([3.0, -4.0])
np.linalg.norm(q)                 # 5.0
np.linalg.norm(q, ord=1)          # 7.0
np.linalg.norm(q, ord=np.inf)     # 4.0

Choosing a ruler

Question you are asking	Norm to reach for
Straight-line gap between two points	L2
Total absolute change, or sparse counts on a grid	L1
Worst-case single deviation, or a hard bound to enforce	Max
General-purpose default for beginners	L2

Two rules of thumb do most of the work. First, if your data has many irrelevant small dimensions, L1 tends to be more robust because errors add linearly rather than being squared. Second, if you need a guaranteed ceiling, only the max norm gives you one, since $|v|_\infty \le |v|_2 \le |v|_1$ for any vector.

That ordering is worth memorising:

_____ holds for every vector $v$.

The L2 norm of the vector $(3, 4)$ is _____.

The L1 norm of the vector $(3, 4)$ is _____.

The max norm of the vector $(3, 4)$ is _____.

The distance between two vectors is defined as the norm of their _____.

The set of all vectors with L2 norm 1 forms a _____.

Pitfalls worth avoiding

Forgetting the absolute value in L1 or max. The norm of $(3, -4)$ is 7, not $-1$. NumPy handles this, but by-hand arithmetic does not.
Passing ord=2 thinking it means "two dimensions". ord is the exponent, not the dimension count. ord=1 and ord=np.inf are the ones you switch.
Assuming all norms give the same ranking. For $a = (1, 1)$ and $b = (2, 0)$: L2 says $b$ is larger (2.0 vs 1.41), L1 ties both at 2. Pick the norm before you compare.
Subtracting in the wrong direction. Distances are symmetric, but the sign of b - a still matters inside code when you reuse diff for something else.
Dividing by a norm without checking for zero. The zero vector has norm 0, so normalising it produces nan.

What to carry forward

Norms turn an arrow into a number. L2 is the straight line, L1 is the grid walk, and max is the widest side; distance is the norm of a difference. Compute them with np.linalg.norm(v, ord=...) and pick the ruler to match the question.

Next lesson uses this size idea to compare directions rather than lengths: the dot product and cosine similarity, where dividing by the L2 norms makes the comparison scale-free.

Front: What does the L2 norm compute? Back: The square root of the sum of squared components, the straight-line length.

Front: What does the L1 norm compute? Back: The sum of absolute component values, the city-block distance.

Front: What does the max norm (L-infinity) compute? Back: The largest absolute value among the components.

Front: L2 norm of (3, 4)? Back: 5

Front: L1 norm of (3, 4)? Back: 7

Front: Max norm of (3, 4)? Back: 4

Front: How is the distance between vectors u and v defined? Back: The norm of their difference, ||u - v||.

Front: What is the unit ball shape of the L2 norm? Back: A circle.

Front: What is the unit ball shape of the L1 norm? Back: A diamond.

Front: What is the unit ball shape of the max norm? Back: A square.

Front: Which NumPy call gives the L2 norm of v? Back: np.linalg.norm(v), since L2 is the default.

Front: How do you request the L1 norm in NumPy? Back: np.linalg.norm(v, ord=1)

Front: How do you request the max norm in NumPy? Back: np.linalg.norm(v, ord=np.inf)

Front: State the ordering of the three norms for any vector. Back: Max norm <= L2 norm <= L1 norm.

Front: Which norm is a normalisation risky for and why? Back: Any norm, when the vector is zero, because the norm is 0 and you divide by zero.