# 🎯 Dot Product and Cosine Similarity

By Better Wise

In the last lesson we gave a vector a size, its norm. Now we ask a different question: given two vectors, how do they sit relative to each other? The dot product is the single operation that answers this, because it packs length and angle into one number. Everything here runs in a few lines of NumPy.

By the end of this lesson you can:

- compute a dot product by hand and with `np.dot`
- read the sign of a dot product as an angle
- normalize a vector and compute cosine similarity
- rank items by similarity and recognize when the raw dot product misleads you

## Multiplying Two Vectors into a Single Number

Draw a and b as arrows from the same origin. The dot product asks: how much of b lies along a? The algebra is simple: multiply matching components, then add.

```python
import numpy as np

a = np.array([3.0, 4.0])
b = np.array([4.0, 3.0])
np.dot(a, b)            # 3*4 + 4*3 = 24.0
```

Geometrically, $\mathbf{a}\cdot\mathbf{b} = \|\mathbf{a}\|\|\mathbf{b}\|\cos\theta$. That identity follows from the law of cosines applied to the triangle with sides a, b and a − b: $\|\mathbf{a}-\mathbf{b}\|^2 = \|\mathbf{a}\|^2 + \|\mathbf{b}\|^2 - 2\,\mathbf{a}\cdot\mathbf{b}$.

The big consequence: when both vectors are unit length, the dot product *is* the cosine of the angle between them. Length disappears, direction remains.

> **Note:** 
> `np.dot(a, b)` and `a @ b` are the same operation for 1-D arrays. For matrices, `@` is matrix multiplication, which arrives in Module 2.

**Try it:** compute $[1, 2]\cdot[3, -1]$ by hand, then check with NumPy.

**Click to reveal the answer**

$1\cdot3 + 2\cdot(-1) = 1$. In NumPy: `np.dot(np.array([1, 2]), np.array([3, -1]))` returns `1`.

## Reading the Angle From the Sign

You rarely need the exact angle. The sign alone tells you the relationship, which is why the dot product is such a fast similarity check.

```mermaid
flowchart LR
  DP["📐<br/>a · b"] --> POS["positive"] --> AC["acute, θ below 90°"]
  DP --> ZERO["zero"] --> RT["right angle, orthogonal"]
  DP --> NEG["negative"] --> OB["obtuse, θ above 90°"]
```

| Dot product | Angle | Meaning in practice |
| --- | --- | --- |
| large positive | close to 0° | vectors agree strongly |
| zero | exactly 90° | no shared direction at all |
| large negative | close to 180° | vectors oppose each other |

**Try it:** without computing norms, decide whether $[2, 1]$ and $[-1, 3]$ are acute, right, or obtuse.

**Click to reveal the answer**

$2\cdot(-1) + 1\cdot3 = -2 + 3 = 1$, so the angle is acute. Small positive value, so slightly acute, nearly orthogonal.

## Cosine Similarity: Comparing Direction Only

Divide the dot product by both norms and you get a pure direction score, independent of how long the vectors are:

$$
\cos\theta = \frac{\mathbf{a}\cdot\mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}
$$

This value always lands in $[-1, 1]$. Identical direction gives 1, orthogonal gives 0, opposite gives −1. That fixed range is what makes it safe to compare across pairs of very different magnitudes.

To compute a dot product you multiply _____ components and then _____ the results.
The geometric form of the dot product is a·b = ||a|| ||b|| _____.
Cosine similarity divides the dot product by the product of the two _____.
Vectors pointing in exactly opposite directions have cosine similarity _____.

## Normalizing and Ranking in NumPy

Normalizing means scaling a vector down to length 1, keeping its direction. Then the plain dot product of two normalized vectors already *is* their cosine similarity.

```python
def cos_sim(u, v):
    return np.dot(u, v) / (np.linalg.norm(u) * np.linalg.norm(v))

prefs = np.array([1.0, 2.0, 0.0])
items = {
    "action":  np.array([3.0, 1.0, 0.0]),
    "romance": np.array([0.0, 2.0, 3.0]),
    "comedy":  np.array([1.0, 2.0, 0.5]),
}
for name, vec in items.items():
    print(name, round(cos_sim(prefs, vec), 3))
# comedy 0.976, action 0.707, romance 0.496
```

The ranking is the deliverable: comedy, then action, then romance. This is exactly the pattern behind recommendation systems and text search, where each item is a vector of features or word counts.

**Try it:** change `prefs` to `[1.0, 2.0, 0.0] * 10`. Do the three scores move?

**Click to reveal the answer**

No. Scaling `prefs` multiplies the dot product and one norm by 10 each, and the factors cancel. Cosine similarity is scale invariant by design.

## The Scale Trap

The raw dot product has no such protection. If you double b, the dot product doubles too, while the angle does not change. That means ranking by raw dot product secretly rewards long vectors: the document that repeats itself, the item with the most ratings, the user who clicked the most.

A concrete case: document A has dot product 50 against the query with norm 100, document B has dot product 20 with norm 25. Raw ranking picks A, but the cosine scores are 0.5 and 0.8, so B matches the query's direction better.

Cosine has the opposite blind spot. If magnitude carries meaning (how intense, how many, how strong), cosine throws that information away, and an item seen once looks as similar as an item seen a thousand times.

**Try it:** a vector pair has norms 3 and 7 and cosine similarity 0.6. What is their raw dot product?

**Click to reveal the answer**

$0.6 \times 3 \times 7 = 12.6$. The dot product is just the cosine rescaled by the two lengths.

## Rules of Thumb for Similarity Work

- Dot product and cosine similarity agree only when both vectors are unit length. Otherwise they answer different questions.
- Sign is your fastest check: positive means agree, zero means orthogonal, negative means oppose.
- Normalize first with `v / np.linalg.norm(v)`, then reuse the dot product everywhere.
- Guard against the zero vector. Its norm is 0, so cosine similarity is undefined and NumPy returns `nan`. Check `np.linalg.norm(v) == 0` first.
- Cosine similarity is not a distance. For clustering or nearest neighbors, use `1 - cos_sim`, which is 0 for identical and 2 for opposite.
- Report both scores when magnitudes are meaningful. Cosine alone can hide real differences in intensity.

## What to Carry Into the Checkpoint

- The dot product multiplies components, sums them, and equals $\|\mathbf{a}\|\|\mathbf{b}\|\cos\theta$.
- Cosine similarity is the dot product with both lengths divided out, so it measures direction only and lives in $[-1, 1]$.
- Normalizing in NumPy is one line, and the raw dot product then behaves like cosine similarity.
- The scale trap is the reason to normalize before ranking, and the reason to keep magnitudes when they matter.

Front: What two operations make up a dot product?
Back: Multiply matching components, then sum the products.

Front: Compute a·b for a = [1, 2] and b = [3, -1].
Back: 1·3 + 2·(-1) = 1.

Front: What does a positive dot product say about the angle?
Back: The angle is acute, below 90 degrees.

Front: What does a zero dot product mean geometrically?
Back: The vectors are orthogonal, at 90 degrees.

Front: State the geometric formula for the dot product.
Back: a·b = ||a|| ||b|| cos θ.

Front: How is cosine similarity defined?
Back: cos θ = (a·b) / (||a|| ||b||), a value in [-1, 1].

Front: Cosine similarity of two vectors pointing the same way?
Back: 1.

Front: Cosine similarity of two exactly opposite vectors?
Back: -1.

Front: Which NumPy call returns a vector's Euclidean norm?
Back: np.linalg.norm(v).

Front: How do you normalize v in NumPy?
Back: v / np.linalg.norm(v).

Front: Why does a raw dot product favor long vectors?
Back: It grows with both magnitudes, so length inflates the score even at the same angle.

Front: What happens to cosine similarity if you scale one vector by 10?
Back: It is unchanged; scaling alters length, not direction.

Front: How do you turn cosine similarity into a distance?
Back: cosine distance = 1 - cosine similarity.

Front: Why must you check for a zero vector before cosine similarity?
Back: Its norm is 0, so the division is undefined and NumPy yields nan.

Next comes the checkpoint on vectors and similarity, where norms and cosine similarity are mixed into one set of exercises.

---

## Contents

Part of [🧮 Mathematics for Machine Learning 1: Linear Algebra Foundations](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations.md)

- **📐 Vectors, Norms and Similarity**
  - [➡️ Vectors as Arrows, Points, and NumPy Arrays](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/vectors-as-arrows-points-and-numpy-arrays.md)
  - [📏 Norms and Distances: Measuring Vector Size](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/norms-and-distances-measuring-vector-size.md) (sign-in required)
  - [🎯 Dot Product and Cosine Similarity](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/dot-product-and-cosine-similarity.md) — this lesson
  - [✅ Checkpoint: Vectors and Similarity](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-vectors-and-similarity.md)
- **🔄 Matrices and Transformations**
  - [🧱 Matrices as Transformations and Matrix Multiplication as Composition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/matrices-as-transformations-and-matrix-composition.md)
  - [🧭 Inverse, Rank, and Span](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/inverse-rank-and-span.md)
  - [🔀 Basis and Change of Basis](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/basis-and-change-of-basis.md)
  - [✅ Checkpoint: Matrices and Transformations](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-matrices-and-transformations.md)
- **🔆 Projections, Eigenvectors and SVD**
  - [📉 Projections and Least Squares Intuition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/projections-and-least-squares-intuition.md)
  - [🌀 Eigenvectors and Eigen-Decomposition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/eigenvectors-and-eigen-decomposition.md)
  - [🔍 A First Look at SVD](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/first-look-at-svd.md) (premium content)
  - [✅ Checkpoint: Projections, Eigen and SVD](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-projections-eigen-and-svd.md) (premium content)
- [Course cheatsheet](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/cheatsheet.md) (premium content)