Common Distance Measures
Cover image generated by ChatGPT.
Introduction
How to calculate distance is an important aspect of modern statistical analysis. Choosing different distance metrics may lead to different results in statistical analysis methods. The following introduces commonly used distance metrics.
Euclidean Distance
Euclidean Distance, also known as $L_2$ distance, is one of the most common distance metrics. It refers to the straight-line distance between two points.
As shown in the figure, suppose
$$ \begin{align*} & x=(x_1 , \cdots , x_n); \\ & y=(y_1 , \cdots , y_n), \end{align*} $$
the distance between the two points $d(x, y)$ can be expressed as
$$ \begin{align*} d(x,y) & = \sqrt{(x_1-y_1)^2 +(x_2-y_2)^2 + \cdots +(x_n-y_n)^2} \\ & = \sqrt{\sum_{i = 1}^n (x_i - y_i)^2}. \end{align*} $$
We can also compute the vector from the origin $(0, \cdots, 0)$ to the point $x$, as shown in the figure below:
Here, the vector representation of $x$ is denoted as $\overrightarrow{x}$. The length of this vector, known as the norm or the $L_2$ norm, is denoted as $\|x\|_2$, and is calculated as follows:
$$ \|\vec{x}\|_2 = \sqrt{|x_1|^2 + \cdots + |x_n|^2} $$
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Euclidean distance is
$$ \sqrt{(1 - 1)^2 + (2 - 0)^2 + (3 - 6)^2} = \sqrt{13} \approx 3.61 $$
Manhattan Distance
Manhattan Distance, also known as City Block Distance or $L_1$ distance, refers to the total distance traveled only along the coordinate axes. It was first used to describe the shortest driving path in cities with large square blocks, such as Manhattan in New York City, USA.
As shown in the figure above, the red, blue, and green lines all represent the Manhattan distance between two points, and all paths have the same length.
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n), \end{align*} $$
the distance between the two points $d(x, y)$ can be expressed as
$$ \begin{align*} d(x, y) & = |x_1 - y_1| + |x_2 - y_2| + \cdots + |x_n - y_n| \\ & = \sum_{i = 1}^n |x_i - y_i|. \end{align*} $$
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Manhattan distance is
$$ |1 - 1| + |2 - 0| + |3 - 6| = 5 $$
Chebyshev Distance
Chebyshev Distance, also known as Chessboard Distance or $L_\infty$ distance, refers to the maximum difference between two points along any dimension.
| a | b | c | d | e | f | g | h | ||
| 8 |        | 8 | |||||||
| 7 | 7 | ||||||||
| 6 | 6 | ||||||||
| 5 | 5 | ||||||||
| 4 | 4 | ||||||||
| 3 | 3 | ||||||||
| 2 | 2 | ||||||||
| 1 | 1 | ||||||||
| a | b | c | d | e | f | g | h | ||
As shown in the figure above, defining the edge length of each square as 1, for any given square—such as the king (♔) in the image—its eight neighboring squares have a distance of 1, the next surrounding ring has a distance of 2, and so on.
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
the distance between the two points $d(x, y)$ can be expressed as
$$ \begin{align*} d(x, y) & = \max(|x_1 - y_1|, |x_2 - y_2|, \cdots, |x_n - y_n|) \\ & = \max_i(|x_i - y_i|). \end{align*} $$
The Chebyshev distance is actually the limiting form of the $L_k$ norm as $k \to \infty$. The definition of the $L_k$ norm is as follows:
$$ \|w\|_k = \left( \sum_{i=1}^n |w_i|^k \right)^{1/k}. $$
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
The distance vector between the two points can be expressed as
$$ \left( \sum_{i=1}^n |x_i - y_i|^k \right)^{1/k}. $$
As $k \to \infty$, we obtain the distance between the two points as
$$ \begin{align*} d(x, y) & = \lim_{k \to \infty} \left( \sum_{i=1}^n |x_i - y_i|^k \right)^{1/k} \\ & = \max_i(|x_i - y_i|), \end{align*} $$
which is the Chebyshev distance.
The proof of this limit is as follows:
設 $a_i = |x_i - y_i|$ ，令 $a = \max_i \{a_i\}$ ，則：
Upper bound:
$$ \sum_{i=1}^n a_i^k \le n \cdot a^k \quad \Rightarrow \quad \left( \sum_{i=1}^n a_i^k \right)^{1/k} \le a \cdot n^{1/k} $$
Lower bound:
$$ \sum_{i=1}^n a_i^k \ge a^k \quad \Rightarrow \quad \left( \sum_{i=1}^n a_i^k \right)^{1/k} \ge a $$
By Squeeze Theorem, we have:
$$ \begin{align*} & \qquad a \le \left( \sum_{i=1}^n a_i^k \right)^{1/k} \le a \cdot n^{1/k} \\ & \Rightarrow \lim_{k \to \infty} \left( \sum_{i=1}^n a_i^k \right)^{1/k} = a = \max_i \{a_i\}, && \text{when } \lim_{k \to \infty} n^{1/k} = 1. \end{align*} $$
Therefore, as $k \to \infty$, the limit of the $L_k$ norm is exactly the Chebyshev distance, that is:
$$ d(x, y) = \max_i |x_i - y_i|. $$
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Chebyshev distance is
$$ \max \left( |1 - 1| + |2 - 0| + |3 - 6| \right) = 3 $$
Minkowski Distance
Minkowski Distance, also known as the $L_k$ distance, is a generalized form of both Euclidean distance and Manhattan distance.
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
The distance between two points can be expressed as
$$ d(x, y) = \left( \sum_{i=1}^n |x_i - y_i|^k \right)^{1/k}. $$
We can also observe the following points:
When $k \to 1$, the $L_k$ norm degenerates to the Manhattan Distance ($L_1$ distance)
$$ \| x - y \|_1 = \sum_{i=1}^n |x_i - y_i|. $$
When $k \to 2$, the $L_k$ norm degenerates to the Euclidean Distance ($L_2$ distance)
$$ \| x - y \|_2 = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}. $$
When $k \to \infty$, the $L_k$ norm becomes the Chebyshev Distance ($L_\infty$ distance)
$$ \| x - y \|_\infty = \max_i(|x_i - y_i|). $$
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Minkowski distance is
$$ \left( |1 - 1|^3 + |2 - 0|^3 + |3 - 6|^3 \right)^{1/3} = \sqrt[3]{35} \approx 3.27 $$
Canberra Distance
Canberra Distance is a numerical measure of the distance between points in a vector space. It is more sensitive to differences in small values, emphasizing relative differences rather than absolute differences. It is commonly used to detect small but relatively significant changes.
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
The distance between two points can be expressed as
$$ d(x, y) = \sum_{i=1}^{n} \frac{|x_i - y_i|}{|x_i| + |y_i|} $$
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Canberra distance is
$$ \frac{|1 - 1|}{|1| + |1|} + \frac{|2 - 0|}{|2| + |0|} + \frac{|3 - 6|}{|3| + |6|} = \frac{4}{3} \approx 1.33 $$
Cosine Distance
Before discussing Cosine Distance, it is necessary to mention Cosine Similarity.
Cosine Similarity measures similarity by calculating the cosine of the angle between two vectors formed by any two points and the origin. Since the cosine calculation applies to the real number space $\mathbb{R}$, the range of Cosine Similarity is $(-1, 1)$; however, when calculating distance between two points, it is often applied in the positive real space $\mathbb{R}^+$, so the range is usually $(0, 1)$.
Suppose there exist two points
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
The cosine similarity between two points is
$$ \begin{align*} \cos(\theta) & = {x \cdot y \over \|x\| \|y\|} \\ & = \frac{ \sum\limits_{i=1}^{n}{x_i \times y_i} }{ \sqrt{\sum\limits_{i=1}^{n}{(x_i)^2}} \times \sqrt{\sum\limits_{i=1}^{n}{(y_i)^2}} }. \end{align*} $$
In cosine similarity, the smaller the angle between the vectors formed by two points and the origin, the closer the value is to 1; conversely, it approaches 0 or -1.
Cosine distance is calculated as
$$ \text{Cosine Distance} = 1 - \text{Cosine Similarity}, $$
which can be expressed as
$$ d(x, y) = 1 - \cos(\theta), $$
where $\theta$ is the angle between the vectors formed by $x$ and $y$ with the origin.
Suppose there exist two points
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their cosine distance is
$$ 1 - \frac{1 \times 1 + 2 \times 0 + 3 \times 6}{\sqrt{1^2 + 2^2 + 3^2} \times \sqrt{1^2 + 0^2 + 6^2}} = 1 - \frac{19}{\sqrt{14} \times \sqrt{37}} \approx 0.17 $$
Bray–Curtis Dissimilarity
Bray–Curtis Dissimilarity is a commonly used distance measure in biology, ecology, and taxonomy to assess the difference between two samples in species composition or feature distribution. Its range is between $(0, 1)$, where 0 indicates complete similarity and 1 indicates complete dissimilarity.
Its properties are as follows:
- If for any dimension $x_i = y_i = 0$, that dimension does not affect the calculation, making it suitable for sparse vector data.
- It does not satisfy the distance axioms, so it is considered a “dissimilarity” rather than a strictly defined “distance.”
Suppose there exist two samples
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
The similarity between the two samples can be expressed as
$$ \begin{align*} \text{Bray–Curtis}(x, y) & = \frac{\sum_{i=1}^{n} |x_i - y_i|}{\sum_{i=1}^{n} (x_i + y_i)} \\ & = 1 - \frac{2 \sum_{i=1}^{n} \min(x_i, y_i)}{\sum_{i=1}^{n} (x_i + y_i)}. \end{align*} $$
Suppose there exist two samples
$$ \begin{align*} & x=(1, 2, 3); \\ & y=(1, 0, 6). \end{align*} $$
Then their Bray–Curtis dissimilarity is
$$ \frac{|1 - 1| + |2 - 0| + |3 - 6|}{(1 + 1) + (2 + 0) + (3 + 6)} = \frac{5}{13} \approx 0.38 $$
Hamming Distance
Hamming Distance refers to the number of differing characters at corresponding positions between two strings of equal length. In other words, it is the number of character substitutions required to transform one string into another. It is important to note that Hamming Distance cannot be used for continuous numerical vectors.
Suppose there exist two strings
$$ \begin{align*} & x=(x_1, \cdots, x_n); \\ & y=(y_1, \cdots, y_n). \end{align*} $$
Then the Hamming distance between the two strings is
$$ d(x, y) = \sum_{i=1}^{n} \delta(x_i, y_i), $$
其中，$ \delta(x_i, y_i) = \begin{cases} 0, & \text{if } x_i = y_i; \\ 1, & \text{if } x_i \ne y_i. \end{cases} $
Suppose there are three data strings of equal length as follows:
- First set: $x = 1234567$
- Second set: $y = 7654321$
- Third set: $z = 1234321$
We can obtain the Hamming distances between each pair as follows:
| Pair | Comparison | Hamming Distance |
|---|---|---|
| $d(x, y)$ | First set: 1234567 Second set: 7654321 | 6 |
| $d(x, z)$ | First set: 1234567 Third set: 1234321 | 3 |
| $d(y, z)$ | Second set: 7654321 Third set: 1234321 | 3 |
The characters marked in red indicate positions where the characters differ.
Jaccard Distance
Mentioning Jaccard Distance inevitably involves the Jaccard Index.
The Jaccard Index, also called the Jaccard similarity coefficient, is a method used to compare the similarity between two finite sets. It is defined as follows:
Suppose there exist two finite sets $A \in \mathbb{R}^m$ and $B \in \mathbb{R}^n$, then
$$ \begin{align*} \text{Jaccard Index} = J(A, B) & = \frac{|A \cap B|}{|A \cup B|} \\ & = \frac{|A \cap B|}{|A| + |B| - |A \cap B|}, \end{align*} $$
where $0 \le J(A, B) \le 1$.
From this, the Jaccard distance can be derived as
$$ d(A, B) = 1 - \frac{|A \cap B|}{|A \cup B|} $$
Suppose there exist two sets
$$ \begin{align*} & A=\{1, 2, 3\}; \\ & B=\{1, 0, 6\}. \end{align*} $$
Then their Jaccard distance is
$$ 1 - \frac{|{1}|}{|{0,1,2,3,6}|} = 1 - \frac{1}{5} = 0.8 $$
Conclusion
Different distance metrics are suitable for different data types and application scenarios, serving as core tools for measuring relationships between data. Distance not only represents the gap between points but can also represent similarity between sets, providing models with different perspectives to capture more meaningful patterns and differences.
References
歐幾里得距離. (March 9, 2025). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/欧几里得距离
曼哈頓距離. (December 12, 2023). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/曼哈頓距離
切比雪夫距離. (April 16, 2024). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/切比雪夫距离
Minkowski distance. (June 20, 2025). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://en.wikipedia.org/wiki/Minkowski_distance
Canberra distance. (March 30, 2024). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://en.wikipedia.org/wiki/Canberra_distance
餘弦相似性. (January 15, 2025). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/余弦相似性
布雷-柯蒂斯相異度. (September 26, 2021). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/布雷-柯蒂斯相异度
漢明距離. (May 30, 2025). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/汉明距离
雅卡爾指數. (August 30, 2023). Wikipedia, The Free Encyclopedia. Retrieved June 28, 2025, from https://zh.wikipedia.org/zh-tw/雅卡尔指数
Related Posts
