Proof of the Variance of the Sum of Two Random Variables

This is a classic proof in mathematical statistics. That is, whether the variance of the sum of two random variables equals the sum of their variances.

The theorem and the proof as following:

Theorem

Proof

To prove this theorem, we first need to understand the definition of variance. For a random variable $X$, the variance $\text{Var}(X)$ is defined as follows:

$$ \text{Var} (X) = \text{E} \left[ \left(X - \text{E} (X) \right)^2 \right] $$

where, $\text{E}(X)$ denotes the expected value of $X$, which is defined as follows:

• $X$ is a discrete random variable, and its domain is $\boldsymbol{X}$. $$ \text{E} (X) = \sum_{x \in \boldsymbol{X}} x \cdot f_X (x) $$ where $f_X (x)$ is the probability mass function (pmf).

• $X$ is a continuous random variable, and its domain is $\boldsymbol{X}$. $$ \text{E} (X) = \int_{x \in \boldsymbol{X}} x \cdot f_X (x) dx $$ where $f_X (x)$ is the probability density function (pdf).

Moreover, it is known that the expectation operator $\text{E}$ is linear, such that

$$ \text{E} (aX + bY) = a \text{E} (X) + b \text{E} (Y) $$

where $X$ and $Y$ are two random variables defined on the same probability space, and $a$ and $b$ are arbitrary real numbers.

For two random variables $X$ and $Y$, the covariance is defined as:

$$ \text{Cov} (X, Y) = \text{E} \left[ (X - \text{E} (X)) \cdot (Y - \text{E} (Y)) \right] $$

From these definitions, we can decompose $\text{Var}(X + Y)$ and obtain:

$$ \begin{align*} \text{Var} (X + Y) & = \text{E} \left[ \left( (X + Y) - \text{E} (X + Y) \right)^2 \right] \\ & = \text{E} \left[ \left( X + Y - \text{E} (X) - \text{E} (Y) \right)^2 \right] \\ & = \text{E} \left[ \left( X - \text{E} (X) + Y - \text{E} (Y) \right)^2 \right] \\ & = \text{E} \left[ \left( (X - \text{E} (X)) + (Y - \text{E} (Y)) \right)^2 \right] \\ & = \text{E} \left[ (X - \text{E} (X))^2 + 2 \cdot (X - \text{E} (X)) \cdot (Y - \text{E} (Y)) + (Y - \text{E} (Y))^2 \right] \\ & = \text{E} \left[ (X - \text{E} (X))^2 \right] + 2 \cdot \text{E} \left[ (X - \text{E} (X)) \cdot (Y - \text{E} (Y)) \right] + \text{E} \left[ (Y - \text{E} (Y))^2 \right] \\ & = \text{Var} (X) + 2 \text{Cov} (X, Y) + \text{Var} (Y) \\ & = \text{Var} (X) + \text{Var} (Y) + 2 \text{Cov} (X, Y) \end{align*} $$

Therefore, the theorem is proved.

Independence

When two random variables $X$ and $Y$ are independent, their covariance is 0. The proof is as follows:

From the above proof, we also know:

When $X \mathrel{\perp\!\!\!\perp} Y$, $\text{Cov} (X,Y)=0$, and consequently

$$ \text{Var} (X + Y) = \text{Var} (X) + \text{Var} (Y) $$