# The Variance Sum Law is Interesting

Friday October 30, 2020

The Variance Sum Law is simple enough that it's often given without explanation, but I think it's interesting enough that it deserves a little thought.

The Variance Sum Law says that for two independent random variables $$A$$ and $$B$$ with variances $$\sigma_A^2$$ and $$\sigma_B^2$$, the sum $$A+B$$ has variance $$\sigma_{A+B}^2 = \sigma_A^2 + \sigma_B^2$$.

Explaining an application of this, you might consider an example: say $$a = 10$$ and $$b = 3$$.

This kind of example is used in at least one course to introduce analysis of variance. After introducing the $$13 = 10 + 3$$ part, they continue:

"Although the derivation isn't as simple as it seems, the decomposition holds for the sum of the squared distances, too."

It's possible to find a derivation, but they're generally not very satisfying. And you know that the sum of squares is not the same as the square of the sum: $$13^2 \neq 10^2 + 3^2$$ because $$169 \neq 109$$. So what's going on?

Using a single example helps explain what one element of data looks like, but a single example doesn't have a variance.

We can make the example just slightly more complete to show how variances add.

Say $$A$$ is $$+10$$ half the time and $$-10$$ half the time, while $$B$$ is equally likely to be $$+3$$ or $$-3$$. So $$\sigma_A^2 = 100$$ and $$\sigma_B^2 = 9$$.

All our squares will be positive, so we can ignore negative outcomes by symmetry here.

With that simplification, notice $$A + B$$ is equally likely to be $$10 - 3 = 7$$ or $$10 + 3 = 13$$.

$\frac{(10-3)^2 + (10-3)^2}{2} = \frac{7^2 + 13^2}{2} = \frac{49 + 169}{2} = \frac{218}{2} = 109 = 100 + 9 = 10^2 + 3^2$

The addition and subtraction cancel out after squaring so that the variances add.

This property, that $$(a+b)^2 + (a-b)^2 = 2(a^2 + b^2)$$, is what makes variance decomposition work, for uncorrelated components.

It's simple algebra, but it's still kind of neat. Here's a geometric illustration:

$(a-b)^2 + (a+b)^2 = b^2 + a^2 + a^2 + b^2$

Maybe this is completely intuitive for some people, but for me this balancing of increased and decreased squares is something of a pleasant surprise. Stop and smell the mathematical roses, you know?

There's also a Pythagorean connection, which I think is neat, but I haven't so far gotten it lined up with my intuition.