Bayes' Rule for Ducks

Sunday February 23, 2014

You look at a thing.

duck?

Is it a duck?

Re-phrase: What is the probability that it's a duck, if it looks like that?

Bayes' rule says that the probability of it being a duck, if it looks like that, is the same as the probability of any old thing being a duck, times the probability of a duck looking like that, divided by the probability of a thing looking like that.

\[ Pr(duck | looks) = \frac{Pr(duck) \cdot Pr(looks | duck)}{Pr(looks)} \]

This makes sense:

Here's an example of a strange world to test this in:

ducks

There are ten things. Six of them are ducks. Five of them look like ducks. Four of them both look like ducks and are ducks. One thing looks like a duck but is not a duck. Maybe it's a fake duck? Two ducks do not look like ducks. Ducks in camouflage. Test the equality of the two sides of Bayes' rule:

\[ Pr(duck | looks) = \frac{Pr(duck) \cdot Pr(looks | duck)}{Pr(looks)} \]

\[ \frac{4}{5} = \frac{\frac{6}{10} \cdot \frac{4}{6}}{\frac{5}{10}} \]

It's true here, and it's not hard to show that it must be true, using two ways of expressing the probability of being a duck and looking like a duck. We have both of these:

\[ Pr(duck \cap looks) = Pr(duck|looks) \cdot Pr(looks) \]

\[ \displaystyle Pr(duck \cap looks) = Pr(looks|duck) \cdot Pr(duck) \]

Check those with the example as well, if you like. Using the equality, we get:

\[ Pr(duck|looks) \cdot Pr(looks) = Pr(looks|duck) \cdot Pr(duck) \]

Then dividing by \( Pr(looks) \) we have Bayes' rule, as above.

\[ Pr(duck | looks) = \frac{Pr(duck) \cdot Pr(looks | duck)}{Pr(looks)} \]

This is not a difficult proof at all, but for many people the result feels very unintuitive. I've tried to explain it once before in the context of statistical claims. Of course there's a wikipedia page and many other resources. I wanted to try to do it with a unifying simple example that makes the equations easy to parse, and this is what I've come up with.

This post was originally hosted elsewhere.