Lecture 3 - CH 4 - Bayes' Theorem

Bayes' Theorem allows us to calculate P(B|A) when we don't have the usual information that would allow us to calculate it. I.e. we don't know P(B and A) or P(A).

We know P(B|A) = P(B and A) / P(A)
We'll derive alternative expressions for both the numerator and the denominator.

Since P(A|B) = P(A and B) / P(B) and P(A and B) = P(B and A), we can multiply both side by P(B) and rearrange the terms and get:
P(B and A) = P(A|B)P(B)
which becomes our numerator.

For the denominator, we note that
P(A) = P(A|B)P(B) + P(A|B')P(B')

It's worth thinking about this for a second. Time's up! What it means is that the total probability of A is equal to the conditional probability of A given B (times the probability of B) plus the conditional probability of A given B' (times the probability of B'). Since B and B' cover the entire sample space, the sum of those conditional probabilities (times their respective factors) equals the entire probability of A.

If there were more than two possibilities for the condition - for instance B1, B2, B3, ... - then the total probability of A would be the some of all three of the conditionals, i.e.:
P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) + P(A|B3)P(B3) + ...

This is true as long as the events B1, B2, B3, ... are mutually exclusive and exhaustive of the sample space.

Now, back to our original formula that we were trying to evaluate:

Our new expression for P(B|A) becomes:and allows us to calculate P(B|A) if we know P(B), P(A|B) and P(A|B').