Bayesian Updating

In this minisode Kyle and Linh Da discuss the meaning of Bayesian Updating.

The root concept is Bayes Theorem:

\(Pr(A|B) = \dfrac{Pr(B|A) \cdot Pr(A)}{Pr(B)}\)

Although a more helpful way of thinking about this alongside our discussion is to replace \(A\) with \(H\) for Hypothesis, and \(B\) with \(E\) for Evidence.

\(Pr(H|E) = \dfrac{Pr(E|H) \cdot Pr(H)}{Pr(E)}\)

In this case \(Pr(H)\) would represent your probability / belief representing the likelihood that the hypothesis is true. Given new evidence \(E\), you can evaluate how likely that hypothesis was to generate that Evidence (i.e. \(Pr(E|H)\)).

In our farmers market example, we had…

• \(H_P\) - Box contains 100% (P)omegranates
• \(H_M\) - Box contains (M)ix of 50% pomegranates and 50% lemons
• \(H_L\) - Box contains 100% (L)emons

We established our prior beliefs as \(H_i = \frac{1}{3}\) for \(i=0,1,2\).

When the farm pulls out one example fruit which turns out to be a pomegranate, Linh Da points out that ``it’s not the bad of just lemons’’ (a.k.a. \(Pr(H_L) = 0\)). But that doesn’t mean the other hypothesis are not 50 / 50! What are they?

$ Pr(H_P|E) = = = = 66.% $

$ Pr(H_M|E) = = = = 33.% $

If you’re having a hard time understanding why \(Pr(E) = .50\), it’s because this is your expectation of the evidence you got independent of which hypothesis turns out to be true. Thus, it’s based on your prior (original) belief over each hypothesis and their likelihood of producing the observation. In other words…

$ \[\begin{split} Pr(E) &= Pr(E|H_P) \cdot Pr(H_P) + Pr(E|H_M) \cdot Pr(H_M) + Pr(E|H_L) \cdot Pr(H_L) \\ &= 1.0 \cdot \frac{1}{3} + 0.50 \cdot \frac{1}{3} + 0.0 \cdot \frac{1}{3} \\ &= 0.5 \end{split}\]

$