--:--
--:--

2014-06-27
bayesian-updating

## Bayesian Updating

In this minisode Kyle and Linh Da discuss the meaning of Bayesian Updating.

The root concept is Bayes Theorem:

$$Pr(A|B) = \dfrac{Pr(B|A) \cdot Pr(A)}{Pr(B)}$$

Although a more helpful way of thinking about this alongside our discussion is to replace $$A$$ with $$H$$ for Hypothesis, and $$B$$ with $$E$$ for Evidence.

$$Pr(H|E) = \dfrac{Pr(E|H) \cdot Pr(H)}{Pr(E)}$$

In this case $$Pr(H)$$ would represent your probability / belief representing the likelihood that the hypothesis is true. Given new evidence $$E$$, you can evaluate how likely that hypothesis was to generate that Evidence (i.e.Â $$Pr(E|H)$$).

In our farmers market example, we hadâ€¦

• $$H_P$$ - Box contains 100% (P)omegranates
• $$H_M$$ - Box contains (M)ix of 50% pomegranates and 50% lemons
• $$H_L$$ - Box contains 100% (L)emons

We established our prior beliefs as $$H_i = \frac{1}{3}$$ for $$i=0,1,2$$.

When the farm pulls out one example fruit which turns out to be a pomegranate, Linh Da points out that itâ€™s not the bad of just lemonsâ€™â€™ (a.k.a. $$Pr(H_L) = 0$$). But that doesnâ€™t mean the other hypothesis are not 50 / 50! What are they?

$Pr(H_P|E) = = = = 66.%$

$Pr(H_M|E) = = = = 33.%$

If youâ€™re having a hard time understanding why $$Pr(E) = .50$$, itâ€™s because this is your expectation of the evidence you got independent of which hypothesis turns out to be true. Thus, itâ€™s based on your prior (original) belief over each hypothesis and their likelihood of producing the observation. In other wordsâ€¦

$$\begin{split} Pr(E) &= Pr(E|H_P) \cdot Pr(H_P) + Pr(E|H_M) \cdot Pr(H_M) + Pr(E|H_L) \cdot Pr(H_L) \\ &= 1.0 \cdot \frac{1}{3} + 0.50 \cdot \frac{1}{3} + 0.0 \cdot \frac{1}{3} \\ &= 0.5 \end{split}$$

Â