Exam Questions - Block 3: Beta-Binomial prior to posterior

[SebastianVeuskens]

Describe the passage from the prior to the posterior for a Beta-Binomial case

[andrealusa]

The Beta-Binomial model is a combination of a Binomial distribution as the likelihood (the distribution that rules the experiment) and a Beta distribution as the prior for the parameter (a flexible distribution defined on the interval [0, 1]).
The Beta distribution is chosen as the prior because it is conjugate to the Binomial distribution, meaning that if you start with a Beta prior and update it with Binomial likelihood, the resulting posterior distribution is also a Beta distribution.

This model is commonly used to infer the probability of success in a series of binary trials, where I want to investigate how, given what I observed in my experiment, my opinion about ϴ will change:

• n=n° of trails;
• r=n° of successes;
• n-r=n° of failures.

So:

• $L(\Theta_{j}; r; n)=\binom{n}{r}*\Theta_{j}^{r}*(1-\Theta_{j})^{n-r}$
  The binomial coefficient doesn’t depend on theta, so it’s not that important;

• ϴ ~ Beta(a,b):
  If a=b=1, then it's the same to say that ϴ ~ Uniform(0,1)

• The posterior ϴ|r,n ~ Beta(a’, b’):
  a’= a + r ;
  b’= b + (n-r).

COMMENTS:

• $E[\Theta|a,b]=\frac{a}{a+b}$;
• $V[\Theta|a,b]=\frac{a*b}{(a+b+1)*(a+b)^{2}}$.

a+b is the size of the virtual sample that allows me to declare the solution (should be less than the n° of trails n).
If n is small and the prior has expectation = 0.5 and a very small variance, which is what we get when a=b, then a’ and b’ won’t change that much.

[SebastianVeuskens]

Amazing!

One thought on the size of a and b:
In my opinion, we dont impose any restriction on the value of a+b, except that both should variables should be non-negative.

1. If a+b is higher than the number of trails, we expect our posterior likelihood to be dominated by the prior distribution.
2. On the other hand, if a+b is (much) smaller than our number of trials n, we expect our posterior likelihood to be dominated by the likelihood (This related to the concept of indifferent priors -> Beta(a=1, b=1) is such a indifferent prior with a+b=2 very small, thus the likelihood dominates and determines our parameter inference)

[SebastianVeuskens]

An additional note:

The Jeffrey's prior for the Beta-Binomial model is a $Beta(\frac{1}{2},\frac{1}{2})$ distribution.

This is even scarcer information than for the indifference uniform prior $Beta(1, 1)$.

The $Beta(\frac{1}{2},\frac{1}{2})$ distribution looks like this: