20 Beta Bernoulli - M3L8HW2 – Bayesian Statistics

Caution

Section omitted to comply with the Honor Code

--- title: "Beta Bernoulli - M3L8HW2" subtitle: "Bayesian Statistics: From Concept to Data Analysis" categories: - Bayesian Statistics keywords: - Beta Distribution - Bernoulli Distribution - Homework - Honors --- ::::: {.content-visible unless-profile="HC"} ::: {.callout-caution} Section omitted to comply with the Honor Code ::: ::::: ::::: {.content-hidden unless-profile="HC"} ::: {#exr-beta-bernoulli-1} Identify which of the following conditions (possibly more than one) must be true for the sum of n Bernoulli random variables (with success probability p) to follow a binomial distribution. ::: ::: {.solution .callout-tip collapse="true"} #### Solution: - [x] [**p**]{.underline} **must be the same for each of the Bernoulli random variables** - [x] **each Bernoulli random variable is independent of all others** - [ ] $p$ must be less than .5 - [ ] the sum must exceed $n$ - [ ] the sum must be greater than zero ::: For Questions 2-4, consider the following: ::: {#exr-beta-bernoulli-2} In Lesson 6.3 we found the prior predictive distribution for a Bernoulli trial under a uniform prior on the success probability $\theta$. We now derive the prior predictive distribution when the prior is any conjugate beta distribution. There are two straightforward ways to do this. The first approach is the same as in the lesson. The marginal distribution of $y$ is $$ f(y)= \int^1_0 f(y \mid \theta)f(\theta)d\theta $$ Now $f(\theta)$ is a beta PDF, but the same principles apply: we can move constants out of the integral and find a new normalizing constant to make the integral evaluate to 1. Another approach is to notice that we can write Bayes' theorem as $$ f(\theta \mid y)= \frac{f(y \mid \theta)f(\theta)}{f(y)} $$ If we multiply both sides by $f(y)$ and divide both sides by $f(\theta∣y)$, then we get $$ f(y) = \frac{f(y \mid \theta)f(\theta)}{f(\theta\mid y)} $$ where: - $f(\theta)$ is the **beta prior PDF** and - $f(\theta \mid y)$ is the updated **beta posterior PDF**. Both approaches yield the same answer. What is the prior predictive distribution $f(y)$ for this model when the prior for $\theta$ is $Beta(a,b)$? ::: ::: {.solution .callout-tip collapse="true"} #### Solution: $$ f(y)=\frac{Γ(a+b)}{Γ(a+b+1)}\cdot \frac{Γ(a+y)}{Γ(a)} \cdot \frac{Γ(b+1−y)}{Γ(b)} \qquad \text{for } y = 0,1 $$ All the terms involving $\theta$ canceled out as they should. ::: ::: {#exr-beta-bernoulli-3} Now suppose the prior for $\theta$ is $Beta(2,2)$. What is the prior predictive probability that $y^∗=1$ for a new observation $y^∗$? ::: ::: {.solution .callout-tip collapse="true"} #### Solution: $$ \begin{aligned} \mathrm{Beta}(2,2) &= \frac{Γ(2+2)}{Γ(2+2+1)} \cdot \frac{Γ(2+y)}{Γ(2)} \cdot \frac{Γ(2+1−y)}{Γ(2)} \\ &= \frac{Γ(4)}{Γ(5)} \cdot \frac{Γ(3)}{Γ(2)} \cdot \frac{Γ(2)}{Γ(2)} && \text{(subst. y=1)} \\ &= \frac{4! 3! 2!}{5! 2! 2!} && \text{( subst. } x! = \Gamma(x+1) )\\ &= \frac{12}{24} && \text{ (simplifying)} \end{aligned} $$ ::: ::: {#exr-beta-bernoulli-4} After specifying our $Beta(2,2)$ prior for $\theta$, we observe 10 Bernoulli trials, 3 of which are successes. What is the posterior predictive probability that $y^∗$=1? for the next (11th) observation $y^∗$? Round your answer to two decimal places. ::: ::: {.solution .callout-tip collapse="true"} #### Solution: $$\begin{aligned} Beta(5,9) &= \frac{Γ(5+9)}{Γ(5+9+1)}\cdot \frac{Γ(5+y)}{Γ(5)} \cdot \frac{Γ(9+1−y)}{Γ(9)}\\ &= \frac{Γ(14)}{Γ(15)} \cdot \frac{Γ(6)}{Γ(5)} \cdot \frac{Γ(9)}{Γ(9)} && \text{(subst. y=1)}\\ &= \frac{13! 5! 8!}{14! 4! 8!} && \text{ (subst. x! =} \mathrm{Gamma}(x+1))\\ &= \frac{5}{14} && \text{(simplifying)} \end{aligned} $$ ::: :::::