83 Homework on Estimating the number of components in Bayesian settings - M5L09HW2

Caution

Section omitted to comply with the Honor Code

--- title : 'Homework on Estimating the number of components in Bayesian settings - M5L09HW2' subtitle : 'Bayesian Statistics: Mixture Models' categories: - Bayesian Statistics keywords: - Mixture Models - Homework - Number of Components - Bayesian Estimation --- ::::: {.content-visible unless-profile="HC"} ::: {.callout-caution} Section omitted to comply with the Honor Code ::: ::::: ::::: {.content-hidden unless-profile="HC"} ::: {#exr-1} Let $K^∗$ be the prior expected number of occupied components in a mixture model with $K$ components where the weights are given a Dirichlet prior $(w_1, \ldots, w_K) \sim \text{Dir}(2K, \ldots, 2K)$. If you have $n = 400$ observations, what is the expected number of occupied components, $E(K^∗)$ according to the exact formula we discussed in the lecture? Round your answer to one decimal place. ::: ::: {.callout-note .solution collapse="true"} ## Solution To compute the expected number of occupied components, we can use the formula: $$ E(K^*) = \sum_{i=0}^n \frac{\alpha}{\alpha+i-1} $$ Thus, the expected number of occupied components is given by: ```{r} #| label: exr-1-sol E.Kstar = 0 for (i in 1:400){ E.Kstar = E.Kstar + 2/(2+i-1) } round(E.Kstar,1) ``` 11.1 ::: ::: {#exr-2} Consider the same setup as the previous question, what is the expected number of occupied components, $E(K^∗)$ according to the exact formula we discussed in the lecture if $n = 100$ instead? Round your answer to one decimal place. ::: ::: {.callout-note .solution collapse="true"} ## Solution ```{r} n = 100 alpha = 2 EKstar = 0 for(i in 1:n){ EKstar = EKstar + alpha/(alpha + i -1) } print(EKstar) ``` 8.4 ::: ::: {.callout-note .solution collapse="true"} ## Solution What would be the answer to the previous question if you used the approximate formula instead of the exact formula? Remember to round your answer to one decimal place. ::: ::: {#exr-3-sol} ```{r} #| label: exr-3-sol alpha = 2 n = 100 Kstar = alpha*log((n+alpha-1)/alpha) round( Kstar,1) ``` 7.8 ::: ::: {#exr-4} If you have $n = 200$ observations and a priori expect the mixture will have about 2 occupied components (i.e., $E(K^∗) \approx 2$ a priori), what value of $\alpha$ should you use for the prior $(w_1, \ldots, w_K) \sim \text{Dir}(\alpha K, \ldots, \alpha K)$. Use the approximation $E(K^∗) \approx \alpha \log\left(\frac{n + \alpha - 1}{\alpha}\right)$ to provide an answer, which should be rounded to two decimal ::: ::: {.callout-note .solution collapse="true"} ## Solution ```{r} n = 200 alpha = 2 alpha*log((n+alpha-1)/alpha) ``` 0.31 ::: :::::