5 Probability and Bayes’ Theorem - M1L2HW2

Caution

Section omitted to comply with the Honor Code

--- title: "Probability and Bayes' Theorem - M1L2HW2" subtitle: "Bayesian Statistics: From Concept to Data Analysis" categories: - Bayesian Statistics keywords: - Conditional Probability - Bayes' Law - Homework - Honors --- ::::: {.content-visible unless-profile="HC"} ::: {.callout-caution} Section omitted to comply with the Honor Code ::: ::::: ::::: {.content-hidden unless-profile="HC"} ::: {#exr-honor1-1} Which must be true if random variable X is a continuous RV with PDF $f(x)$? ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: - $f(x) \ge 0 \: \forall x$ - $\lim_{x \to \infty} f(x)=1$ ::: ::: {#exr-honor1-2} If $X\sim \mathrm{Exp}(3)$, what is the value of $\mathbb{P}r(X>1/3)$? ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: 0.37 $$ \begin{aligned} \mathbb{P}r(X>1/3) &= \int_{1/3}^\infty 3 e^{−3x} dx \\ &= −e|_{−3x}^\infty \\ &= 0- (-e^{-3/3}) = e^-1 = 0.368 \end{aligned} $$ ::: ::: {#exr-honor1-3} Suppose $X∼Uniform(0,2)$ and $Y∼Uniform(8,10)$. What is the value of $\mathbb{E}(4X+Y)$? ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $$ \begin{aligned} \mathbb{E}(4X+Y)&=4\mathbb{E}(X)+\mathbb{E}(Y) \\ &=4(1)+9 \\ &= 13\end{aligned} $$ ::: [Adding normals]{.column-margin} For the following questions: Suppose $X∼N(1,25)$ and $Y∼N(−2,9)$ and that $X$ and $Y$ are independent. We have $Z=X+Y∼N(μ,σ^2)$ because the sum of normal random variables also follows a normal distribution. ::: {#exr-honor1-4} [Adding normals]{.column-margin} What is the value of $\mu$? ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $\mu=\mathbb{E}[Z]=\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]=1+(−2)$ ::: ::: {#exr-honor1-5} [Adding normals]{.column-margin} What is the value of $σ^2$? ::: ::: {.callout-note collapse="true"} #### Hint If two random variables are `independent`, the variance of their sum is the sum of their variances. ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $=Var(Z)=Var(X+Y)=Var(X)+Var(Y)=25+9=34$ ::: ::: {#exr-honor1-6} [Adding normals]{.column-margin} If RVs X and Y are not independent, we still have $$ \mathbb{E}(X+Y)=\mathbb{E}(X)+\mathbb{E}(Y) $$ but now $$ \mathbb{V}ar(X+Y)=Var(X)+Var(Y)+2 Cov(X, Y) $$ where $$ Cov(X, Y)=\mathbb{E}[(X−\mathbb{E}(X))\cdot(Y−\mathbb{E}(Y))] $$ is called the **covariance** between $X$ and $Y$. ::: {.callout-important} A convenient identity for calculating variance: $$ \mathbb{V}ar(X) = \mathbb{E}[(X−\mathbb{E}[X])^2 ]=\mathbb{E}[X^2]−(\mathbb{E}[X])^2 $$ ::: Which of the following is an analogous expression for the covariance of X and Y? ::: ::: {.callout-note collapse="true"} ### Hint 1. Expand the terms inside the expectation in the definition of $Cov(X, Y)$ 2. Recall that $\mathbb{E}(X)$ and $\mathbb{E}(Y)$ are just constants. ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $$ Cov(X,Y) = \mathbb{E}(XY)−\mathbb{E}(X)\mathbb{E}(Y) $$ since $$ \begin{aligned} Cov(X,Y)&\equiv \mathbb{E}[(X−\mathbb{E}[X])(Y−\mathbb{E}[Y])] \\ &= \mathbb{E}[XY−X\mathbb{E}(Y)−\mathbb{E}(X)Y+\mathbb{E}(X)\mathbb{E}(Y)] \\ &= \mathbb{E}[XY]−\mathbb{E}[X\mathbb{E}(Y)]−\mathbb{E}[\mathbb{E}(X)Y]+\mathbb{E}[\mathbb{E}(X)\mathbb{E}(Y)] \\ &= \mathbb{E}[XY]- \mathbb{E}(X)\mathbb{E}(Y)− \cancel{ \mathbb{E}(X)\mathbb{E}(Y)}+\cancel{\mathbb{E}(X)\mathbb{E}(Y)} \end{aligned} $$ ::: ::: {#exr-honor1-7} [Adding normals]{.column-margin} Consider again $X∼N(1,5^2)$ and $Y∼N(−2,3^2)$, but this time $X$ and $Y$ are not independent. Then $Z=X+Y$ is still normally distributed with the same mean found in (@exr-honor1-4). What is the variance of Z if $\mathbb{E}(XY)=−5$? ::: ::: {.callout-note collapse="true"} #### Hint Use the formulas introduced in Question (@exr-honor1-6). ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $$ \begin{aligned} Var(Z) &= Var(X) + Var(Y) + 2Cov(X,Y) \\ &= 25 + 9 + 2Cov(X,Y) \\ &= 34 + 2 (\mathbb{E}[XY] − \mathbb{E}[X] \mathbb{E}[Y] ) \\ &= 34 + 2 (−5−1(−2))=34−2(3) =28 \end{aligned} $$ ::: ::: {#exr-honor1-8} Use the definition of conditional probability to show that for events A and B, we have $$ \begin{aligned} \mathbb{P}r(A \cap B) = \mathbb{P}r(B \mid A)\mathbb{P}r(A) = \mathbb{P}r(A \mid B)\mathbb{P}r(B) \end{aligned} $$ ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: $$ \begin{aligned} \mathbb{P}r(B \mid A)= \frac{\mathbb{P}r(A \cap B)}{\mathbb{P}r(A)} \\ \mathbb{P}r(B \mid A)\mathbb{P}r(A)=\mathbb{P}r(A \cap B) \end{aligned} $$ ::: ::: {#exr-honor1-9} Show that the two expressions for independence $\mathbb{P}r(A\mid B)=\mathbb{P}r(A)$ and $\mathbb{P}r(A∩B) = \mathbb{P}r(A)\mathbb{P}r(B)$ are equivalent. ::: ::: {.solution .column-screen-inset-right .callout-tip collapse="true"} #### Solution: Plug these expressions into those from (@exr-honor1-8). ::: ::::