Caution
Section omitted to comply with the Honor Code
Bayesian Statistics: From Concept to Data Analysis
Oren Bochman
Conditional Probability, Bayes’ Law, Homework, Honors
Section omitted to comply with the Honor Code
---
title: "Probability and Bayes' Theorem - M1L2HW2"
subtitle: "Bayesian Statistics: From Concept to Data Analysis"
categories:
- Bayesian Statistics
keywords:
- Conditional Probability
- Bayes' Law
- Homework
- Honors
---
::::: {.content-visible unless-profile="HC"}
::: {.callout-caution}
Section omitted to comply with the Honor Code
:::
:::::
::::: {.content-hidden unless-profile="HC"}
::: {#exr-honor1-1}
Which must be true if random variable X is a continuous RV with PDF $f(x)$?
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
- $f(x) \ge 0 \: \forall x$
- $\lim_{x \to \infty} f(x)=1$
:::
::: {#exr-honor1-2}
If $X\sim \mathrm{Exp}(3)$, what is the value of $\mathbb{P}r(X>1/3)$?
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
0.37
$$
\begin{aligned}
\mathbb{P}r(X>1/3) &= \int_{1/3}^\infty 3 e^{−3x} dx \\
&= −e|_{−3x}^\infty \\
&= 0- (-e^{-3/3}) = e^-1 = 0.368 \end{aligned}
$$
:::
::: {#exr-honor1-3}
Suppose $X∼Uniform(0,2)$ and $Y∼Uniform(8,10)$. What is the value of $\mathbb{E}(4X+Y)$?
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$$
\begin{aligned} \mathbb{E}(4X+Y)&=4\mathbb{E}(X)+\mathbb{E}(Y) \\
&=4(1)+9 \\
&= 13\end{aligned}
$$
:::
[Adding normals]{.column-margin} For the following questions:
Suppose $X∼N(1,25)$ and $Y∼N(−2,9)$ and that $X$ and $Y$ are independent.
We have $Z=X+Y∼N(μ,σ^2)$ because the sum of normal random variables also follows a normal distribution.
::: {#exr-honor1-4}
[Adding normals]{.column-margin} What is the value of $\mu$?
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$\mu=\mathbb{E}[Z]=\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]=1+(−2)$
:::
::: {#exr-honor1-5}
[Adding normals]{.column-margin} What is the value of $σ^2$?
:::
::: {.callout-note collapse="true"}
#### Hint
If two random variables are `independent`, the variance of their sum is the sum of their variances.
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$=Var(Z)=Var(X+Y)=Var(X)+Var(Y)=25+9=34$
:::
::: {#exr-honor1-6}
[Adding normals]{.column-margin} If RVs X and Y are not independent, we still have
$$
\mathbb{E}(X+Y)=\mathbb{E}(X)+\mathbb{E}(Y)
$$
but now
$$
\mathbb{V}ar(X+Y)=Var(X)+Var(Y)+2 Cov(X, Y)
$$
where
$$
Cov(X, Y)=\mathbb{E}[(X−\mathbb{E}(X))\cdot(Y−\mathbb{E}(Y))]
$$
is called the **covariance** between $X$ and $Y$.
::: {.callout-important}
A convenient identity for calculating variance:
$$
\mathbb{V}ar(X) = \mathbb{E}[(X−\mathbb{E}[X])^2 ]=\mathbb{E}[X^2]−(\mathbb{E}[X])^2
$$
:::
Which of the following is an analogous expression for the covariance of X and Y?
:::
::: {.callout-note collapse="true"}
### Hint
1. Expand the terms inside the expectation in the definition of $Cov(X, Y)$
2. Recall that $\mathbb{E}(X)$ and $\mathbb{E}(Y)$ are just constants.
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$$
Cov(X,Y) = \mathbb{E}(XY)−\mathbb{E}(X)\mathbb{E}(Y)
$$
since
$$
\begin{aligned}
Cov(X,Y)&\equiv \mathbb{E}[(X−\mathbb{E}[X])(Y−\mathbb{E}[Y])] \\
&= \mathbb{E}[XY−X\mathbb{E}(Y)−\mathbb{E}(X)Y+\mathbb{E}(X)\mathbb{E}(Y)] \\
&= \mathbb{E}[XY]−\mathbb{E}[X\mathbb{E}(Y)]−\mathbb{E}[\mathbb{E}(X)Y]+\mathbb{E}[\mathbb{E}(X)\mathbb{E}(Y)] \\
&= \mathbb{E}[XY]- \mathbb{E}(X)\mathbb{E}(Y)− \cancel{ \mathbb{E}(X)\mathbb{E}(Y)}+\cancel{\mathbb{E}(X)\mathbb{E}(Y)}
\end{aligned}
$$
:::
::: {#exr-honor1-7}
[Adding normals]{.column-margin} Consider again $X∼N(1,5^2)$ and $Y∼N(−2,3^2)$, but this time $X$ and $Y$ are not independent. Then $Z=X+Y$ is still normally distributed with the same mean found in (@exr-honor1-4). What is the variance of Z if $\mathbb{E}(XY)=−5$?
:::
::: {.callout-note collapse="true"}
#### Hint
Use the formulas introduced in Question (@exr-honor1-6).
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$$
\begin{aligned}
Var(Z) &= Var(X) + Var(Y) + 2Cov(X,Y) \\
&= 25 + 9 + 2Cov(X,Y) \\
&= 34 + 2 (\mathbb{E}[XY] − \mathbb{E}[X] \mathbb{E}[Y] ) \\
&= 34 + 2 (−5−1(−2))=34−2(3) =28
\end{aligned}
$$
:::
::: {#exr-honor1-8}
Use the definition of conditional probability to show that for events A and B, we have
$$
\begin{aligned}
\mathbb{P}r(A \cap B) = \mathbb{P}r(B \mid A)\mathbb{P}r(A) = \mathbb{P}r(A \mid B)\mathbb{P}r(B)
\end{aligned}
$$
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
$$
\begin{aligned}
\mathbb{P}r(B \mid A)= \frac{\mathbb{P}r(A \cap B)}{\mathbb{P}r(A)} \\
\mathbb{P}r(B \mid A)\mathbb{P}r(A)=\mathbb{P}r(A \cap B)
\end{aligned}
$$
:::
::: {#exr-honor1-9}
Show that the two expressions for independence $\mathbb{P}r(A\mid B)=\mathbb{P}r(A)$ and $\mathbb{P}r(A∩B) = \mathbb{P}r(A)\mathbb{P}r(B)$ are equivalent.
:::
::: {.solution .callout-tip collapse="true"}
#### Solution:
Plug these expressions into those from (@exr-honor1-8).
:::
::::