Let \(X\) be a random variable with \(\mu_1=\mathbf{E}(X)\) and \(\mu_2=\mathbf{E}(X^2)\). For any number \(a\) define the mean squared error
\[J(a)=\mathbf{E}\big[(X-a)^2\big] \]
and the absolute error
\[K(a)=\mathbf{E}\big[|X-a|\big] \]
- Write \(J(a)\) in terms of \(a\), \(\mu_1\), and \(\mu_2\) ?
- Use the above answer to calculate \(\frac{d J(a)}{d\, a}\) .
- Find the \(a\) which is the solution to \(\frac{d J(a)}{d\, a}=0 ?\) Comment on this answer in light of the name “Expected Value” and argue that it is actually a minimum.
- Assume that \(X\) only takes values \(\{x_1,x_2,\dots,x_n\}\). Use the fact that
\[ \frac{d\ }{d a} |x-a| = \begin{cases} -1 & \text{if \(a < x\)}\\
1 & \text{if \(a > x\)}\end{cases}
\]
to show that as long as \(a \not\in \{x_1,x_2,\dots,x_n\}\) one has
\[ \frac{d K(a)}{d\, a} =\mathbf{P}(X<a) – \mathbf{P}(X>a)\] - Now show that if \( a \in (x_k,x_{k+1})\) then \(\mathbf{P}(X<a) – \mathbf{P}(X>a) = 2\mathbf{P}(X \leq x_k) – 1\).
- The median is any point \(a\) so that both \(\mathbf{P}(X\leq a) \geq \frac12 \) and \(\mathbf{P}(X\geq a) \geq\frac12\). Give an example where the median is not unique. (That is to say there is more than one such \(a\).
- Use the above calculations to show that if \(a\) is any median (not equal to one of the \(x_k\)), then it solves \(\frac{d K(a)}{d\, a} =0\) and that it is a minimizer.
