2086 Lecture 2 Expectation Variance And Probability Distributions

Lecture Note Expectation and Variance: Lecture 2 Notes (Part I).pdf Lecture Note Probability Distribution: Lecture 2 Notes (Part II).pdf

Next: 2086 Lecture 3 - Estimation and Maximum Likelihood

Expectation

Expectation is the expected value of the random variable $X$, if we do multiple tries, we expecting to get this result in average.

Discrerte RVs:

$$\mathbb{E}[X] = \sum_{x \in \mathcal{X}} x\, p(x)$$

Continuous RVs:

$$\mathbb{E}[f(X)] = \int_{\mathcal{X}} f(x)\,p(x)\,dx$$

For example, consider a uniform distribution, where $x=0,1$ and $p_0 = p_1 = 0.5$ Then $\mathbb{E}[X] = 0\times0.5 + 1\times0.5 = 0.5$

Rule of Expectation

$$\mathbb{E}[f(X)] \neq f(\mathbb{E}[X])$$

They are equal if and only if $f(x)=ax+b$, otherwise we might get result like $\mathbb{E}[X^2] =E[X]^2$ when $f(X) = x^2$ which is incorrect.

But we can rewrite it as:

$$\mathbb{E}_X[f(X)] = \mathbb{E}_{X_f}[X_f].$$

Where $X_f = f(X)$

Linearity of Expectation

$$\mathbb{E}[bX + c] = b\,\mathbb{E}[X] + c$$ $$\mathbb{E}_X[XY] = Y\,\mathbb{E}_X[X]$$ $$\mathbb{E}_{X,Y}[X + Y] = \mathbb{E}_X[X] + \mathbb{E}_Y[Y]$$

Variance

Variance measures the spread of data, i.e. how different it is from expectation to a random variable we draw.

$$\mathbb{V}[X] = \mathbb{E}[X^2] - \mathbb{E}[X]^2$$