Intro to Probability - FAQ Week 4
In this space I post the most common and recurrent questions of
the week. It can be used as additional material you can use to further
and test your understanding.
Question
How does one show that $\sum_{n=r}^\infty\binom{n-1}{r-1}p^r(1-p)^{n-r}=1?$
Answer
One begins with the geometric series and the validity of
$$
\sum_{m=0}^\infty(1-p)^m=\frac{1}{1-(1-p)}=\frac{1}{p},
$$
which is valid for $p\in(0,1)$ and takes $r-1$ derivatives to arrive
at
$$
(-1)^{r-1}\sum_{m=r-1}^\infty
m(m-1)\cdots(m-r+2)(1-p)^{m-r+1}=(-1)^{r-1}\frac{(r-1)!}{p^r},
$$
which rewrites as
$$
1=\sum_{m=r-1}^\infty \frac{m!}{(m-r+1)!(r-1)!}p^r(1-p)^{m-r+1}=
\sum_{m=r-1}^\infty \binom{m}{r-1}p^r(1-p)^{m-r+1}=
\sum_{n=r}^\infty \binom{n-1}{r-1}p^r(1-p)^{n-r}
$$
where the last identiy is obtained by changing the summation variable
from $m$ to $n=m+1$.
Question
Where does the Gaussian distribution come from?
Answer
There are many possible answers to this question. Here is a commonly
used derivation/explanation. As a general principle, the normal
distribution is the distribution that is observed when a large number
of measurements that are affected by the same noise (read have the
same distribution) are combined. Take for instance the example of
rolling a fair die $n$ times and adding the outcomes: how does the
distribution of sums look like? Equivalently consider rolling n
independent and fair dice and building the sum of their outcomes.
When $n$ is large the histogram more and more resembles a Gaussian
bell shaped curve. You can see this by running the experiment online
following this link.
This is a manifestation of the so-called central limit theorem,
which states that
$$
\frac{X_1+X_2+\dots+ X_n-n\mu}{\sqrt{n}\sigma}=
\frac{\frac{X_1+X_2+\dots+ X_n}{n}-\mu}{\sigma/\sqrt{n}}=Z_n
$$
exhibits the following convergence property
$$
P(Z_n\leq x)\to\Phi(x)\text{ as }n\to\infty,
$$
whenever $X_1,\dots,X_n$ are independent and identically distributed
random variables with common mean $\mu$ and common variance
$\sigma^2>0$. Recall that $\Phi$ is the cumulative distribution
function of a normally distributed random variable with mean 0 and
variance 1. Notice that $\frac{X_1+X_2+\dots+ X_n}{n}$ is the
average of the random variables and $\sigma/\sqrt{n}$ the standard
deviation of this average as follows from
$$
\operatorname{Var}\bigl( \frac{X_1+X_2+\dots+ X_n}{n}\bigr) =
\frac{1}{n^2}\operatorname{Var}\bigl( X_1+X_2+\dots+ X_n\bigr) =
\frac{1}{n^2}\bigl(\operatorname{Var}(X_1)+\operatorname{Var}(X_2)
+\dots+\operatorname{Var}(X_n)\bigr) =\frac{\sigma^2}{n}
$$