Review of Convergence of Random Sequences

Definitions, Propositions and Theorems

Characteristic Function

The characteristic function for a random variable $X$ is
$${\varphi }_X(t)=E[e^{itX}]=E[\cos (tX)+i\sin (tX)].$$

Fourier Inversion Formula

$$F^{\prime }(x)=\frac{1}{2\pi }{\int }_{-\infty }^{+\infty }{e}^{-itx}{\varphi }_X(t)dt.$$

Levy-Cramer continuity theorem

Let { X n } \{X_n\} {Xn​} be a random sequence with distribution function { F n } \{F_n\} {Fn​}. Let $X$ be a random variable with distribution $F$. The following are equivalent:

1. $X_n$ converges weakly to $X$.
2. $F_n(x)$ converges to $F(x)$ for every continuous point $x\in \mathbb{R}$.
3. $E[f(X)]\to E[f(X)]$ for every bounded continuous function $f$(equivalently bounded Lipschitz function).
4. ${\varphi }_{X_n}(t)$ converges pointwisely to ${\varphi }_X(t)$.

Theorem 1

If a function $\varphi (t)$ is continuous at 0 and the limit of the characteristic functions of a random sequence, then $\varphi (t)$ must be the characteristic function of some probability measure.

Markov Inequality

For a non-negative random variable $X$, we have $$pr(X\ge t)\le \frac{E[X]}{t}.$$ An instant collary is
$$pr(X\ge t)\le \frac{E[X^p]}{t^p}.$$

proof:
$$\begin{array}{rl}E[X]& ={\int }_0^{\infty }xdF(x)\\ & \ge {\int }_t^{\infty }xdF(x)\\ & \ge t\cdot pr(X\ge t).\end{array}$$

$L_p$ Convergence implies Convergence in prob.

A direct conclusion followed by the Markov Inequality.

Weak Law of Large Numbers (WLLN)

Version 1:
The average mean of uncorrelated random variables with finite mean $\mu$ and variance converges to $\mu$ in $L_2$.
version 2:
The average mean of i.i.d random variables with finite mean $\mu$ converges to $\mu$ in prob…
proof:
${\varphi }_{\frac{S_n}{n}}(t)={\varphi }_X^n(\frac{t}{n})$. Taylor series lead to
$${\varphi }_X(\frac{t}{n})=1+\frac{i\mu t}{n}+o(\frac{1}{n}).$$Note ${\varphi }_X(\frac{t}{n})-1\to \frac{i\mu t}{n}$. Since $(1+\frac{i\mu t}{n}{)}^n\to e^{i\mu t}$, we have ${\varphi }_{\frac{S_n}{n}}(t)\to e^{i\mu t}$.

Borel-Cantelli Lemma

If ${\sum }_{n=1}^{\infty }pr(A_n)<\infty$, we have
$$pr(A_{n}i.o)(=pr(\underset{n}{\mathrm{limsup}}{A}_n))=0.$$Otherwise if additionally $A_n$ are independent, we have
$$pr(A_{n}i.o)=1.$$

Proposition 2

$X_n\to X$ IFF every subsequence of $X_n$ has a further subsequence that converges a.s. to $X$.

Strong Law of Large Numbers (SLLN)

The average mean of i.i.d integrable random variables converges to the mean a.s…

Kolmogorov’s Inequality

An improvement of Chebyshev’s Inequality.
$$pr(\underset{1\le k\le n}{\sup }|S_k|>\epsilon )<\frac{{\sum }_{i=1}^n{\sigma }_j^2}{{ϵ}^2}.$$

Levy’s Theorem

If $X_n$ are independent, then (i)$S_n$ converges weakly; (ii)$S_n$ converges in prob.; (iii)$S_n$ converges a.s.; are equivalent.

Kolmogorov’s one series Theorem

If $X_n$ are independent r.v. with mean 0 and ${\sum }_{n=1}^{\infty }var(X_n)<\infty$, then $S_{\infty }$ converges a.s…

Kolmogorov’s three-series theorem

Let $X_n$ be independent. Let $Y_i=X_i{1}_{|X_i|\le A}$ with $A>0$. Then $S_{\infty }$ converges a.s. IFF

1. ${\sum }_{n=1}^{\infty }pr(|X_n|>A)<\infty .$
2. ${\sum }_{n=1}^{\infty }E{Y}_n$ converges.
3. ${\sum }_{n=1}^{\infty }var(Y_n)<\infty$.

Kolmogorov’s 0-1 law

Let F n ′ = σ { X n , X n + 1 , ⋯   } \mathcal{F}'_n=\sigma\{X_n,X_{n+1},\cdots\} Fn′​=σ{Xn​,Xn+1​,⋯} and $\mathcal{T}={\cap }_n{\mathcal{F}}_n^{\prime }$ be the tail sigma-field. If $X_n$ are independent and $A\in \mathcal{T}$ then $pr(A)=0$ or $1$.

Kronecker’s lemma

If $a_n\uparrow \infty$ and ${\sum }_{i=1}^{\infty }\frac{x_i}{a_i}$ converges then $\frac{{\sum }_{i=1}^n{x}_i}{a_n}$ converges.

Central Limit Theorem (CLT)

For i.i.d r.v’s. with mean $\mu$ and $var(X_i)={\sigma }^2<\infty$, $\frac{(S_n-n\mu )}{\sqrt{n}\sigma }$ converges weakly to the standard normal distribution.

Lindeberg’s Theorem

Let $F_i$ be the distribution of $X_i$.
$$\underset{n}{\lim }\frac{1}{s_n^2}\sum _{i=1}^n{\int }_{|x|\ge ϵs_n}{x}^{2}d{F}_i(x)=0$$ for each $ϵ>0$ is sufficient for CLT to hold.

Reference

Durrett, Rick. Probability: theory and examples. Vol. 49. Cambridge university press, 2019.
S.R.S.Varadhan, Probability Theory (Courant Lecture Notes), 2000