Lecture 1. Limits and Sequences

Many analytical problems can only be solved by performing, not finitely many, but rather countably many steps. In order to make sure that the steps do indeed lead somewhere, convergence has to be established. Here we set up an example to motivate the introduction of the convergence concept. The example builds upon the one used in Lecture 0 in the construction of real numbers. Assume that real numbers have not been introduced yet and consider the function $$ f:\mathbb{Q}\to \mathbb{Q}\, ,\: x\mapsto x^2-2 $$ as well as the question as to where its graph intercepts the $x$-axis. If we try and identify the positive intersection point, we could start with a guess $x_0\in \mathbb{Q}$ and then see if we can improve it. To do so, we take the tangent line to the graph of $f$ at the point $\bigl( x_0,f(x_0)\bigr)$ and determine the location $x_1$ where it intersects the $x$-axis. See Figure. It appears to deliver a better approximation to the zero we are after. By repeating this construction we can arguably obtain an ever better approximation. Observing that \begin{multline*} f(x)=x^2-2=(x-x_0)^2+2xx_0-x_0^2-2\\ =2x_0(x-x_0)+x_0^2-2+(x-x_0)^2\approx 2x_0(x-x_0)+x_0^2-2\, , \end{multline*} which indicates that the line $y=2x_0(x-x_0)+x_0^2-2$ is a good approximation of $f$ near $x=x_0$ (better than first order since we are neglecting quadratic terms only). Using this tangent line, we can then compute its intersection with the $x$-axis as $$ x_1=\frac{x_0}{2}+\frac{1}{x_0}\in \mathbb{Q}\, . $$ Repeating this procedure as alluded to above we obtain a recursively defined sequence $(x_n)_{n\in \mathbb{N} }$ of rational numbers given by $$\begin{cases} x_0\in \mathbb{Q}\, ,&\\ x_{n+1}=\frac{x_n}{2}+\frac{1}{x_n}\in \mathbb{Q}\, ,& \end{cases} $$ for $n\geq 1$. The construction seems to indicate that the sequence is approaching the positive solution of $x^2-2=0$. In Lecture 0 we used this very sequence as a prototypical example of how to introduce real numbers as "limits'' of sequences of rational numbers. The formal construction was based on Cauchy sequences, which can be defined without any reference to a limit.

We remark that the above construction of an approximating sequence to the solution of an equation is known with the name of Newton's method and has widespread applications to more general (higher dimensional) problems.

A criterion for convergence

Definition (Boundedness)

We say that a sequence $(x_n)_{n\in\mathbb{N}}$ is bounded above (below) iff there exists a real number $M\in \mathbb{R}$ such that $$ x_n\leq M\text{ for all }n\in \mathbb{N}\quad(x_n\geq M\text{ for all }n\in \mathbb{N})\, . $$

Theorem (Monotone Sequences)

Every monotone increasing sequence $(x_n)_{n\in\mathbb{N}}$ of real numbers, i.e. satisfying $$ x_n\leq x_{n+1},\: n\in \mathbb{N}, $$ which is also bounded above, converges.

Since $\mathbb{R}$ is complete, it is enough to show that $(x_n)_{n\in\mathbb{N}}$ is a Cauchy sequence. Towards a contradiction, assume that this is not case, i.e. that $$ \exists\: \varepsilon>0\text{ s.t. }\forall k\in \mathbb{N}\:\exists\: n_k\geq k, l_k\geq 0\text { with }x_{n_k+l_k}-x_{n_k}>\varepsilon, $$ and, without loss of generality, that $n_{k+1}\geq n_k+l_k$. Then one has that \begin{align*} x_{n_k+l_k}&\geq x_{n_k+l_k}-x_{n_k}+x_{n_k}\\&> \varepsilon+x_{n_{k-1}+l_{k-1}}-x_{n_{k-1}}+x_{n_{k-1}}\geq \dots\\&\geq (k-1)\varepsilon+x_{n_1}, \end{align*} and $k$ can be found so large that $$ M < x_{n_1}+(k-1)\varepsilon\leq x_{n_k+l_k}\leq M. $$ This contradiction concludes the proof.

Corollary

Any non-empty subset $S$ of reals which is bounded above (below) has a supremum or least upper bound (infimum or greatest lower bound) $\sup S$ ($\,\inf S$).

Since $S\neq\emptyset$, there is $x_1\in S$. Now, either $x\leq x_1$ for all $x\in S$ or there is $x_2\in S$ such that $x_2>x_1$. Again, either $x\leq x_2$ for all $x\in S$ or there is $x_3\in S$ such that $x_3>x_2$. This procedure either ends after $k$ steps, in which case $x_k=\sup S$, or yields an increasing sequence $(x_n)_{n\in\mathbb{N}}$ in $S$. As $S$ is bounded, so is this sequence and, by the previous theorem, is therefore Cauchy. As such it possesses a limit $x_\infty=\sup S$. Notice that, by construction, there cannot be a smaller lower bound.

Exercise

In the above proof, the contructed sequence may not deliver an upper bound at all. Take $S=[0,2]$ and $x_n=1-1/n$, for instance. It is therefore necessary to make sure that the sequence "grows" enough.

Since $S$ is not empty, we claim that $x_1\in S$ exists such that $x_1+1$ is an upper bound for $S$. Take any $x_1^1\in S$. If $x_1^1+1$ is an upper bound, set $x_1=x^1_1$, else pick $x_1^2\in S$ with $x_1^1+1<x_1^2$ and set $x_1=x_1^2$ if $x_1^2+1$ is an upper bound. If $x_1^2+1$ is not an upper bound, repeat the process. After $k$ steps we will have found an element $x^k_1\in S$ with $x_1^1+k<x^k_1$ and thus the procedure has to end before $x^k_1>M$, where $M$ is an upper bound for $S$. After $x_1\in S$ has been found, choose $x_2\in S$ such that $x_2+1/2$ is an upper bound for $S$. Clearly either $x_2=x_1$ does the job or we can proceed as in the construction of $x_1$ to obtain $x_2\in S$ with the desired property. In this way we obtain a monotone non-decreasing sequence $(x_n)_{n\in \mathbb{N}}$ in $S$ such that $$ x\leq x_n+1/n\: \forall x\in S $$ Letting $n\to\infty$ and using the Theorem on Monotone Sequences above, we obtain a limit $x_\infty$ for which it holds that $$ x\leq x_\infty\: \forall x\in S. $$ Since $S\ni x_n\to x_\infty$ as $n\to\infty$, this upper bound is minimal.

Remark

Example

We choose the following strategy:
(i) Prove $(x_n)_{n\in\mathbb{N}}$ is increasing and bounded above, so that $\lim_{n \to \infty} x_n = x$ exists in $\mathbb{R}$.
(ii) Find the limit $x$.
For (i) we use mathematical induction: First observe that $$ x_1=1\, ,\: x_2 = \sqrt{3+1}=2>x_1\text{ (induction basis)} $$ Then assume that $x_n > x_{n-1}$ and show that $x_{n+1} >x_n$ (induction step). Indeed $$ x_{n+1} - x_n = \sqrt{3+x_n} - \sqrt{3+x_{n-1}} = \frac{x_n - x_{n-1}}{\sqrt{3+x_n}+ \sqrt{3+x_{n-1}}}>0. $$ We use induction again to show that $(x_n)_{n\in\mathbb{N}}$ is bounded above by $3$. In fact $x_1 = 1<3$ and, assuming that $x_n \le 3$, we see that $$ x_{n+1} =\sqrt{3+x_n}\le\sqrt{3+3}=\sqrt{6} <3\, . $$ Convergence follows. As to (ii) we observe that $$ x=\sqrt{3+x} \iff x^2 = 3+x \iff x^2 - x -3 = 0\, . $$ since $\lim _{n\to\infty}x_n=\lim_{n\to\infty}x_{n+1}=x$. It follows that $x = \frac{1\pm \sqrt{13}}{2}$ and, since $x\ge 1$, we conclude that $$ x=\frac{1 + \sqrt{13}}{2}\, . $$

Theorem (Euler Number)

$\lim_{n\to\infty} (1+\frac{1}{n})^n$ exists, is called Euler's number, and is denoted by $e$.

Exercise

Example

Elementary Properties of Limits.

Proposition (Linearity)

The following hold for any two convergent sequences $(x_n)_{n\in\mathbb{N}},(y_n)_{n\in\mathbb{N}}$
(i) $\lim_{n\to\infty} (x_n+y_n)=\lim_{n\to\infty} x_n +\lim_{n\to\infty} y_n$.
(ii) $\lim_{n\to\infty} (x_n-y_n)=\lim_{n\to\infty} x_n -\lim_{n\to\infty} y_n$.
(iii) $\lim_{n\to\infty} (x_ny_n)=(\lim_{n\to\infty} x_n )(\lim_{n\to\infty} y_n)$.
(iv) $\text{ If }\lim_{n\to\infty} y_n\ne 0,\text{ then } \lim_{n\to\infty}\frac{x_n}{y_n}=\frac{\lim_{n\to\infty} x_n}{\lim_{n\to\infty} y_n}$.

We only prove (iii). The other claims can be obtained similarly. Observe that convergence of a sequence $(x_n)_{n\in\mathbb{N}}$ to a limit $x_\infty\in\mathbb{R}$ implies that the sequence is bounded. Indeed, given $\varepsilon>0$, there is $N\in\mathbb{N}$ such that $$ |x_n-x_\infty|\leq\varepsilon\text{ for }n\geq N, $$ but latter is equivalent to $$ x_\infty-\varepsilon\leq x_n\leq x_\infty+\varepsilon\text{ for } n\geq N, $$ and hence $\min(m_N,x_\infty-\varepsilon)\leq x_n\leq \max(M_N,x_\infty+\varepsilon)$, where $m_N=\min _{n=1,\dots,N-1}x_n$ and $M_N=\max _{n=1,\dots,N-1}x_n$. By assumption we can therefore find $M\in\mathbb{R}$ such that $$ \sup_{n\in\mathbb{N}}|x_n|\leq M\text{ and }\sup_{n\in\mathbb{N}}|y_n|\leq M. $$ Given $\varepsilon>0$, $N$ can be found such that $$ |x_n-x_\infty|\leq\frac{\varepsilon}{2M}\text{ and } |y_n-y_\infty|\leq\frac{\varepsilon}{2M}\text{ for }n\geq N. $$ Then it holds $$ |x_ny_n-x_\infty y_\infty|\leq |x_n-x_\infty||y_n|+ |x_\infty||y_n-y_\infty|\leq M \frac{\varepsilon}{2M}+M \frac{\varepsilon}{2M}\leq \varepsilon $$ and the proof is complete.

Theorem (Squeeze Theorem)

Assume that $x_n\le y_n\le z_n\text{ for } n\ge 1\text{ and that} \lim_{n\to\infty} x_n=\lim_{n\to\infty} z_n=x$. Then $\lim_{n\to\infty} y_n=x$.

Example

Convergence implies that for any $\epsilon>0$ there is a $N$ such that $$ |x_n-x|\leq\epsilon\text{ provided }n\ge N\, . $$ Notice that \begin{equation*} \Big |\frac{x_1+\cdots +x_n}{n}-x\Big |=\frac{1}{n}\Big|{x_1-x+\cdots+x_{N-1}-x +x_N-x+\cdots+x_n-x}\Big|\le \frac{(N-1)(M+|x|)}{n}+\epsilon \le \frac{2(N-1)M}{n}+\epsilon, \end{equation*} where $M$ is a bound for $(|x_n|)_{n\in\mathbb{N}}$. Therefore $$ \Big |\frac{x_1+\dots +x_n}{n}-x\Big |<2\epsilon, $$ if $n\ge N_1=\max\{N, 1+\frac{2(N-1)M}{\epsilon}\}$, which is the claim.

Limits superior and inferior

Definition (Limits Superior and Inferior)

The limits superior and inferior of a sequence $(x_n)_{n\in\mathbb{N}}$ are given by \begin{gather*} \lim \sup_{n\to \infty} x_n = \lim_{n\to \infty} \bar{x}_n\\ \lim \inf_{n\to \infty} x_n = \lim_{n\to \infty} \underline{x}_n \end{gather*}

Remark

Theorem

$\lim_{n \to \infty} x_n$ exists if and only if $$ \lim_{n\to \infty} \inf x_n =\lim_{n\to \infty} \sup x_n\, . $$

Exercise

Examples

(b) Since \begin{align*} \overline{x_n+y_n} & = \sup\{x_m + y_m : m\ge n\}\\ & \le \sup\{x_m: m\ge n\} + \sup\{y_m: m\ge n\}\\ & =\overline{x_n} + \overline{y_n} \end{align*} we have that $$ \lim_{n\to \infty} \overline{x_n + y_n} \le \lim_{n \to \infty} (\overline{x_n} +\overline{y_n} )= \lim_{n\to \infty} \overline{x_n} + \lim_{n \to \infty} \overline{y_n}\, , $$ and the claim follows.

Definition (Subsequence)

Given a sequence $(x_n)_{n\in\mathbb{N}}$ and a sequence of indeces $(n_j)_{j\in\mathbb{N}}$, we say that $(x_{n_j})_{j\in \mathbb{N}}$ is a subsequence of $(x_n)_{n\in\mathbb{N}}$ if $n_1 < n_2<\dots$.

Example

Theorem

If $(x_n)_{n\in\mathbb{N}}$ is a sequence in $\mathbb{R}$, then $$ \lim_{n \to \infty} x_n = x\iff \lim_{k \to \infty}x_{n_k} = x\text{ for all subsequences }(x_{n_k})_{k\in \mathbb{N}}\text{ of }(x_n)_{n\in\mathbb{N}}\, . $$

It is straightforward and left as an exercise.

Proposition

Given a bounded sequence $(x_n)_{n\in\mathbb{N}}$ in $\mathbb{R}$, there exists a subsequence $(x_{n_k})_{k\in \mathbb{N} }$ such that $$ \limsup_{n \to \infty} x_n = \lim_{k \to \infty}x_{n_k}\, . $$

We sketch the idea of the proof. Since $$ \limsup_{n \to \infty} x_n = \lim_{n \to \infty} \overline{x}_n $$ where $\overline{x}_n = \sup \{x_n: m \ge n\}$ and $\bar{x}_k - 1/k$ is not an upper bound, we can choose $n_k\geq k$ inductively such that $$ \bar{x}_k \ge x_{n_k} >\bar{x}_k - 1/k $$ and such that $n_1 < n_2 < \dots$ Then $\lim_{k\to\infty} x_{n_k}=\lim\sup _{n\to\infty}x_n$.

Theorem (Bolzano-Weierstrass)

Every bounded sequence in $\mathbb{R}$ has a convergent subsequence

Either the set $\{ x_n\, |\, n\in \mathbb{N}\}$ is finite or not. In the first case, one value $v$ must be repeat infinitely many times in the sequence. Then, there is a subsequence with $x_{n_k}=v$ for $k\in \mathbb{N}$. In the second case the set of values is infinite and, by boundedness, we can find $m\leq M$ such that $$ \{x_n\, |\, n\in \mathbb{N}\}\subset I_0:=[m, M]\, . $$ Then we define subintervals $I_k$ recursively as follows. Split $I_k$ in half and choose one half that contains infinitely many sequence elements (there must be at least one!) and call it $I_{k+1}$. Then pick $x_{n_k}\in I_k$ and observe that $$ |x_{n_k} - x_{n_\ell} | \le \frac{M-m}{2^k}\text{ for }\ell>k\, , $$ which makes $(x_{n_k})_{k\in \mathbb{N}}$ a Cauchy sequence. It therefore has a limit.