In this section we discuss some topics from measure theory that are a bit more advanced than the topics in the previous sections of this chapter. However, measure-theoretic ideas are essential for a deep understanding of probability, since probability is itself a measure. In particular, σ-algebras play a fundamental role, even for applied probability, in encoding the state of information about a random experiment.
Suppose that $S$ is a set, playing the role of a universal set for a particular mathematical model. It is sometimes impossible to include all subsets of $S$ in our model, particularly when $S$ is uncountable. In a sense, the more sets that we include, the harder it is to have consistent theories. However, we almost always want the collection of admissible subsets to be closed under the basic set operations. This leads to some important definitions.
Suppose that $S$ is a collection of subsets of $S$. Then $S$ is said to be an algebra (or field) if
Suppose that $S$ is an algebra of subsets of $S$. Show that $\emptyset \in S$.
Suppose that $S$ is an algebra of subsets of $S$ and that $A_{i}\in S$ for each $i$ in a finite index set $I$. Show that
Thus it follows that an algebra of sets is closed under complements and under finite unions and intersections. However in many mathematical theories, probability in particular, this is not sufficient; we often need the collection of admissible subsets to be closed under countable unions and intersections.
Suppose that $S$ is a collection of subsets of $S$. Then $S$ is said to be a σ-algebra (or σ-field) if
Clearly a σ-algebra of subsets is also an algebra of subsets, so the basic results for algebras still hold.
Show that if $A_{i}\in S$ for each $i$ in a countable index set $I$, then $i\in I\cap A_{i}\in S$. Hint: Use DeMorgan's law.
Thus a σ-algebra of subsets of $S$ is closed under countable unions and intersections. This is the reason for the symbol σ in the name.
Suppose that $S$ is a set and that $S$ is a finite algebra of subsets of $S$. Show that $S$ is also a σ-algebra. Hint: any countable union of sets in $S$ reduces to a finite union.
However, there are algebras that are not σ-algebras:
Show that the collection of co-finite subsets defined below is an algebra of subsets of $\mathbb{N}$, but not a σ-algebra:
$$F=\{A\subseteq \mathbb{N}\colon A\text{\hspace{0.5em}is finite}\lor (A)\text{\hspace{0.5em}is finite}\}$$Recall that $(S)$ denotes the collection of all subsets of $S$, called the power set of $S$. Trivially, $(S)$ is a the largest σ-algebra of $S$, and as noted above, is sometimes too large to be useful. At the other extreme, the smallest σ-algebra of $S$ is given in the following exercise.
Show that $\{\emptyset , S\}$ is a σ-algebra.
In many cases, we want to construct a σ-algebra that contains certain basic sets. The following exercises show how to do this.
Suppose that $S_{i}$ is a σ-algebras of subsets of $S$ for each $i$ in a nonempty index set $I$. Show that the intersection of the σ-algebras is also a σ-algebra of subsets of $S$:
$$i\in I\cap S_{i}$$Suppose now that $B$ is a collection of subsets of $S$. Think of the sets in $B$ as basic sets; but in general $B$ will not be a σ-algebra. The σ-algebra generated by $B$ is the intersection of all σ-algebras that contain $B$, which by the previous exercise really is a σ-algebra:
$$\sigma (B)=\{S\colon S\text{\hspace{0.5em}is a \sigma -algebra of subsets of \hspace{0.5em}}S\land B\subseteq S\}$$Show that $\sigma (B)$ is the smallest σ algebra containing $B$
Suppose that $A$ is a subset of $S$. Show that
$$\sigma (\{A\})=\{\emptyset , A, (A), S\}$$Suppose that $A$ and $B$ are subsets of $S$. List the 16 (in general distinct) sets in $\sigma (\{A, B\})$. Open the Venn diagram applet and check your answer.
Suppose that $\{A_{1}, A_{2}, , A_{n}\}$ is a collection of $n$ subsets of $S$. Show that there are $2^{2^{n}}$ (in general distinct) sets in the σ-algebra generated by the given collection.
Suppose that $S$ is a set with σ-algebra $S$, and that $R\subseteq S$. Show that $R=\{A\cap R\colon A\in S\}$ is a σ-algebra of subsets of $R$. When $R\in S$ (which is usually the case), note that $R=\{B\in S\colon B\subseteq R\}$. $R$ is the σ-algebra on $R$ induced by $S$.
In this subsection, we will discuss some natural σ-algebras that are used for special types of sets. First, if $S$ is countable, we use the power set $(S)$ as the σ-algebra. Thus, all sets are admissible. For $\mathbb{R}$, the set of real numbers, we use the σ-algebra generated by the collection of all intervals. This is sometimes called the Borel σ-algebra, named after Emil Borel.
Suppose that $\left(\begin{array}{c}S_{1}\\ S_{2}\\ \\ S_{n}\end{array}\right)$ is a sequence of sets and that $S_{i}$ is a σ-algebra of subsets of $S_{i}$ for each $i\in \{1, 2, , n\}$. For the product set
$$S=S_{1}\times S_{2}\times \times S_{n}$$we use the σ-algebra generated by the collection of all product sets:
$$S=\sigma (\{A_{1}\times A_{2}\times \times A_{n}\colon \forall i\in \{1, 2, , n\}\colon A_{i}\in S_{i}\})$$We extend this idea to an infinite product. Thus, suppose that $\left(\begin{array}{c}S_{1}\\ S_{2}\\ \end{array}\right)$ is an infinite sequence of sets and that $S_{i}$ is a σ-algebra of subsets of $S_{i}$ for each $i\in $. For the product set
$$S=S_{1}\times S_{2}\times $$we use the σ-algebra generated by the collection of all cylinder sets:
$$S=\sigma (\{A_{1}\times A_{2}\times \times A_{n}\times S_{n+1}\times S_{n+2}\times \colon n\in \land \forall i\in \{1, 2, , n\}\colon A_{i}\in S_{i}\})$$Combining the product construction with our earlier remarks about $\mathbb{R}$, note that for $\mathbb{R}^{n}$, we use the σ-algebra generated by the collection of all products of intervals. This is the Borel σ-algebra for $\mathbb{R}^{n}$.
Recall that a set usually comes with a σ-algebra of admissible subsets. Thus, suppose that $S$ and $T$ are sets with σ-algebras $S$ and $T$, respectively. If $(f, S, T)$, then a natural requirement is that the inverse image of any admissible subset of $T$ be an admissible subset of $S$. Formally $f$ is said to be measurable if
$$\forall B\in T\colon f^{(-1)}(B)\in S$$Suppose that $R$, $S$, $T$ are sets with σ-algebras $R$, $S$, and $T$, respectively. Show that if $(f, R, S)$ is measurable and $(g, S, T)$ is measurable, then $(g\circ f, R, T)$ is measurable.
Suppose that $(f, S, T)$, and that $T$ is a σ-algebra of subsets of $T$. Show that the collection
$$\sigma (f)=\{f^{(-1)}(B)\colon B\in T\}$$is a σ-algebra of subsets of $S$, called the σ-algebra generated by $f$. Hint: Recall that the inverse image preserves all set operations.
The σ-algebra generated by $f$ is the smallest σ-algebra on $S$ that makes $f$ measurable (relative to the given σ-algebra on $T$). More generally, suppose that $T_{i}$ is a set with σ algebra $T_{i}$ for each $i$ in a nonempty index set $I$, and that $(f_{i}, S, T_{i})$ for each $i\in I$. The σ-algebra generated by this collection of functions is
$$\sigma (\{f_{i}\colon i\in I\})=\sigma (\{f_{j}^{(-1)}(B)\colon i\in I\land B\in T_{i}\})$$Again, this is the smallest σ-algebra on $S$ that makes $f_{i}$ measurable for each $i\in I$.
Most of the sets encountered in applied probability are either countable, or subsets of $\mathbb{R}^{n}$ for some $n$, or more generally, subsets of a product of a countable number of sets of these types. In this subsection, we will explore some of theses special cases.
Suppose that $S$ is countable and is given the σ-algebra $(S)$. Show that any function on $S$ is measurable.
Recall that the set of real numbers $\mathbb{R}$ is given the σ-algebra generated by the collection of intervals, the Borel σ-algebra. All of the real-valued elementary functions are measurable. The elementary functions include algebraic functions (which in turn include the polynomial and rational functions), the usual transcendental functions (exponential, logarithm, trigonometric), and the usual functions constructed from these.
Suppose that $\left(\begin{array}{c}S_{1}\\ S_{2}\\ \\ S_{n}\end{array}\right)$ is a sequence of sets and that $S_{i}$ is a σ-algebra of subsets of $S_{i}$ for each $i\in \{1, 2, , n\}$. Recall that for the product set $S_{1}\times S_{2}\times \times S_{n}$ we use the σ-algebra $S$ generated by the collection of all product sets of the form $A_{1}\times A_{2}\times \times A_{n}$ where $A_{i}\in S_{i}$ for each $i$.
If $f$ is a function from $S$ into $T_{1}\times T_{2}\times \times T_{n}$, then $f=\left(\begin{array}{c}f_{1}\\ f_{2}\\ \\ f_{n}\end{array}\right)$, where $f_{i}$ is the $(i)$ coordinate function, mapping $S$ into $T_{i}$. As we might expect, $f$ is measurable if and only if $f_{i}$ is measurable for each $i$.
We won't be overly pedantic about measure-theoretic assumptions in this project. Unless we say otherwise, we assume that all sets that appear are measurable (that is, members of the appropriate σ-algebras), and that all functions are measurable (relative to the appropriate σ-algebras).