Measure Theory

In this section we discuss some topics from measure theory that are a bit more advanced than the topics in the previous sections of this chapter. However, measure-theoretic ideas are essential for a deep understanding of probability, since probability is itself a measure. In particular, σ-algebras play a fundamental role, even for applied probability, in encoding the state of information about a random experiment.

Algebras and σ-algebras of sets

Suppose that

S

is a set, playing the role of a universal set for a particular mathematical model. It is sometimes impossible to include all subsets of

S

in our model, particularly when

S

is uncountable. In a sense, the more sets that we include, the harder it is to have consistent theories. However, we almost always want the collection of admissible subsets to be closed under the basic set operations. This leads to some important definitions.

Algebras of Sets

Suppose that

S

is a collection of subsets of

S

. Then

S

is said to be an algebra (or field) if

Suppose that $S$ is an algebra of subsets of $S$ . Show that $S$ .

Suppose that $S$ is an algebra of subsets of $S$ and that $A i S$ for each $i$ in a finite index set $I$ . Show that

$i I A i S$ . Hint: Use induction on the number of elements in $I$ .
$i I A i S$ . Hint: Use part (a) and DeMorgan's law.

Thus it follows that an algebra of sets is closed under complements and under finite unions and intersections. However in many mathematical theories, probability in particular, this is not sufficient; we often need the collection of admissible subsets to be closed under countable unions and intersections.

σ-Algebras of Sets

Suppose that

S

is a collection of subsets of

S

. Then

S

is said to be a σ-algebra (or σ-field) if

Clearly a σ-algebra of subsets is also an algebra of subsets, so the basic results for algebras still hold.

Show that if $A i S$ for each $i$ in a countable index set $I$ , then $i I A i S$ . Hint: Use DeMorgan's law.

Thus a σ-algebra of subsets of

S

is closed under countable unions and intersections. This is the reason for the symbol σ in the name.

Suppose that $S$ is a set and that $S$ is a finite algebra of subsets of $S$ . Show that $S$ is also a σ-algebra. Hint: any countable union of sets in $S$ reduces to a finite union.

Show that the collection of co-finite subsets defined below is an algebra of subsets of , but not a σ-algebra:

F A A is finite A is finite

General Constructions

Recall that

S

denotes the collection of all subsets of

S

, called the power set of

S

. Trivially,

S

is a the largest σ-algebra of

S

, and as noted above, is sometimes too large to be useful. At the other extreme, the smallest σ-algebra of

S

is given in the following exercise.

Show that $S$ is a σ-algebra.

In many cases, we want to construct a σ-algebra that contains certain basic sets. The following exercises show how to do this.

Suppose that $S i$ is a σ-algebras of subsets of $S$ for each $i$ in a nonempty index set $I$ . Show that the intersection of the σ-algebras is also a σ-algebra of subsets of $S$ :

i I S i

Suppose now that

B

is a collection of subsets of

S

. Think of the sets in

B

as basic sets; but in general

B

will not be a σ-algebra. The σ-algebra generated by

B

is the intersection of all σ-algebras that contain

B

, which by the previous exercise really is a σ-algebra:

Show that $σ B$ is the smallest σ algebra containing $B$

$B σ B$
If $S$ is a σ-algebra of subsets of $S$ and $B S$ then $σ B S$ .

Suppose that $A$ is a subset of $S$ . Show that

σ A A A S

Suppose that $A$ and $B$ are subsets of $S$ . List the 16 (in general distinct) sets in $σ A B$ . Open the Venn diagram applet and check your answer.

Suppose that $A 1 A 2 A n$ is a collection of $n$ subsets of $S$ . Show that there are $2 2 n$ (in general distinct) sets in the σ-algebra generated by the given collection.

Argue that there are $2 n$ sets of the form $B 1 B 2 B n$ where for each $i$ , either $B i A i$ or $B i A i$
Argue that every set in the σ-algebra is a union of some (perhaps none, perhaps all) of the sets in (a).

Suppose that $S$ is a set with σ-algebra $S$ , and that $R S$ . Show that $R A R A S$ is a σ-algebra of subsets of $R$ . When $R S$ (which is usually the case), note that $R B S B R$ . $R$ is the σ-algebra on $R$ induced by $S$ .

Special Cases

In this subsection, we will discuss some natural σ-algebras that are used for special types of sets. First, if

S

is countable, we use the power set

S

as the σ-algebra. Thus, all sets are admissible. For

, the set of real numbers, we use the σ-algebra generated by the collection of all intervals. This is sometimes called the Borel σ-algebra, named after Emil Borel.

Suppose that

S 1 S 2 S n

is a sequence of sets and that

S i

is a σ-algebra of subsets of

S i

for each

i 12 n

. For the product set

We extend this idea to an infinite product. Thus, suppose that

S 1 S 2

is an infinite sequence of sets and that

S i

is a σ-algebra of subsets of

S i

for each

i

. For the product set

Combining the product construction with our earlier remarks about

, note that for

n

, we use the σ-algebra generated by the collection of all products of intervals. This is the Borel σ-algebra for

n

Measurable Functions

Recall that a set usually comes with a σ-algebra of admissible subsets. Thus, suppose that

S

and

T

are sets with σ-algebras

S

and

T

, respectively. If

f S T

, then a natural requirement is that the inverse image of any admissible subset of

T

be an admissible subset of

S

. Formally

f

is said to be measurable if

Suppose that $R$ , $S$ , $T$ are sets with σ-algebras $R$ , $S$ , and $T$ , respectively. Show that if $f R S$ is measurable and $g S T$ is measurable, then $g f R T$ is measurable.

Suppose that $f S T$ , and that $T$ is a σ-algebra of subsets of $T$ . Show that the collection

σ f f B B T

is a σ-algebra of subsets of $S$ , called the σ-algebra generated by $f$ . Hint: Recall that the inverse image preserves all set operations.

The σ-algebra generated by

f

is the smallest σ-algebra on

S

that makes

f

measurable (relative to the given σ-algebra on

T

). More generally, suppose that

T i

is a set with σ algebra

T i

for each

i

in a nonempty index set

I

, and that

f i S T i

for each

i I

. The σ-algebra generated by this collection of functions is

Again, this is the smallest σ-algebra on

S

that makes

f i

measurable for each

i I

Special Cases

Most of the sets encountered in applied probability are either countable, or subsets of

n

for some

n

, or more generally, subsets of a product of a countable number of sets of these types. In this subsection, we will explore some of theses special cases.

Suppose that $S$ is countable and is given the σ-algebra $S$ . Show that any function on $S$ is measurable.

Recall that the set of real numbers

is given the σ-algebra generated by the collection of intervals, the Borel σ-algebra. All of the real-valued elementary functions are measurable. The elementary functions include algebraic functions (which in turn include the polynomial and rational functions), the usual transcendental functions (exponential, logarithm, trigonometric), and the usual functions constructed from these.

Suppose that

S 1 S 2 S n

is a sequence of sets and that

S i

is a σ-algebra of subsets of

S i

for each

i 12 n

. Recall that for the product set

S 1 S 2 S n

we use the σ-algebra

S

generated by the collection of all product sets of the form

A 1 A 2 A n

where

A i S i

for each

i

f

is a function from

S

into

T 1 T 2 T n

, then

f f 1 f 2 f n

, where

f i

is the

i

coordinate function, mapping

S

into

T i

. As we might expect,

f

is measurable if and only if

f i

is measurable for each

i

General Assumption

We won't be overly pedantic about measure-theoretic assumptions in this project. Unless we say otherwise, we assume that all sets that appear are measurable (that is, members of the appropriate σ-algebras), and that all functions are measurable (relative to the appropriate σ-algebras).

9. Measure Theory

Algebras and σ-algebras of sets

Algebras of Sets

σ-Algebras of Sets

General Constructions

Special Cases

Measurable Functions

Special Cases

General Assumption