Suppose that $S$ is an interval of integers, either finite or infinite. A birth-death chain on $S$ is a Markov chain $X=\left(\begin{array}{c}X_{0}\\ X_{1}\\ X_{2}\\ \end{array}\right)$ on $S$ with transition probability matrix $P$ of the form.

$$P(x, x-1)=q(x)\text{,\hspace{0.5em}}P(x, x)=r(x)\text{,\hspace{0.5em}}P(x, x+1)=p(x)\text{;\hspace{1em}}x\in S$$where $p$, $q$, and $r$ are nonnegative functions on $S$ with $p(x)+q(x)+r(x)=1$ for $x\in S$. If the interval $S$ has a minimum value $a$ then of course we must have $q(a)=0$. If $r(a)=1$, the boundary point $a$ is said to be absorbing and if $p(a)=1$, then $a$ is said to be reflecting. Similarly, if the interval $S$ has a maximum value $b$ then of course we must have $p(b)=0$. If $r(b)=1$, the boundary point $b$ is said to be absorbing and if $p(b)=1$, then $b$ is said to be reflecting. Several other special models that we have studied are birth-death chains:

Describe each of the following as a birth-death chain.

- The Ehrenfest chain.
- The modified Ehrenfest chain.
- The Bernoulli-Laplace chain
- The simple random walk on $\mathbb{Z}$

If $S$ is finite, classification of the states as recurrent or transient is simple, and depends only on the state graph. In particular, if the chain is irreducible, then the chain is recurrent. In this paragraph, we will study the recurrence and transience of birth-death chains when $S=\mathbb{N}$. We assume that $p(x)> 0$ for all $x\in \mathbb{N}$ and that $q(x)> 0$ for all $x\in $ (but of course we must have $q(0)=0$). Thus, the chain is irreducible. We will use the test for recurrence derived earlier with $A=$, the set of positive states. Essentially, we will compute the probability that the chain never hits 0, starting in a positive state.

Show that the functional equation $P_{A}u=u$ for an unknown function $u$ on $A$ is equivalent to the following system of equations:

$$u(2)-u(1)=\frac{q(1)}{p(1)}u(1)$$ $$u(x+1)-u(x)=\frac{q(x)}{p(x)}(u(x)-u(x-1))\text{,\hspace{1em}}x\in \{2, 3, \}$$Use the result in the previous exercise to show that

$$u(x+1)-u(x)=\frac{q(1)q(x)}{p(1)p(x)}u(1)\text{,\hspace{1em}}x\in $$Use the result in the previous exercise to show that

$$u(x)=\sum_{i=0}^{x-1} \frac{q(1)q(i)}{p(1)p(i)}u(1)$$Now use the test for recurrence to conclude that the chain is recurrent if and only if

$$\sum_{i=0}^{\infty} \frac{q(1)q(i)}{p(1)p(i)}=\infty $$Note that $r$, the function that assigns to each state $x$ the probability of an immediate return to $x$, plays no direct role in whether the chain is transient or recurrent. Indeed all that matters are the ratios $\frac{q(x)}{p(x)}$ for $x\in $

Suppose that $p$ and $q$ (and hence $r$) are constant on $$. Show that the chain is recurrent if $q\ge p$ and transient if $q< p$.

Show that the birth-death chain is positive recurrent if and only if

$$C=\sum_{x=0}^{\infty} \frac{p(0)p(x-1)}{q(1)q(x)}< \infty $$in which case the invariant probability density function $f$ is given by

$$f(x)=\frac{1}{C}\frac{p(0)p(x-1)}{q(1)q(x)}\text{,\hspace{1em}}x\in S$$Note again that $r$, the function that assigns to each state $x$ the probability of an immediate return to $x$, plays no direct role in whether the chain is transient or null recurrent, or positive recurrent.

In the positive recurrent case, show that the birth-death chain is reversible.

Suppose that $p$ is constant on $\mathbb{N}$ and $q$ is constant on $$. Prove the following results:

- The chain is positive recurrent if $q> p$ and the invariant distribution is the geometric distribution on $\mathbb{N}$ with parameter $\frac{p}{q}$: $$f(x)=(1-\frac{p}{q})\left(\frac{p}{q}\right)^{x}\text{,\hspace{1em}}x\in \mathbb{N}$$
- The chain is null recurrent if $q=p$ .
- The chain is transient if $q< p$.