Aimathic
Login | English | Deutsch

Precalculus math worksheets

Build your own math worksheets from 35,000+ problems for grades 3 to 12, from fractions to AP Calculus. Every problem comes with step-by-step solutions.

Transition matrices

Learning goals

  • Represents state changes with transition matrices and state vectors.
  • Computes successive states by matrix multiplication.
  • Interprets matrix entries as transition proportions or counts.
  • Analyzes long-run or steady-state behavior in simple models.

Click problems to add them to your worksheet.

Transition matrices 1
A two-state process uses the row-stochastic transition matrix \(T=\begin{pmatrix}0.80&0.20\\0.35&0.65\end{pmatrix}\). Rows represent the current state \(A,B\), and columns represent the next state \(A,B\). If the process is currently in state \(B\), what is the probability that it moves to state \(A\) next?

Hints

- First identify which dimension of the matrix represents the current state. - Use the row for the current state \(B\). - Then use the column for the next state \(A\).

Solution

Use the entry in row \(B\), column \(A\). That entry is \(0.35\).

Answer

\(0.35\)
Transition matrices 2
Determine whether \(P=\begin{pmatrix}0.60&0.40\\0.25&0.75\end{pmatrix}\) is a row-stochastic transition matrix.

Hints

- Recall the two requirements for a row-stochastic matrix. - Check the sign of every entry. - Check each row sum separately.

Solution

Every entry is nonnegative. The first row sums to \(0.60+0.40=1\), and the second row sums to \(0.25+0.75=1\). Therefore, \(P\) is row-stochastic.

Answer

Yes. \(P\) is row-stochastic.
Transition matrices 3
A two-state process has states \(A\) and \(B\), in that order, and row-stochastic transition matrix \(T=\begin{pmatrix}1&0\\0.30&0.70\end{pmatrix}\). Which state is absorbing?

Hints

- Look for a row with a \(1\) in its own-state column and \(0\) elsewhere. - That row represents a state that cannot be left.

Solution

1. A state is absorbing if, once entered, the process stays there with probability \(1\). 2. The first row is \((1,0)\), so a process in state \(A\) remains in \(A\) with probability \(1\). 3. Therefore, state \(A\) is absorbing.

Answer

State \(A\).
Transition matrices 4
A two-state process has states \(A\) and \(B\), in that order. It uses the row-stochastic transition matrix \(T=\begin{pmatrix}0.80&0.20\\0.40&0.60\end{pmatrix}\), where rows give the current state and columns give the next state. The current probability distribution is \(\mathbf{p}_0=\begin{pmatrix}0.75&0.25\end{pmatrix}\). Find the distribution after one step.

Hints

- Under the row-stochastic convention, multiply the row state vector on the left by the transition matrix. - The resulting probabilities should add to \(1\).

Solution

1. For the row-stochastic convention, compute \(\mathbf{p}_1=\mathbf{p}_0T\). 2. Thus, \(\mathbf{p}_1=\begin{pmatrix}0.75&0.25\end{pmatrix}\begin{pmatrix}0.80&0.20\\0.40&0.60\end{pmatrix}=\begin{pmatrix}0.70&0.30\end{pmatrix}\).

Answer

\(\mathbf{p}_1=\begin{pmatrix}0.70&0.30\end{pmatrix}\).
Transition matrices 5
A proposed row-stochastic transition matrix is \(T=\begin{pmatrix}0.7&0.3\\0.6&0.5\end{pmatrix}\). Is it valid? Explain.

Hints

- A valid transition matrix must satisfy more than a restriction on its individual entries. - Check the total probability of leaving each current state.

Solution

1. The entries are all between \(0\) and \(1\), but that is not enough. 2. The second row sums to \(0.6+0.5=1.1\), not \(1\). Thus it is not a valid row-stochastic matrix.

Answer

No. The second row sums to \(1.1\), so the matrix is not row-stochastic.
Transition matrices 6
In a two-state process, rows of \(T=\begin{pmatrix}0.75&0.25\\0.45&0.55\end{pmatrix}\) represent current states \(A,B\), and columns represent next states \(A,B\). Are the probabilities of \(A\) going to \(B\) and \(B\) going to \(A\) equal? State both.

Hints

- Each transition has a starting state and an ending state. - Reversing the transition reverses which row and column you read.

Solution

1. From \(A\) to \(B\) uses the first row and second column, so the probability is \(0.25\). 2. From \(B\) to \(A\) uses the second row and first column, so the probability is \(0.45\). 3. These transition probabilities are not equal.

Answer

No. \(P(A\to B)=0.25\) and \(P(B\to A)=0.45\).
Transition matrices 7
A two-state process has states \(A\) and \(B\). In one step, \(70\%\) of those in \(A\) stay in \(A\), and \(20\%\) of those in \(B\) move to \(A\). Write its row-stochastic transition matrix. Use rows for the current state and columns for the next state, both ordered \(A,B\).

Hints

- Each row gives the two possible destinations from one starting state. - Probabilities in a row must add to \(1\).

Solution

1. From \(A\), the probabilities of next states \(A,B\) are \(0.7\) and \(0.3\). 2. From \(B\), the probabilities are \(0.2\) and \(0.8\). 3. Therefore \(T=\begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}\).

Answer

\(T=\begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}\)
Transition matrices 8
A two-state model has initial row distribution \(\mathbf{p}_0=\begin{pmatrix}1&0\end{pmatrix}\) for states \(A\) and \(B\), in that order, and transition matrix \(T=\begin{pmatrix}0.6&0.4\\0.1&0.9\end{pmatrix}\). Find \(\mathbf{p}_1=\mathbf{p}_0T\).

Hints

- Which state has all the initial probability? - Which row of \(T\) gives the next-state probabilities for people starting in \(A\)?

Solution

1. The starting row vector selects the first row of \(T\). 2. Therefore \(\mathbf{p}_1=\begin{pmatrix}0.6&0.4\end{pmatrix}\).

Answer

\(\mathbf{p}_1=\begin{pmatrix}0.6&0.4\end{pmatrix}\)
Transition matrices 9
A dynamic system has states \(S_1, S_2, S_3, S_4\). The entry \(m_{ij}\) of its transition matrix \(M\) is the proportion moving from state \(S_j\) to state \(S_i\). Construct the \(4\times4\) matrix using these entries: \(m_{11}=0.75\), \(m_{21}=0.15\), \(m_{41}=0.10\) \(m_{12}=0.20\), \(m_{22}=0.80\) \(m_{33}=0.60\), \(m_{43}=0.40\) \(m_{14}=0.05\), \(m_{34}=0.05\), \(m_{44}=0.90\) All entries not listed are \(0\).

Hints

- Use the first index for the row and the second index for the column. - Fill unspecified entries with zero. - Check each column sum after constructing the matrix.

Solution

1. There are four states, so \(M\) has four rows and four columns. 2. The first index determines the row, and the second determines the column. 3. Place the given entries in their indexed positions and fill the remaining positions with zeros: \(M=\begin{pmatrix}0.75&0.20&0&0.05\\0.15&0.80&0&0\\0&0&0.60&0.05\\0.10&0&0.40&0.90\end{pmatrix}\). 4. Each column sums to \(1\), which is consistent with the stated transition convention.

Answer

\(M=\begin{pmatrix}0.75&0.20&0&0.05\\0.15&0.80&0&0\\0&0&0.60&0.05\\0.10&0&0.40&0.90\end{pmatrix}\)
Transition matrices 10
A two-state weather model has states sunny \(S\) and rainy \(R\), with transition matrix \(T=\begin{pmatrix}0.70&0.30\\0.40&0.60\end{pmatrix}\), where rows give today’s state and columns give tomorrow’s state. Given that today is sunny, use \(T^2\) to find the probability that the weather is rainy two days from now.

Hints

- Square the one-step transition matrix to obtain two-step probabilities. - Use the sunny row and rainy column for the requested entry. - The matrix product adds the probabilities of the two possible intermediate-state paths.

Solution

1. The two-step transition matrix is \(T^2\). 2. Its sunny-to-rainy entry is \(0.70\cdot0.30+0.30\cdot0.60=0.21+0.18=0.39\). 3. Therefore the probability of rain two days from now, given sunshine today, is \(0.39\).

Answer

The sunny-to-rainy entry of \(T^2\) is \(0.39\), so the probability is \(0.39\).
Transition matrices 11
The matrix \(T=\begin{pmatrix}0.50&0.30&0.20\\0.10&0.60&0.30\\0.25&x&0.50\end{pmatrix}\) is row-stochastic. Find \(x\).

Hints

- Use the defining row-sum property of a row-stochastic matrix. - Only one row contains the unknown entry. - Set that row's total equal to \(1\).

Solution

The entries in the third row must sum to \(1\): \(0.25+x+0.50=1\). Therefore, \(x=0.25\).

Answer

\(x=0.25\)
Transition matrices 12
A service has \(60\) customers in plan \(A\) and \(40\) customers in plan \(B\). Its monthly row-stochastic transition matrix is \(T=\begin{pmatrix}0.70&0.30\\0.20&0.80\end{pmatrix}\), where rows give the current plan and columns give the next plan. Find the customer distribution after one month.

Hints

- Match the state-vector orientation to the row-stochastic convention. - Each next-state total combines contributions from both current states. - Check that the total number of customers is preserved.

Solution

Use the current row vector \(\begin{pmatrix}60&40\end{pmatrix}\): \(\begin{pmatrix}60&40\end{pmatrix}T =\begin{pmatrix}60\cdot0.70+40\cdot0.20&60\cdot0.30+40\cdot0.80\end{pmatrix} =\begin{pmatrix}50&50\end{pmatrix}\).

Answer

\(50\) customers in plan \(A\) and \(50\) customers in plan \(B\)
Transition matrices 13
A customer can be in one of three states each month: Online \(O\), Store \(S\), or Inactive \(I\). Use rows for the current state and columns for the next state, in the order \(O,S,I\). - From \(O\): \(70\%\) stay in \(O\), \(20\%\) move to \(S\), and \(10\%\) move to \(I\). - From \(S\): \(15\%\) move to \(O\), \(75\%\) stay in \(S\), and \(10\%\) move to \(I\). - From \(I\): \(30\%\) move to \(O\), none move directly to \(S\), and \(70\%\) stay in \(I\). Construct the row-stochastic transition matrix.

Hints

- Fix the stated state order before placing any probabilities. - Build one row at a time from the transitions out of that current state. - Check each completed row against the row-stochastic condition.

Solution

Each row records the probabilities from one current state to the three next states. Therefore, \(T=\begin{pmatrix}0.70&0.20&0.10\\0.15&0.75&0.10\\0.30&0&0.70\end{pmatrix}\). Each row sums to \(1\), as required.

Answer

\(\begin{pmatrix}0.70&0.20&0.10\\0.15&0.75&0.10\\0.30&0&0.70\end{pmatrix}\)
Transition matrices 14
The same two-state process can be represented with either convention below. Row-stochastic convention: \(R=\begin{pmatrix}0.8&0.2\\0.4&0.6\end{pmatrix}\), where rows are current states and columns are next states. Column-stochastic convention: \(C=R^T\), where columns are current states and rows are next states. Initially there are \(70\) objects in state \(A\) and \(30\) in state \(B\). a) Use the row-vector update with \(R\). b) Use the column-vector update with \(C\). c) Explain why the two calculations give the same next-state counts.

Hints

- Match the state-vector orientation to the stated transition convention. - In the row-stochastic form, multiply the row state vector on the left. - In the column-stochastic form, multiply the column state vector on the right of the matrix.

Solution

1. With the row-stochastic convention, \(\begin{pmatrix}70&30\end{pmatrix}R=\begin{pmatrix}68&32\end{pmatrix}\). 2. Since \(C=R^T=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}\), the column-stochastic update is \(C\begin{pmatrix}70\\30\end{pmatrix}=\begin{pmatrix}68\\32\end{pmatrix}\). 3. Transposing the transition matrix and switching the state vector from row orientation to column orientation describes the same transition probabilities, so both conventions produce the same physical counts.

Answer

a) \(\begin{pmatrix}68&32\end{pmatrix}\) b) \(\begin{pmatrix}68\\32\end{pmatrix}\) c) The conventions are transposes of one another and use matching state-vector orientations, so they represent the same process.
Transition matrices 15
A row-stochastic two-state model uses \(T=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\). Initially, a fraction \(p\) is in state \(A\) and \(1-p\) is in state \(B\). After one step, \(55\%\) are in state \(A\). Find \(p\).

Hints

- Both starting states can contribute to state \(A\) after one step. - Add the two contributions and match the stated final fraction. - Check the result is a probability between \(0\) and \(1\).

Solution

1. The next-step fraction in \(A\) is \(0.8p+0.3(1-p)\). 2. Set it equal to \(0.55\): \(0.8p+0.3-0.3p=0.55\). 3. Thus \(0.5p=0.25\), giving \(p=0.5\).

Answer

\(p=0.5\), or \(50\%\).
Transition matrices 16
In a two-state process, the row-stochastic transition matrix is \(T=\begin{pmatrix}0.5&0.5\\0.25&0.75\end{pmatrix}\). The starting distribution is \(\mathbf{p}_0=\begin{pmatrix}1&0\end{pmatrix}\). Find the distributions after one and two steps.

Hints

- The new state distribution is found by multiplying the current distribution by \(T\). - For the second step, use the result of the first step as the starting row. - Check that each pair of probabilities totals \(1\).

Solution

1. The one-step distribution is \(\mathbf{p}_1=\mathbf{p}_0T=\begin{pmatrix}0.5&0.5\end{pmatrix}\). 2. Multiply again: \(\mathbf{p}_2=\mathbf{p}_1T\). 3. The next fraction in the first state is \(0.5\cdot0.5+0.5\cdot0.25=0.375\); the second is \(0.625\).

Answer

\(\mathbf{p}_1=\begin{pmatrix}0.5&0.5\end{pmatrix}\), \(\mathbf{p}_2=\begin{pmatrix}0.375&0.625\end{pmatrix}\).
Transition matrices 17
A three-state process has row-stochastic transition matrix \(T=\begin{pmatrix}0.6&0.3&0.1\\0.45&0.4&0.15\\0.3&0.3&0.4\end{pmatrix}\). The states are \(A\), \(B\), and \(C\), in that order. Check by calculating all three entries of \(\pi T\) that \(\pi=\begin{pmatrix}\frac{1}{2}&\frac{1}{3}&\frac{1}{6}\end{pmatrix}\) is stationary. Explain what stationary means here.

Hints

- Use the stated row-stochastic convention to choose the multiplication order. - Find each next-state probability using one column of the transition matrix. - Compare the distribution before and after one step. What would equality mean for the whole group?

Solution

1. The first product entry is \(\frac{1}{2}\cdot0.6+\frac{1}{3}\cdot0.45+\frac{1}{6}\cdot0.3=\frac{1}{2}\). 2. The second entry is \(\frac{1}{2}\cdot0.3+\frac{1}{3}\cdot0.4+\frac{1}{6}\cdot0.3=\frac{1}{3}\). The third is \(\frac{1}{2}\cdot0.1+\frac{1}{3}\cdot0.15+\frac{1}{6}\cdot0.4=\frac{1}{6}\). 3. Thus \(\pi T=\pi\). The overall shares do not change from one step to the next, although individual states can change.

Answer

\(\pi T=\begin{pmatrix}\frac{1}{2}\cdot0.6+\frac{1}{3}\cdot0.45+\frac{1}{6}\cdot0.3&\frac{1}{2}\cdot0.3+\frac{1}{3}\cdot0.4+\frac{1}{6}\cdot0.3&\frac{1}{2}\cdot0.1+\frac{1}{3}\cdot0.15+\frac{1}{6}\cdot0.4\end{pmatrix}=\begin{pmatrix}\frac{1}{2}&\frac{1}{3}&\frac{1}{6}\end{pmatrix}=\pi\). The overall state shares stay the same after a step.
Transition matrices 18
Three competing car-sharing companies, \(A\), \(B\), and \(C\), share a city market. The entry \(m_{ij}\) gives the proportion of customers who move from company \(j\) to company \(i\), including customers who stay with the same company. The weekly transition matrix is \(M=\begin{pmatrix}0.8&0.1&0.05\\0.1&0.7&0.15\\0.1&0.2&0.8\end{pmatrix}\). At week \(0\), the \(20{,}000\) customers are distributed as follows: \(10{,}000\) with \(A\), \(6000\) with \(B\), and \(4000\) with \(C\). 1) Find the customer distribution after one week. 2) Find the customer distribution after two weeks.

Hints

- Write the current customer counts as a column state vector. - Multiply the transition matrix by the current state vector. - Use the week-1 result as the input for week 2. - Check that the total number of customers remains constant.

Solution

1. The initial state vector is \(\mathbf{v}_0=\begin{pmatrix}10{,}000\\6000\\4000\end{pmatrix}\). After one week, \(\mathbf{v}_1=M\mathbf{v}_0\) \(=\begin{pmatrix}0.8\cdot10{,}000+0.1\cdot6000+0.05\cdot4000\\0.1\cdot10{,}000+0.7\cdot6000+0.15\cdot4000\\0.1\cdot10{,}000+0.2\cdot6000+0.8\cdot4000\end{pmatrix}\) \(=\begin{pmatrix}8800\\5800\\5400\end{pmatrix}\). 2. Apply the transition matrix again: \(\mathbf{v}_2=M\mathbf{v}_1\) \(=\begin{pmatrix}0.8\cdot8800+0.1\cdot5800+0.05\cdot5400\\0.1\cdot8800+0.7\cdot5800+0.15\cdot5400\\0.1\cdot8800+0.2\cdot5800+0.8\cdot5400\end{pmatrix}\) \(=\begin{pmatrix}7890\\5750\\6360\end{pmatrix}\).

Answer

1) After one week: \(A: 8800\), \(B: 5800\), \(C: 5400\). 2) After two weeks: \(A: 7890\), \(B: 5750\), \(C: 6360\).
Transition matrices 19
A two-state process has row-stochastic transition matrix \(T=\begin{pmatrix}0.80&0.20\\0.30&0.70\end{pmatrix}\). Find a stationary probability distribution \(\begin{pmatrix}p&q\end{pmatrix}\), where \(p+q=1\), satisfying \(\begin{pmatrix}p&q\end{pmatrix}T=\begin{pmatrix}p&q\end{pmatrix}\).

Hints

- A stationary distribution is unchanged by one transition. - Use the condition that the two probabilities sum to \(1\). - Substitute one probability in terms of the other into one component of the stationary equation.

Solution

From the first component of the stationary equation, \(p=0.80p+0.30q\). Since \(q=1-p\), \(p=0.80p+0.30(1-p)=0.50p+0.30\). Thus, \(0.50p=0.30\), so \(p=0.60\) and \(q=0.40\).

Answer

\(\begin{pmatrix}0.60&0.40\end{pmatrix}\)
Transition matrices 20
A membership program has row-stochastic transition matrix \(T=\begin{pmatrix}0.85&0.15\\0.10&0.90\end{pmatrix}\), where the two states are active and inactive membership. In the long run, let the stationary distribution be \(\begin{pmatrix}p&1-p\end{pmatrix}\). a) Find the stationary distribution. b) If the program has \(5000\) members in all, interpret the stationary distribution as long-run member counts.

Hints

- A stationary distribution is unchanged when the transition matrix is applied. - Use one state probability together with the fact that the two probabilities sum to \(1\). - After finding the probabilities, convert them to counts using the total membership.

Solution

1. A stationary distribution is unchanged by one transition, so \(\begin{pmatrix}p&1-p\end{pmatrix}T=\begin{pmatrix}p&1-p\end{pmatrix}\). 2. Using the active-state component gives \(0.85p+0.10(1-p)=p\). 3. Simplifying gives \(0.75p+0.10=p\), so \(0.25p=0.10\) and \(p=0.40\). 4. Therefore, the stationary distribution is \(\begin{pmatrix}0.40&0.60\end{pmatrix}\). 5. With \(5000\) members, the corresponding long-run counts are \(0.40\cdot5000=2000\) active and \(0.60\cdot5000=3000\) inactive.

Answer

a) \(\begin{pmatrix}0.40&0.60\end{pmatrix}\) b) About \(2000\) active members and \(3000\) inactive members in the long run.
Transition matrices 21
A three-state process has row-stochastic transition matrix \(T=\begin{pmatrix}0.6&0.4&0\\0.2&0.5&0.3\\0&0.2&0.8\end{pmatrix}\). Find a stationary row distribution \(\pi=(a,b,c)\) satisfying \(\pi T=\pi\) and \(a+b+c=1\). Explain why none of the three states is absorbing.

Hints

- In a stationary distribution, the next-step share of each state matches its current share. - Compare the incoming and outgoing probability contributions for the first and third states. - Use the fact that all state shares sum to \(1\). - An absorbing state has probability \(1\) of staying in that state.

Solution

1. Stationarity of the first component gives \(0.6a+0.2b=a\), hence \(b=2a\). 2. Stationarity of the third component gives \(0.3b+0.8c=c\), hence \(c=\frac{3}{2}b=3a\). 3. Normalize: \(a+b+c=6a=1\), so \(\pi=(\frac{1}{6},\frac{1}{3},\frac{1}{2})\). 4. Each state has a positive chance of leaving it, so none is absorbing.

Answer

\(\pi=(\frac{1}{6},\frac{1}{3},\frac{1}{2})\). No state is absorbing because each has a positive probability of changing states.
Transition matrices 22
The school chess club and debate club each track members who attend a weekly meeting and members who miss it, in that order. Their row-stochastic transition matrices are \(T_{\text{chess}}=\begin{pmatrix}0.8&0.2\\0.2&0.8\end{pmatrix}\) and \(T_{\text{debate}}=\begin{pmatrix}0.9&0.1\\0.3&0.7\end{pmatrix}\). Find each club's stationary share of members who attend. Which club has the higher long-run attendance share?

Hints

- In each model, the two stationary shares must sum to \(1\). - Write one balance equation for the share of members who attend. - Solve for the share separately in each model before comparing them.

Solution

1. Let the stationary row distribution be \((p,1-p)\). 2. For the chess club, stationarity gives \(0.8p+0.2(1-p)=p\), so \(p=\frac{1}{2}\). 3. For the debate club, stationarity gives \(0.9p+0.3(1-p)=p\), so \(0.4p=0.3\) and \(p=\frac{3}{4}\). 4. The debate club's stationary attendance share is \(75\%\), compared with \(50\%\) for the chess club.

Answer

Chess club: \(0.8p+0.2(1-p)=p\) gives \(p=50\%\). Debate club: \(0.9p+0.3(1-p)=p\) gives \(p=75\%\). The debate club has the higher stationary attendance share.
Transition matrices 23
A school's theater club tracks whether a room is in use or empty at the end of each hour. The table gives the chance of its status one hour later. <table> <tr><th>Now</th><th>Empty next hour</th><th>In use next hour</th></tr> <tr><td>Empty</td><td>\(0.80\)</td><td>\(0.20\)</td></tr> <tr><td>In use</td><td>\(0.10\)</td><td>\(0.90\)</td></tr> </table> a) Write the transition matrix with rows and columns ordered (empty, in use). b) If the room starts empty, what is the chance it is in use after one hour? c) Suppose the chance the room is in use has settled to a value \(p\) that stays unchanged after another hour. Use your matrix to write an equation for \(p\) and solve it. Check your answer by multiplying the row vector \((1-p,p)\) by your matrix. Why is this value different from part b)?

Hints

- Match each matrix row to the current room status and each column to its next status. - Where is the one-step chance from empty to in use? - What is the in-use entry when you multiply the current state vector by the matrix? - What must be true of that entry when the distribution is unchanged by another step?

Solution

1. In the stated order, the matrix is \(T=\begin{pmatrix}0.80&0.20\\0.10&0.90\end{pmatrix}\). 2. Starting empty, one transition gives an in-use probability of \(0.20\). 3. The in-use entry of \((1-p,p)T\) is \(0.20(1-p)+0.90p\). Setting this equal to \(p\) gives \(0.30p=0.20\), so \(p=\frac{2}{3}\). 4. Check: \((\frac{1}{3},\frac{2}{3})T=(\frac{1}{3},\frac{2}{3})\). The one-hour probability depends on the chosen start; the steady share stays unchanged after a transition.

Answer

a) \(T=\begin{pmatrix}0.80&0.20\\0.10&0.90\end{pmatrix}\). b) \(0.20\). c) \(p=\frac{2}{3}\), because \(p=0.20(1-p)+0.90p\) and \((\frac{1}{3},\frac{2}{3})T=(\frac{1}{3},\frac{2}{3})\). Part b starts from an empty room, whereas the steady-state vector does not change after another hour.
Transition matrices 24
A three-state process has row-stochastic transition matrix \(T=\begin{pmatrix}0.6&0.3&0.1\\0&0.7&0.3\\0&0&1\end{pmatrix}\). It starts in state \(A\), so \(\mathbf{p}_0=\begin{pmatrix}1&0&0\end{pmatrix}\). a) Find \(\mathbf{p}_1\), \(\mathbf{p}_2\), and \(\mathbf{p}_3\). b) Which state is absorbing? c) Explain, without using eigenvalues, why the long-run distribution is \(\begin{pmatrix}0&0&1\end{pmatrix}\).

Hints

- Apply the transition matrix repeatedly to the current row distribution. - An absorbing state has probability \(1\) of staying in itself once reached. - Think about what repeated factors such as \(0.6^n\) and \(0.7^n\) do as the number of steps grows.

Solution

1. \(\mathbf{p}_1=\mathbf{p}_0T=\begin{pmatrix}0.6&0.3&0.1\end{pmatrix}\). 2. \(\mathbf{p}_2=\mathbf{p}_1T=\begin{pmatrix}0.36&0.39&0.25\end{pmatrix}\). 3. \(\mathbf{p}_3=\mathbf{p}_2T=\begin{pmatrix}0.216&0.381&0.403\end{pmatrix}\). 4. State \(C\) is absorbing because a process in \(C\) stays in \(C\) with probability \(1\). 5. The probability of remaining in \(A\) for \(n\) consecutive steps is \(0.6^n\), which approaches \(0\). Once the process reaches \(B\), the probability of remaining there for another \(m\) consecutive steps is \(0.7^m\), which also approaches \(0\). 6. There is no transition out of \(C\). Thus probability continually leaves the nonabsorbing states and accumulates in \(C\), so the long-run distribution is \(\begin{pmatrix}0&0&1\end{pmatrix}\).

Answer

a) \(\mathbf{p}_1=\begin{pmatrix}0.6&0.3&0.1\end{pmatrix}\), \(\mathbf{p}_2=\begin{pmatrix}0.36&0.39&0.25\end{pmatrix}\), \(\mathbf{p}_3=\begin{pmatrix}0.216&0.381&0.403\end{pmatrix}\) b) State \(C\) c) Probability eventually leaves states \(A\) and \(B\), while state \(C\) cannot be left, so the distribution approaches \(\begin{pmatrix}0&0&1\end{pmatrix}\).
Transition matrices 25
A two-state Markov chain uses the row-stochastic transition matrix \(T=\begin{pmatrix}0.75&0.25\\0.25&0.75\end{pmatrix}.\) The initial row state is \(\mathbf p_0=(1,0)\), and \(\mathbf p_{n+1}=\mathbf p_nT\). a) Find \(\mathbf p_1,\mathbf p_2,\mathbf p_3\). b) Let \(d_n=p_{n,A}-p_{n,B}\). Derive a recurrence for \(d_{n+1}\) in terms of \(d_n\), and then give a formula for \(d_n\). c) Use part b to prove the limiting state as \(n\to\infty\). d) Verify directly that the limiting state is stationary.

Hints

- Compute the first states by row-vector multiplication. - Subtract the two component update equations to obtain a recurrence for their difference. - Combine the limiting difference with the fact that the two probabilities always sum to \(1\). - A stationary state is unchanged by multiplication by \(T\).

Solution

1. \(\mathbf p_1=(0.75,0.25)\), \(\mathbf p_2=(0.625,0.375)\), and \(\mathbf p_3=(0.5625,0.4375)\). 2. If \(\mathbf p_n=(a_n,b_n)\), then \(d_n=a_n-b_n\). The update gives \(d_{n+1}=(0.75a_n+0.25b_n)-(0.25a_n+0.75b_n)=0.5(a_n-b_n)=0.5d_n\). 3. Since \(d_0=1\), \(d_n=(0.5)^n\to0\). Also \(a_n+b_n=1\), so \(a_n=\frac{1+d_n}{2}\to\frac12\) and \(b_n=\frac{1-d_n}{2}\to\frac12\). 4. Finally, \((0.5,0.5)T=(0.5,0.5)\), so the limiting state is stationary.

Answer

a) \(\mathbf p_1=(0.75,0.25)\), \(\mathbf p_2=(0.625,0.375)\), \(\mathbf p_3=(0.5625,0.4375)\) b) \(d_{n+1}=0.5d_n\), so \(d_n=(0.5)^n\) c) \(\mathbf p_n\to(0.5,0.5)\) d) \((0.5,0.5)T=(0.5,0.5)\)

All problems may be used, copied and printed free of charge for school and tutoring, including paid tutoring. Commercial adaptations as well as publication or redistribution on the internet are not permitted.

Building free math worksheets