Introduction

An Example

Consider the mortgage crisis of recent years. The housing market can be thought of in terms of either houses that find owners or owners that find houses.

The basics of Markov models.
Other examples. Disease.

A Simple Version of the "Mortgage Game"
Based on a casual look at the data on mortgage defaults, let's estimate the likelihood that a homeowner misses a payment at 15%. Once an account is late there's an equal chance it will move into default or back to normal, "OK" status. Further, half the mortgages in default result in foreclosure and half return to "OK" via some sort of workout. Finally, we assume that the market has stalled so all foreclosed houses remain in that condition forever.
"States"
Mutually exclusive and exhaustive.
Further Assumption
Path independence.

Take a closer look at the Foreclosed state. Do we notice anything different about it compared to the others? There's no way out. Based on our assumptions, a house that enters this state will remain in this state forever. A house can move back and forth between OK, LATE, and DEFAULT for a long time, but once it hits foreclosure, that's it. We call this an "absorbing state."
Back to our game. Before we can play, we need to talk about how to use a random number table.

How to Read a Random Number Chart

A random number chart is just a table of rows and columns of random numbers. We "use" the chart to generate a sequence of random numbers. We begin by picking an arbitrary starting point and then selecting numbers down a column or across the row from there.
Process. Decide whether you'll move vertically or horizontally and up or down. Then close your eyes and pick a starting number. After this, we'll move one number at a time in the directions you chose. Each new number will be like pulling a number out of a hat or throwing dice. We'll decide what happens in the game based on what number we get.

Unsupported math environment "center"

Unsupported math environment "center"

0.673	0.272	0.697	0.963	0.378		0.006	0.620	0.831	0.023	0.613
0.307	0.082	0.608	0.224	0.882		0.059	0.501	0.780	0.023	0.154
0.412	0.615	0.999	0.387	0.171		0.646	0.751	0.550	0.328	0.599
0.233	0.008	0.207	0.259	0.810		0.215	0.248	0.753	0.956	0.350
0.181	0.858	0.210	0.155	0.159		0.854	0.644	0.953	0.660	0.803
0.416	0.555	0.778	0.557	0.186		0.969	0.416	0.796	0.410	0.389
0.437	0.310	0.847	0.214	0.733		0.392	0.897	0.018	0.464	0.274
0.306	0.769	0.532	0.487	0.970		0.245	0.697	0.542	0.900	0.650
0.648	0.069	0.643	0.787	0.109		0.001	0.693	0.620	0.625	0.777
0.372	0.466	0.094	0.667	0.574		0.563	0.837	0.118	0.840	0.628

asdfsdfasdfasd

State	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10	T11	T12	T13	T14	T15
OK	1000	850	798	779	754	729	706	683	661	639	618	598	579	560	542
Late	0	150	128	120	117	113	109	106	102	99	96	93	90	87	84
Def	0	0	75	64	60	58	57	55	53	51	50	48	46	45	43
Fore	0	0	0	38	69	99	128	157	184	211	236	261	285	308	331

Unsupported math environment "tabular"

Unsupported math environment "tabular"

Sometimes we are interested in a set of conditions that an entity can be in. A study of romance and family formation might note that people can be single, involved, or married and that people can move from one of these "states" to another. Or, we might think about the health of a population and note that people can be uninfected, currently infected, or immune. Other examples:
Drug Use: non-user, casual user, addict, in-recovery

Consider the following scenario. A house that is lived in by its owner is called an owner-occupied housing unit (OOHU). In general, owners have a mortgage – a bank has given them a loan to pay off the previous owner and each month they pay back a portion of what they owe the bank. When all is well, an OOHU stays an OHUU from one month to the next, but in these times people lose their jobs and mortgage terms change and sometimes people cannot afford to make their mortgage payments. If several payments are missed, a house "goes into default." When it's in default the homeowner can resume payments and come to terms with the bank or the bank can foreclose and the house is said to be "in foreclosure." When the foreclosure process is complete the lender owns the bank and it becomes a REO (real estate owned) property. Either while in foreclosure or when it's a REO, a house can be sold.
We will keep things simple: a house can be an OOHU, "in trouble," or REO.
Basic Definitions
Assumptions/Requirements
Chains vs. Processes
Chains. Regular, absorbing, cyclical
"State"
Absorbing states, Transient states,
Eigen vector.
Stop and Think Questions
1. A cyclical chain is a set of states among which you can cycle but in which you are stuck once you enter one of them. Show how a chronic, incurable, but intermittent disease fits this description. Show how the popular image of the rat-race fits this description.
2. The book says (104-5) that for a regular chain (one with no absorbing state) there is some n for which the multiperiod transition matrix Pn has no zeros. Why does this make sense as a criteria for a chain to be non-absorbing?

Criteria
Mutually exclusive and exhaustive set of states
Path independence
Questions We Can Ask and Answer
Is there an equilibrium distribution among states? If there's an absorbing state, the answer is trivial (e.g., "in the long run, we're all dead"). If not, then the limit of the multi-period transition matrix – at equilibrium, each state will contribute proportionally to the population of all the other states.
How long does it take to get to the equilibrium from a given starting configuration?
What is the average length of a spell in any given state?
Distinctions and Definitions
Markov Chain = individual transitions among states over time
Markov Process = population distribution among states over time
Transient state = a state that can be entered and exited
Absorbing state = a state that can be entered but not exited
Regular chain = all states "communicate" with one another – no absorbing states
Tips and Tricks