Stochastic process -- Markov chain

Markov The process

The present state is known , The future state has nothing to do with the past state .
$$P(X(t_{n+1})\le x_{n+1}|X(t_1)=x_1,...,X(t_n)=x_n)=P(X(t_{n+1})\le x_{n+1}|X(t_n)=x_n)$$

State transition matrix

If the state transition probability is independent of a certain state , Call this process homogeneous Markov The process .
$$p_{ij}=P(X_{n+k}=j|X_n=i)$$
i It's the current state ,j Is the next status

In matrix form
$$P(k)=\left[\begin{array}{cccc}{p}_{11}(k)& p_{12}(k)& ...& p_{1n}(k)\\ ...& ...& ...& ...\\ ...& ...& ...& ...\\ p_{n1}(k)& ...& ...& p_{nn}(k)\end{array}\right]$$

C-K equation

$$p_{ij}=\sum _r{p}_{ir}(k)p_{rj}(l)$$
In the form of a matrix
$$P(k+l)=P(k)P(l)=[P(1){]}^{k+l}$$

n The transition probability of each step can be determined by 1 Step transition probability acquisition