In this blog, I try to introduce the concept of Markov chain , so first, we give the definition of markov chain. And we state that every markov chain will have a fixed point.
Then we introduce the random walk and how to connect the concept of Markov Chain and the random Walk. And we proof some basic facts of the expection values.
How to leave the maze without memorize anything using random walk.
附上 wiki 的 Markov Chain inequality:
If X is a nonnegative random variable and a > 0, then the probability that X is at least a is at most the expectation of X divided by a:
From the definition of expectation:

However, X is a non-negative random variable thus,

From this we can derive,

From here it is easy to see that

Chebyshev's inequality
這個東西也重要

for any
a > 0. Here
Var(X) is the
variance of X, defined as:
![\operatorname{Var}(X) = \operatorname{E}[(X - \operatorname{E}(X) )^2].](https://wikimedia.org/api/rest_v1/media/math/render/svg/71c7a116967cab98cb1eb56e626497e77ce354a2)
Chebyshev's inequality follows from Markov's inequality by considering the random variable

and the constant

for which Markov's inequality reads

This argument can be summarized (where "MI" indicates use of Markov's inequality):
