Markov property / no after-effect - Case Studies definitions

Markov properties / no aftereffect

Foundation Markov Decision Process (Markov Decision Process), dynamic programming (Dynamic Programming) is the Markov property / no after-effect. In short, * future has nothing to do with the past, and now only about *
that is:
$P({X_{n+1}|X_0, . . . , X_n}) = P({X_{n+1}|X_n} )$

What is the "after-effect"

Look at a few of Markovin (Not markov property example) case:

• Case 1: You can not take the repeated route
  https://blog.csdn.net/qq_30137611/article/details/77655707
  this case "can not take the repeated line," repeated route status information is not recorded in the current state, determine the future not only related to the (next starting point) of the current, past, and also relevant.
  Can not take the repeated route -> after-effect

• Case 2: Take out yesterday what color?
  https://math.stackexchange.com/questions/89394/example-of-a-stochastic-process-which-does-not-have-the-markov-property
  Briefly, a box, there are two black balls , a white ball. Yesterday took out a (do not know the color), today took out a (black ball), remove the color of the ball tomorrow, it depends not only on today, but also on yesterday.

• Case 3:
  https://www.quora.com/What-are-examples-of-non-markovian-processes
  (to be supplemented)

Look at the definition

强化学习第二版Reinforced Learning，second edition：
The state must include information about all aspects of the past agent–environment interaction that make a diference for the future. If it does, then the state is said to have the Markov property.

Probability theory and mathematical statistics Zhejiang Fourth Edition:
Markov or no after-effect: the process or the system at the moment $t_0$State which is known in the case where, at time $t>t_0$The conditional distribution of the process in which the state at time $t_0$In which the state has nothing to do before. In layman's terms, that is, the future does not depend on the past.

Interesting interpretation:
http://blog.sciencenet.cn/blog-350729-665509.html