Markov Chains
What is a Markov Chain?
A Markov chain is a stochastic process where the probability of transitioning to a particular state depends solely on the state attained in the previous event. Due to this feature, Markov chains are sometimes called “memory-less” processes. . They are named after the Russian mathematician Andrey Markov.
Figure 1: Markov chain with two states.
Mathematically, a Markov chain can be defined as a sequence of random variables X1, X2, …, Xn where:
Pr(Xn+1 = x|X1 = x1, X2 = x2, … , Xn = xn) = Pr(Xn+1 = x|Xn = xn) and Pr(X1 = x1, X2 = x2, … , Xn = xn) > 0
The condition that states that the probability of event Xx+1 depends only on the previous event is called the Markov property. Markov chains can be discrete or continuous in time.
Why are Markov Chains Important?
Markov chains have a wide range of applications. They are used in statistics, information theory, game theory, economics, genetics, finance, programmed music generation, and many more.
There are many examples of their implementation. In thermodynamics, they are used to model time-invariant processes with no history considered, and in chemistry, to model multiple reactions. In neurobiology, they appear in mammalian neocortex simulations.
Markov chains are also the base of the entropy concept in information theory, as defined by Claude Shannon in 1948. The PageRank algorithm used by Google to rank Internet pages employs a Markov chain. And, in economics, they are used to simulate a variety of phenomena, such as GDP change and asset prices.
LogicPlum + Markov Chains
Markov chains have a proven record of successful applications in many areas. Their mathematical foundation is quite simple, a fact that makes them easy to understand too. However, their application usually requires a sound understanding of mathematics and statistics.
This difficulty is avoided by LogicPlum’s platform, as it solves all mathematical and statistical calculations in an automated manner, without requiring any human intervention. Furthermore, it provides its users with an automatically generated report that describes in detail the steps taken during the modeling process.
Guide to Further Reading
Wikipedia. (2021). Markov Chain. Available at https://en.wikipedia.org/wiki/Markov_chain
