Introduction to Finite Markov Chains

The aim of this book is to introduce the reader and develop his knowledge on a specific type of Markov processes called Markov chains. This book presents finite Markov chains, in which the state space finite, starting from introducing the readers the finite Markov chains and how to calculate their transition probabilities, as well as the transition probability matrices and the graphical representation of transition. The classification of these Markov chains through classifying the state space in this process is obtained. Also, the subject of absorbing Markov chains and the calculation of absorbing probabilities were also addressed. In addition to the subject of stationarity in Markov chains where the stationary distributions as well as the study of a new type, the so-called quasi-stationary distributions in absorbing Markov chains were studied. Furthermore, this book ends with presenting some applications that appear in practical life. Finally, this book will be a very useful reference or text for the undergraduate course on finite Markov chains as well as researchers in statistics, stochastic processes, stochastic modeling, and operations researches and in all related areas.