Glynna, dirk ormoneitb adepartment of management science and engineering, stanford university, stanford, ca 943059010, usa bdepartment of computer science, stanford university, stanford, ca 943059010, usa received march 2001. Twelve proofs of the markov inequality aleksei shadrin this is the story of the classical markov inequality for the kth derivative of an algebraic polynomial, and of the remarkably many attempts to provide it with alternative proofs that occurred all through the last century. May 18, 2017 the purpose of this talk is to present bernstein and hoeffding type functional inequalities for regenerative markov chains. The emphasis in this book is placed on general models markov chains, random fields, random graphs, universal methods the probabilistic method, the coupling method, the steinchen method, martingale methods, the method of types and versatile tools chernoffs bound, hoeffding s inequality, holleys inequality whose domain of application extends far beyond the present text. Jul 10, 2019 further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential decay with the optimal large deviations rate. Optimal hoeffding bounds for discrete reversible markov chains by carlos a. The markov and chebyshev inequalities we intuitively feel it is rare for an observation to deviate greatly from the expected value. This paper develops a hoeffding inequality for the partial sums. Sham kakade 1 hoeffdings bound we say x is a subgaussian random variable if it has quadratically bounded logarithmic moment generating function,e.
A large deviation inequality for vector functions on finite reversible markov chains by vladislav kargin download pdf 246 kb. Moreover through a pinsker type inequality and a hoeffding type lemma, we are able to loosen up our chernoff type bound to a hoeffding type bound and reveal the subgaussian nature of the sums. Optimal hoeffding bounds for discrete reversible markov chains. Y, has been the method of choice for proving impossibility converse results. Markov s inequality and chebyshevs inequality place this intuition on firm mathematical ground. Markov s inequality often, for a random variable x that we are interested in, we want to know what is the probability that the value of the r. Hoeffdings inequality for uniformly recurrent markov chains. And lets say that we know the value of ex, say ex 1. Pdf we prove a sharp bernstein inequality for generalstatespace. Cherno hoe ding inequality and applications jeff m. Finally, countable chains with a spectral gap can be handled in the same manner. These generally work by making many simple estimates of the full data set, and then judging them as a whole. Because markov s inequality works for all nonnegative random variables, it cannot possibly give the exact probability for all such random variables.
A hoeffding inequality for finite state markov chains and its. Samson, concentration of measure inequalities for markov chains and. If the expectation value of a nonnegative random variable is small, then the random variable must itself be small with high probability. Wolpert department of statistical science duke university, durham, nc, usa for each constant c 0, any nonnegative integrable random variable y satis. Pdf a simple variance inequality for ustatistics of a. In work done independently of this thesis, a version of hoeffdings inequality for general. Dataprocessing, fano dataprocessing inequality su cient statistics fanos inequality dr. Markovs inequality and chebyshevs inequality place this intuition on firm mathematical ground.
Pdf bernsteins inequality for general markov chains. Strong dataprocessing inequalities for channels and bayesian networks yury polyanskiy and yihong wu abstract the dataprocessing inequality, that is, iu. This inequality was originally proved in the 1960s and will imply that pr rb nh rh 2e 2n 2. A large deviation inequality for vector functions on. If x is a nonnegative random variable and a 0, then the. The blue line the function that takes the value \0\ for all inputs below \n\, and \n\ otherwise always lies under. Variants of chernoffhoeffding bounds for random walk on markov chains. Discrete probability models and methods probability on. Applications to economic growth and convergence michael zabek an important question in growth economics is whether the incomes of the worlds poorest nations are either converging towards or moving away from the incomes of the worlds richest nations.
We establish a simple variance inequality for ustatistics whose underlying sequence of random variables is an ergodic markov chain. Eesn e since ex is a convex function of x, the expectation on the right side can be bounded using corollary 1. This paper uses a generalized inverse to derive a hoeffding inequality for partial sums which proceed from uniformly ergodic markov chains. Hoeffding s inequality is a generalization of the chernoff bound, which applies only to bernoulli random variables, and a special case of the azuma hoeffding inequality and the mcdiarmids inequality. A hoeffding inequality for finite state markov chains and. New bernstein and hoeffding type inequalities for regenerative. Citeseerx document details isaac councill, lee giles, pradeep teregowda. Concentration and anticoncentration for markov chains.
In the second half, the authors delve into more advanced topics such as continuous probability, applications of limited independence, entropy, markov chain monte carlo methods. I just wanted to throw in a picture explanation into the mix of great answers. Then for any positive number a we get the following. Hoe dings inequality is similar in spirit to markov s inequality but it is a sharper inequality. This property also holds when almost surely in other words, there exists a zeroprobability event such that. Yao xie, ece587, information theory, duke university. Markov s inequality and other similar inequalities relate probabilities to expectations, and provide frequently loose but still useful bounds for the cumulative distribution function of a random variable. The technique of applying markovs inequality with a free parameter here t and choosing it optimally can be very powerful.
Hoeffdings lemma for markov chains and its applications. Hoe dings inequality is similar in spirit to markovs inequality but it is a sharper inequality. Hoeffding bounds for markov chains 3 which will serve as a bridge between the initial chain xk and the twostate case through a positive semide. Our exponential inequality on the deviation of these sums from their expectation is particularly useful in situations where we require uniform control on the constants appearing in the bound. The constants in this inequality are explicit and depend on computable bounds on the mixing rate of the markov chain. Hoeffding, chernoff, bennet, and bernstein bounds instructor. We provide a generalization of hoeffding s inequality to partial sums that are derived from a uniformly ergodic markov chain. Theorem 3 shaves off the extra leading factors of t in these inequalities, which has. The hoeffding s inequality is a crucial result in probability theory as it provides an upper bound on the probability that the sum of a sample of independent random variables deviates from its expected value. Say that your random variable x can take on one of three values, 0, 1, and 2. Randomized algorithms and probabilistic analysis michael.
Phillips when dealing with modern big data sets, a very common theme is reducing the set through a random process. For any nonnegative random variable mathymath, and constant mathc \gt 0math, markovs inequality puts an upper bound, that is dependent on the expected value of that random variable matheymath, and mathcmath, on that random. Zerosum continuoustime markov games with unbounded transition and discounted payoff rates guo, xianping and hernandezlerma, onesimo, bernoulli, 2005. A generic answer to this, which holds for anynonnegative random variable, is given bymarkovs inequality. Hoeffding s inequality was proven by wassily hoeffding in 1963. Our results assume none of countable state space, reversibility and timehomogeneity of markov chains and cover timedependent functions with various ranges. In bennet inequality, we assume that the variable is upper bounded, and want to. Moreover, through a pinsker type inequality and a hoeffding type lemma, we are able to loosen up our chernoff type bound to a hoeffding type bound and reveal the subgaussian nature of the.
Azumahoeffding inequality for sums of bounded random variables. Further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential decay with the optimal large deviations rate. The remarkable aspect about it is that the inequality holds for any distribution with positive values, no matter what other features that it has. Hoedings inequality for uniformly ergodic markov chains. It is named after the russian mathematician andrey markov, although it appeared earlier in the work of pafnuty chebyshev markov s teacher, and many sources, especially in analysis, refer to it as chebyshevs inequality sometimes, calling it the first chebyshev inequality, while referring to chebyshevs inequality as the second chebyshev. In probability theory, hoeffding s inequality provides an upper bound on the probability that the sum of bounded independent random variables deviates from its expected value by more than a certain amount. Hoeffding s inequality for uniformly recurrent markov chains. We provide a generalization of hoeffdings inequality to partial sums that are derived from a uniformly ergodic markov chain. Similar to the discussion in the previous section, let a 1, a 2. Hoeffdings inequality was proven by wassily hoeffding in 1963. What is an intuitive explanation of markovs inequality. Optimal chernoff and hoeffding bounds for finite markov chains. In our theorem 1 we develop a version of hoeffdings inequality for finite state markov chains.
However, markov bounds are occasionally useful in practical situations and frequently used to prove limit theorems. Markovs inequality often, for a random variable x that we are interested in, we want to know what is the probability that the value of the r. In our survey we inspect each of the existing proofs and describe. It is noteworthy that when deriving exponential inequalities for markov chains or any other process with some. Hoeffdings inequality is a generalization of the chernoff bound, which applies only to bernoulli random variables, and a special case of the azumahoeffding inequality and the mcdiarmids inequality. What are the differences between chebyshev, hoeffding, and. We can prove the above inequality for discrete or mixed random variables similarly using the generalized pdf, so we have the following result, called markovs inequality. A hoeffding inequality for markov chains project euclid.
Our exponential inequality on the deviation of these sums from their expectation is particularly useful in situations where we require uniform control on. Markovs inequality is tight, because we could replace 10 with tand use bernoulli1, 1t, at least with t 1. The counterpart of hoeffding s inequality for markov chains immediately follows. A hoeffding inequality for finite state markov chains and its applications to markovian bandits. Hoeffdings inequality for uniformly ergodic markov chains. On dirichlet eigenvectors for neutral twodimensional markov chains champagnat, nicolas, diaconis, persi, and miclo, laurent, electronic journal of probability, 2012. Furthermore, we generalize these results and show exponential bounds for. Markov and chebyshev inequalities, part 1 duration. Markovs inequality is a helpful result in probability that gives information about a probability distribution. Chernoffhoeffding bounds for markov chains columbia university. The answer is no, if the search is for bounds which are independent of the initial x. Sham kakade 1 hoeffdings bound we say x is a subgaussian random variable if it has quadratically bounded logarithmic moment generating func. Glynn and ormoneit established the hoeffding s inequality for uniformly ergodic markov chains based on the equivalent definition of uniformly ergodic markov chains.
It provides an upper bound to the probability that the realization of a random variable exceeds a given threshold. A key point to notice is that the probability in 1 is with respect to the draw of the training data. Glynna, dirk ormoneitb adepartment of management science and engineering, stanford university, stanford, ca 943059010, usa. A hoeffding inequality for markov chains using a generalized. Strong dataprocessing inequalities for channels and. Gabriela ciolek sharp bernstein and hoeffding type. It may be asked whether the hoeffding inequality can be extended to more general markov chains, such as fregular or vuniformly ergodic markov chains. One use of markovs inequality is to use the expectation to control the probability distribution of a random variable. Finally we show a uniform multiplicative ergodic theorem for our class of markov chains. We illustrate the utility of these results by applying them to six problems in statistics and machine learning.
704 1165 129 654 1070 600 362 293 783 238 761 321 12 1130 358 1411 519 1276 1614 1613 1566 187 561 843 1296 371 1468 966 526 616 665 594 1435 690 587 530