Tutorial V: Some Theorems !

In the last few posts we discussed about probability and probability distribution. It is the time to put an end to our discussion about them by briefly covering some inequality and theorems. So here we go:

In evaluating the performance of a digital communication system, it is often necessary to determine the area under the tail of the PDF. there for there are uppr bound on this and they are mentioned below:

1. Chebyshev inequality

X is an arbitrary random variable with finite mean m_x and finite variance σ_x². For any positive number δ , the chebyshev inequality is defined by

Chebyshev inequality

2. Chernoff bound:

The Chebyshev bound involves area under the two tails of the PDF. In some applications we are interested only in the area under on tail either in the interval (δ, ꝏ) or in the interval (- ꝏ, δ). In such a case , we can obtain an extremely tight upper bound by over bounding g(Y) by an exponential having a parameter that can be optimized to yield as tight an upper bound as possible. By considering tail probability in the area (δ, ꝏ) , the function g(Y) is over bounded as

Chernoff Bound

where g(Y) is defined as

Chernoff Bound (equation 2)

and v >= 0 is the parameter to be optimized.

Central Limit Theorem:

“The sum of statistically independent and identically distributed random variables with finite mean and variance approaches a Gaussian CDF as n → ꝏ.”

The proof of this theorem can be found any where in good text book. The matlab code for the proof will be uploaded if I will enough request. By the way the matlab code is very simple.

In the whole discussion so far we have not discussed about SAMPLING THEOREM. Which we will discuss in near future, under the title “Visiting back to Basics”. From now onwards we will discuss some practical aspects of digital communication i.e. less theory (as usual )and more codes and results.

From next post we will discuss Source Coding . If you are having any request or idea then please feel free to drop a mail or comment.