Introduction to Statistics - Part 2

In Part 1 of the series, we introduced the basic concepts of Statistics and explored the measures of central tendency and measures of dispersion.

In this part of the series, we will introduce concepts around the various probability distributions. This will require some basic understanding of Permutation & Combinations as well as Probability.

Basic Definitions

A Random Variable denotes a quantitative (numerical) value that is associated with the random outcome of a statistical experiment.

A Discrete random variable has a finite (a countable) number of outcomes (values) in a given interval.

A Continuous random variable can assume an infinite (not countable) number of possible outcomes.

A Probability Distribution is a statistical function that describes the probability of occurrence of each of the possible outcomes of the random variable in a statistical experiment.

Discrete Probability Distribution

A Discrete Probability Distribution lists out each possible value of the discrete random variable along with its probability. The distribution must satisfy the following conditions:

• The probability of each value of the discrete random variable is between 0 and 1

• The sum of all the probabilities must equal 1

The mean of a discrete probability distribution is computed using the formula:

    \(\mu = \Sigma{x_i}.{P(x_i)}\), where i = 1 to n

The variance of a discrete probability distribution is computed using the formula:

    \(\sigma^2 = \Sigma({x_i} - \mu)^2.{P(x_i)}\), where i = 1 to n

The standard deviation of a discrete probability distribution is computed using the formula:

    \(\sigma = \sqrt{\Sigma({x_i} - \mu)^2.{P(x_i)}}\), where i = 1 to n

There are two popular discrete probability distributions which are as follows:

• Binomial Probability Distribution

• Poisson Probability Distribution

Binomial Probability Distribution

A Binomial Probability Distribution is a probability experiment that must satisfy the following conditions:

• The experiment has a fixed number of repeated trials represented by n

• Each of the n trials are independent and repeated under identical conditions (the outcome of one trial does not affect the outcome of other trials)

• Each trial has only two possible outcomes - a success (p for probability of success) or as a failure (q for probability of failure = 1 - p)

• The probability of a success is the same for each trial

• The random variable x represents the number of successful outcomes from the n trials

The mathematical formula for binomial probability distribution of exactly x success(es) in n trials is as follows:

    \(_{n}C_{x}.p^{x}.q^{n-x} =\) \(\Large{\frac{n!}{(n-x)!x!}}\)\(.p^{x}.q^{n-x}\), where q = (1 - p), and \(_{n}C_{x}\) = \(\Large{\frac{n!}{(n-x)!x!}}\)

We will look at another example for binomial probability distribution.

Poisson Probability Distribution

A Poisson Probability Distribution is a probability experiment that must satisfy the following conditions:

• Each trial has only two possible outcomes - the event occurs (success) or the event *DOES NOT* occur (failure)

• The experiment consists of counting the number of times (x) the event occurs in a given interval of time

• The probability of the event occuring (success) is the same for each trial

• The number of occurrences in one interval is independent of the number of occurrences in other intervals

• The average number of times the event occurs (success) (\(\mu\)) in a given interval is known and is constant

The mathematical formula for poisson probability distribution of exactly x occurrences of the event (success) in an interval is as follows:

    \(P(x,\mu) = e^{-\mu}\).\(\Large{\frac{\mu^x}{x!}}\), where e is the base of the natural logarithm and is an irrational number that approximately equals 2.71828 and \(\mu\) is the average number of occurrences of the event per interval.

Continuous Probability Distribution

A Continuous Probability Distribution can be described using a graph f(x) that represents a Probability Density Function. The probability density function describes the likelihood that the values of the continuous random variable (x) fall within a given interval. The distribution must satisfy the following conditions:

• Because a continuous random variable (x) can have infinite values, the probability that the continuous random variable takes a specific value is 0

• The probability that the continuous random variable (x) assumes a value within an interval, say P(A ≤ x ≤ B), is defined as the area under the graph f(x) between the points A and B

• The entire area under the graph f(x) over all values of the continuous random variable (x) must equal 1

The most popular continuous probability distribution is the Normal Distribution (which is also referred to as the Gaussian Distribution).

Normal Distribution

A Normal Distribution is a continuous probability distribution for a continuous random variable (x) that must satisfy the following conditions:

• The graph f(x) (referred to as a Normal Curve) is bell-shaped curve (with a long tail) and is symmetrical around the mean

• The mean, median, and mode are equal

• The total area under the normal curve is equal to 1

• 68% of the area under the normal curve falls within 1 standard deviation of the mean

• 95% of the area under the normal curve falls within 2 standard deviations of the mean

• 99.7% of the area under the normal curve falls within 3 standard deviations of the mean

The following diagram illustrates a typical normal curve:

The mathematical equation for a normal probability density function f(x) with a mean of \(\mu\) and standard deviation of \(\sigma\) is given as follows:

    \(f(x,\mu,\sigma) =\)\(\Large{\frac{1}{\sigma\sqrt{2\pi}}}\).\(e^{-\frac{1}{2}(\frac{x - \mu}{\sigma})^2}\), where e is the base of the natural logarithm and is an irrational number that approximately equals 2.71828 and \(\pi\) is an irrational number that approximately equals 3.14159.

For each mean and standard deviation, there is a different normal distribution. In other words, there are infinitely many normal distributions with different means and standard deviations. The normal distribution with a mean of 0 and a standard deviation of 1 is called the Standard Normal Distribution.

The following diagram illustrates the standard normal curve for the standard normal distribution:

The horizontal scale (x-axis) of the standard normal curve of the standard normal distribution corresponds to what is called the Z-scores. A Z-score is a measure that indicates the number of standard deviations a value (x) lies from the mean.

The Z-score for a given x value can be computed using the following formula:

    \(Z = \Large{\frac{x - \mu}{\sigma}}\).

In general, a Z-score of more than 3 standard deviations from the mean is considered an outlier.

For a random variable x that is normally distributed, one can find the probability that x will lie in the given interval by calculating the area under the normal curve for the given interval. To find the area under any normal curve, we first have to convert the upper and lower bounds of the given interval to Z-scores. We then use the standard normal distribution to find the area under the curve.

The following illustration shows the negative Z-scores table that correspond to values which are less than the mean:

The following illustration shows the positive Z-scores table that correspond to values which are greater than the mean:

Let us look at a very simple example to find the area to the left of a given Z-score value.

Now, let us look at another simple example to find the area to the right of a given Z-score value.

Now, we look at one other simple example to find the area between the given two Z-score values.

Finally, we look at a word problem for finding probabilities with the normal distribution.

References

Introduction to Statistics - Part 1

Introduction to Permutation & Combinations

Introduction to Probability

Z-Score Table

© PolarSPARC