DISCRETE DISTRIBUTIONS

Hello, Letícia from Minha Estatística here :). Clique aqui para ir ao post em Português.

This week, we’re taking a brief break from our Graph Series—if you’ve been following along, you know how it’s been an exciting journey so far! For those who are new, welcome aboard! In this week’s post, we’re covering an equally fascinating topic: discrete probability distributions!

As you may or may not know, there are two types of probability distributions: discrete and continuous distributions. Discrete distributions are characterized by modeling random variables that take on a finite or countable number of distinct values. These distributions are used when the possible outcomes of an experiment can be enumerated and counted, such as the number of successes in a sequence of trials or the number of occurrences of an event within a specific interval.

Both distributions, continuous and discrete, have their own cumulative distribution function (CDF), which represents the probability that a random variable will take a value less than or equal to a certain point, meaning \(F(a) = P(X \leq a)\), where \(F(a)\) is the distribution function \(F\) of a random variable \(X\), with\( −∞ < a < +∞ \).

For discrete distributions, the distribution function \(F(x)\) has three properties:

1. If \(a \leq b\), then \(F(a) \leq F(b)\), meaning that it is non-decreasing (monotonicity).
2. Since \(F(a)\) is a probability, the value of the distribution function is always bounded between 0 and 1.
3. The value of the CDF immediately to the right of \(a\) approaches the value of the CDF at \(a\).

The third propertie means that, for a discrete random variable, the CDF is a step function and between these points the value of \(F(x)\) remains constant. For example, if \(X\) can take values 1, 2, 3 with probabilities: \(P(X = 1) = 0.2\), \(P(X = 2) = 0.3\), \(P(X = 3) = 0.5\), the cumulative distribution function (CDF) increases in steps at 1, 2, and 3. It can be represented as:

• \( F(x) = 0 \), for \( x < 1 \)
• \( F(x) = 0.2 \), for \( 1 \leq x < 2 \)
• \( F(x) = 0.5 \), for \( 2 \leq x < 3 \)
• \( F(x) = 1.0 \), for \( x \geq 3 \)

The CDF is a step function that captures the cumulative probability up to each value, \(F(a) = \sum p(a_i)\):

• At \( x = 1 \), \( F(1) = P(X \leq 1) = 0.2 \)
• At \( x = 2 \), \( F(2) = P(X \leq 2) = 0.2 + 0.3 = 0.5 \)
• At \( x = 3 \), \( F(3) = P(X \leq 3) = 0.2 + 0.3 + 0.5 = 1.0 \)

This shows how the three properties of discrete distributions work, more specifically, how they are non-decreasing, bounded to have results between \(0\) and \(1\), and that the CDF increases step-by-step, as the probabilities are accumulated for each possible value of \(X\). The Cumulative Distribution Function (CDF) helps us understand cumulative probabilities up to a certain point.

However, there’s another fundamental concept to consider: the Probability Mass Function (PMF). Unlike the CDF, which focuses on cumulative probabilities, the PMF deals with individual outcomes, assigning probabilities to each specific value of a discrete random variable. The Probability Mass Function (PMF) of a random variable \( X \) is a function \( P(X = x) \) that gives the probability of \( X \) taking on a specific value \( x \). The PMF is expressed as:

\[ P(X = x) = p(x) \]

Where:

• \( X \) is the random variable
• \( x \) is a specific value that \( X \) can take
• \( p(x) \) represents the probability associated with \( x \)

It’s important to note that the PMF applies only to discrete distributions: for continuous random variables, we use the Probability Density Function (PDF) instead.

Now that we've covered the CDF and PMF, let's dive into some of the most commonly used discrete distributions. For each distribution, we will break down its purpose, application, PMF, and key statistical formulas, including the mean and variance.

• Uniform Distribution

The discrete Uniform distribution is a probability distribution where each of the \( k \) possible outcomes has an equal probability of occurring. It is used to model situations where all outcomes within a finite set are equally likely. Let \( X \) be the random variable representing the outcome from a discrete Uniform distribution, with \( k \) possible values, typically ranging from \( 1 \) to \( k \) or from \( a \) to \( b \).

Probability Mass Function (PMF): \( f(x) = \frac{1}{k} \quad (x = 1, 2, \dots, k)\) Mean: \( E[X] = \frac{a + b}{2} \) Variance: \( \text{Var}(X) = \frac{(b - a + 1)^2 - 1}{12} \)

The discrete Uniform distribution is used in various applications where all outcomes within a given range are equally likely, such as in games of chance, e.g., rolling dice, card shuffling, and random number generation.

• Bernoulli Distribution

The Bernoulli distribution is a simple discrete distribution. It describes a single experiment that has only two possible outcomes: success (1) or failure (0), being useful in these situations with binary outcomes, such as coin tosses.

Being a discrete random variable \(X\) follows a Bernoulli (\(X \sim \text{Ber}(p)\)) distribution with parameter \(p\) where \(0 \leq p \leq 1\) we have:

Probability Mass Function (PMF): \(f_{X}(1) = P(X = 1) = p\),
\(f_{X}(0) = P(X = 0) = 1 - p\), and
\(P(X = x) = p^x(1 - p)^{1 - x} \quad (x \in \{0, 1\})\) Mean: \(E[X] = p\) Variance: \(\text{Var}(X) = p(1 - p)\)

The Bernoulli distribution is a special case of the Binomial distribution when \(n = 1\), as shown next.

• Binomial Distribution

The Binomial distribution is a discrete distribution that describes the number of successes in a fixed number of independent Bernoulli trials. Each trial has two possible outcomes: success (1) or failure (0). It's commonly used to model situations like the number of heads in multiple coin tosses, or the number of successful outcomes in a series of trials.

Being a discrete random variable, \(X\) follows a Binomial distribution with parameters \(n\) (the number of trials) and \(p\) (the probability of success on a single trial), written as \(X \sim \text{Bin}(n, p)\), with:

Probability Mass Function (PMF): \(P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k} \quad (k = 0, 1, 2, \dots, n)\) Mean: \(E[X] = np\) Variance: \(\text{Var}(X) = np(1 - p)\)

The Binomial distribution can be considered as the sum of \(n\) independent Bernoulli trials, each with success probability \(p\). It models the probability of obtaining exactly \(k\) successes in \(n\) trials, where each trial has two possible outcomes (success or failure).

• Poisson Distribution

The Poisson distribution is often used to model the number of occurrences of rare events within a fixed interval of time or space. Common applications include modeling events such as radioactive decay, traffic accidents, or the number of phone calls at a call center. These events are typically rare, occur independently and happen at a constant average rate.

Let \(X\) be a random variable that follows a Poisson distribution with parameter \(\lambda\):

Probability Mass Function (PMF): \(f(x) = P(X = k) = e^{-\lambda}\frac{\lambda^x}{x!} \quad(x \geq 0)\) Mean: \(E[X] = \lambda\) Variance: \(\text{Var}(X) = \lambda\)

The sum of two independent Poisson random variables is also Poisson distributed with a rate parameter equal to the sum of the individual rate parameters. That is, if \(X_1\) is a random variable that follows a Poisson distribution with rate parameter \(\lambda_1\), and \(X_2\) is another random variable that follows a Poisson distribution with rate parameter \(\lambda_2\), then \(X_1 + X_2 \sim \text{Poisson}(\lambda_1 + \lambda_2)\) (the sum of these two independent Poisson random variables will also follow a Poisson distribution with rate parameter \(\lambda_1 + \lambda_2\))

The Poisson distribution is useful in scenarios such as predicting the number of customer arrivals at a store, the number of accidents on a road, or the number of emails received in a mailbox, assuming events happen independently and at a constant rate.

• Geometric Distribution

The Geometric distribution is a discrete probability distribution that models the number of trials required to achieve the first success in a sequence of independent trials. It is particularly useful in situations where we are waiting for the first successful outcome. One famous study in the 1980s used the Geometric distribution to model the number of menstrual cycles needed for a woman to become pregnant after deciding to try for pregnancy. In this case, the distribution measures the number of cycles required to achieve the success of pregnancy, based on a fixed probability of conception within each cycle.

Being a discrete random variable, \(X\) follows a Geometric distribution \(X \sim \text{Geom}(p)\) with parameter \(p\) (the probability of success on a single trial) and:

Probability Mass Function (PMF): \(P(X = k) = (1 - p)^{k-1} p \quad (k \geq 1)\) Mean: \(E[X] = \frac{1}{p}\) Variance: \(\text{Var}(X) = \frac{1 - p}{p^2}\)

The Geometric distribution models the number of trials needed to achieve the first success. Its mean is the inverse of the probability of success on each trial. It is used in real-life situations such as predicting the number of attempts needed to get a successful outcome in processes. One example is quality control, where it can help determine the number of trials (inspections) needed to detect the first defect, allowing companies to optimize their inspection processes and reduce waste.

• Negative Binomial Distribution

The Negative Binomial distribution is used to determine the number of failures that occur before achieving a fixed number of successes. It describes the number of failures \( F_r \) that occur before the \( r \)-th success in Bernoulli trials, where the probability of success on each trial is \( p \) and \(n\) is the range, with:

Probability Mass Function (PMF): \( P(F_r = n) = P(T_r = n + r) = \binom{n + r - 1}{r - 1} p^r (1 - p)^n \quad(n = 0, 1, 2, \dots) \) \( T_r = F_r + r \) is the waiting time to the \( r \)-th success. Where \( T_r \sim \text{NB}(r, p) \), with \( T_r \in \{r, r+1, \dots\} \). Mean: \( E(F_r) = \frac{r(1 - p)}{p} \) Variance: \( \text{Var}(F_r) = \frac{r(1 - p)}{p^2}\)

The Negative Binomial distribution is especially useful for modeling the number of failures that occur before a fixed number of successes in a series of independent trials. Its applications include quality control, sales efforts, and other processes where the goal is to reach a certain number of successes despite the possibility of failures along the way.

After discussing all the most important discrete distributions and their applications, it is important to summarize the key characteristics of the distributions we have covered. Below is a summary table that highlights each distribution, along with typical applications, its probability mass function (PMF), mean, and variance. This table serves as a quick reference guide to help you better understand the unique features of each distribution and how they can be applied to various real-world scenarios.

Conclusion

In this discussion of principal discrete distributions, we have explored a variety of probability models that are essential in understanding different types of random processes. Each distribution serves a unique purpose and can be applied to specific real-world situations:

The Uniform distribution models situations where all outcomes have an equal likelihood of occurring. The Bernoulli and Binomial distributions are useful for modeling binary outcomes and the number of successes in repeated trials, respectively. The Geometric distribution is beneficial when modeling the number of trials needed for the first success, while the Poisson distribution is often applied to rare events occurring within fixed intervals. Finally, the Negative Binomial distribution extends the idea of the Geometric distribution to model failures before reaching a fixed number of successes.

These discrete distributions are fundamental tools in fields such as quality control, marketing, healthcare, and many others. Their applications provide insight into various phenomena involving uncertainty, and understanding their characteristics helps in choosing the appropriate distribution for analyzing data and making predictions. Mastery of these distributions is key for anyone working in statistics, data analysis, or probability theory.

Thank you for exploring these essential probability models with me! I hope this information helps in applying these concepts in practical situations!

We're also on Instagram @minhaestatistica, I look forward to seeing you there, until next week,

Letícia - Minha Estatística.

References

• Pitman, J. (1993). Probability. Springer.
• Wasserman, L. (2003). All of Statistics: A Concise Course in Statistical Inference. Springer.
• Dekking, F.M., Kraaikamp,C., Lopuhaä,H.P. & Meester, L.E. (2005). A Modern Introduction to Probability and Statistics: Understanding Why and How. Springer.
• Heumann, C., Schomaker, M. (2016). Introduction to Statistics and Data Analysis: With Exercises, Solutions, and Applications in R. Springer.