Introduction to Probability Theory: Independence, Conditional Probability, and Bayes’ Theorem

Probability theory forms the backbone of modern data science, artificial intelligence, and statistics.

Now, we’ll explore three fundamental concepts: Independence, Conditional Probability, and Bayes’ Theorem — with simple explanations and examples.

What is Probability?

Probability is a measure of how likely an event is to occur. It’s a number between 0 and 1:

0 means the event cannot happen.
1 means the event will definitely happen.
All other numbers in between ( like 0.3, 0.75 ), represent different levels of likelihood.

1. Independence of Events

Two events are said to be independent if the occurrence of one event does not affect the probability of occurrence of the other.

Definition:
Events A and B are independent if:

P (A and B) = P (A) \times P (B)

Example:

Tossing a coin and rolling a dice.
Getting a Head (H) and rolling a 3 are independent because the outcome of the coin toss does not impact the dice roll.

Suppose:

$P(\text{Head}) = 0.5$
$P(\text{Roll 3}) = \frac{1}{6}$

Then:

P(\text{Head and Roll 3}) = 0.5 \times \frac{1}{6} = \frac{1}{12}

Key Point:

Independence means "no influence."
If two events are independent, learning the result of one does not change the probability of occurrence of the other.

2. Conditional Probability

Conditional probability deals with finding the probability of an event given that another event has already occurred.

Definition:
The probability of event A given that event B has occurred is:

$P (A ∣ B) = \frac{P (A and B)}{P (B)}$

Where:

$P(A|B)$ = Probability of A given B.
$P(A \text{ and } B)$ = Probability of both A and B happening.
$P(B)$ = Probability of B happening.

Example:

Let's say :

30% of students play football.
70% of football players also play basketball.

What is the probability that a randomly selected football player also plays basketball?

Here:

$P (Basketball | Football) = 0.7$

This reads as: Given that a student plays football, there’s a 70% chance they also play basketball.

Key Point:

Conditional probability updates our belief based on new information.

3. Bayes’ Theorem

Bayes’ Theorem connects conditional probabilities.
It allows us to "reverse" a condition: from $P(A|B)$ to $P(B|A)$ .

Formula:

$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$

Where:

$P(A)$ = Prior probability of A.
$P(B|A)$ = Likelihood of B given A.
$P(B)$ = Total probability of B.

Why is it useful?
Because sometimes it’s easier to calculate $P(B|A)$ rather than $P(A|B)$ .

A Simple Example of Bayes' Theorem

Suppose:

1% of people have a rare disease.
A test for the disease is 99% accurate.

If a randomly selected person tests positive, what’s the probability they actually have the disease?

Given:

$P(\text{Disease}) = 0.01$
$P(\text{No Disease}) = 0.99$
$P(\text{Positive | Disease}) = 0.99$
$P(\text{Positive | No Disease}) = 0.01$

Using Bayes’ Theorem:

$P(\text{Disease | Positive}) = \frac{P(\text{Positive | Disease}) \times P(\text{Disease})}{P(\text{Positive})}$

First, calculate $P (Positive):$

$P(\text{Positive}) = (P(\text{Positive | Disease}) \times P(\text{Disease})) + (P(\text{Positive | No Disease}) \times P(\text{No Disease}))$ $= (0.99 \times 0.01) + (0.01 \times 0.99)$ $= 0.0099 + 0.0099 = 0.0198$

Now:

$P(\text{Disease | Positive}) = \frac{0.99 \times 0.01}{0.0198} \approx 0.5$

Interpretation:
Even after testing positive, the chance the person actually has the disease is only about 50%!

This is why understanding prior probabilities is so important in real-world problems like medical testing.

📊 Quick Summary Table

My Thoughts

Independence, conditional probability, and Bayes’ theorem are pillars of probability theory.
They teach us how to analyze uncertainty, make better predictions, and update our beliefs when faced with new evidence.

Mastering these concepts sets the foundation for deeper areas like machine learning, artificial intelligence, and decision theory.

Search This Blog

Statistics and Probability