Statistics is the branch of mathematics that deals with collecting, organizing, summarizing, analyzing, and drawing conclusions from data. It is the discipline that turns raw numbers into insight. Every field you can think of leans on statistics: medicine uses it to decide whether a new drug works, economics uses it to forecast growth and inflation, sports teams use it to evaluate players, marketing teams use it to measure campaign performance, and machine learning uses it at almost every step — from exploring a dataset to evaluating a model's accuracy.
In AI and data science, statistics is the foundation. When a model predicts a price, we measure the error using statistical concepts. When we decide whether a feature helps, we look at correlations. When we build a confidence interval around a model's performance, we are using statistical inference. Understanding statistics first makes every later chapter — probability, hypothesis testing, regression, and evaluation — far more intuitive.
Statistics splits into two broad camps:
Think of descriptive statistics as “telling the story of the data you hold” and inferential statistics as “making a bet about the data you do not hold.” A bakery weighing every loaf it baked today is doing descriptive work; using today's loaves to predict tomorrow's average weight is inference.
The population is the entire group you care about — every voter, every transaction, every patient. The sample is the subset you can actually measure. We almost never have access to the whole population: it is too large, too expensive, or too slow to measure. Instead we sample, measure the sample, and infer back to the population.
This introduces an important idea: a sample is only useful if it is representative. A convenience sample — asking only your friends — will almost certainly mislead you. Random sampling helps keep bias out. As you read on, notice how the words “mean” and “variance” get a slightly different symbol depending on whether we are talking about a population (μ, σ2) or a sample (x̄, s2).
Central tendency answers: “what is a typical value in this dataset?” Three classic measures exist.
The mean (arithmetic average) is the sum of all values divided by the count. For a sample of n values x1, x2, ..., xn:
mean = (x1 + x2 + ... + xn) / nThe mean is sensitive to outliers — one extreme value can pull it far from the “typical” value. That is why the median often matters more for skewed data like salaries or house prices.
The median is the middle value once the data is sorted. With an odd count, it is the center value; with an even count, it is the average of the two middle values. Because it depends only on position, not magnitude, the median is robust to extreme values.
The mode is the value that appears most frequently. It is the only measure of central tendency that works for categorical data (like the most common color). A dataset can have several modes (bimodal, multimodal) or no mode at all.
Central tendency tells us where the data centers, but two datasets can share the same mean yet look completely different. Dispersion measures how spread out the values are.
The range is the simplest measure: maximum value minus minimum value. It is easy to compute but only uses two values, so it is easily fooled by a single outlier.
Variance measures the average squared distance of each value from the mean. Squaring the deviations does two jobs: it removes the sign (so negatives and positives do not cancel out) and it punishes large deviations extra hard. The population variance is:
variance = sum((xi - mean)^2) / nThe standard deviation is simply the square root of the variance. It brings the units back to the original scale — if the data is in dollars, the standard deviation is also in dollars, which makes it directly interpretable. A small standard deviation means the values hug the mean; a large one means they scatter widely.
The IQR is the range covered by the middle 50% of the data. It is the difference between the third quartile (Q3, the 75th percentile) and the first quartile (Q1, the 25th percentile). Because it ignores the top and bottom quarters, it is highly robust to outliers and is the basis of the classic box plot.
You may have noticed textbooks compute sample variance with n−1 in the denominator:
sample variance = sum((xi - xbar)^2) / (n - 1)The reason is subtle and worth understanding. When you compute the sample mean x̄ from the data itself, the deviations are no longer fully free: if you know n−1 of the deviations, the last one is forced, because the deviations must sum to zero. We say the data has n−1 degrees of freedom.
More importantly, the sample variance with n in the denominator underestimates the true population variance on average. The sample mean is the single value that minimizes the sum of squared deviations, so distances measured from it tend to be smaller than distances measured from the true population mean. Dividing by n−1 (Bessel's correction) inflates the estimate just enough to make it unbiased — meaning if you repeated the experiment many times, the average of your sample variances would land on the true population variance.
A variable is a characteristic that can take different values across individuals or observations. Variables come in two main flavors:
The type of a variable drives every analysis choice: you compute means for quantitative data but modes for categorical data, and you encode categorical variables differently when building models (a topic we return to in the data-cleaning chapter).
A random variable is a variable whose possible values come from the outcome of a random process. It is a function that maps outcomes of an experiment to numbers. For example, “the sum of two rolled dice” is a random variable, as is “the number of customers arriving in an hour” or “the height of a randomly selected adult.”
Random variables come in two kinds that mirror our variables above: discrete random variables take a countable set of values (like dice sums), while continuous random variables take any value in a range (like heights). We usually denote a random variable with an uppercase letter (X) and a specific observed value with lowercase (x). Random variables are the bridge between statistics and probability, which we explore in the next chapter.
A histogram is the workhorse chart of descriptive statistics. It divides a numeric variable's range into intervals (bins), counts how many observations fall in each bin, and draws bars whose heights are those counts. The histogram reveals the shape of the data at a glance: where it centers, how wide it is, whether it is symmetric or skewed, and whether it has one hump or several.
import matplotlib.pyplot as plt
import numpy as np
scores = np.array([52, 61, 63, 68, 71, 74, 75, 78, 80, 80, 83, 85, 90, 95])
plt.hist(scores, bins=5, edgecolor='white')
plt.xlabel('Score')
plt.ylabel('Count')
plt.title('Histogram of Exam Scores')
plt.show()Choosing the bin width matters: too few bins hide detail, too many bins create noisy spikes. Histograms are a descriptive tool — they summarize the sample in front of you, nothing more.
A percentile is a value below which a given percentage of the data falls. The 25th percentile is the value below which a quarter of observations sit; the 90th percentile, below which 90% sit. Percentiles are the standard language of standardized tests and of performance benchmarks.
Quartiles are just three special percentiles that split the data into four equal parts:
The gap between Q1 and Q3 is the interquartile range you met earlier. Percentiles also power a handy outlier rule: points more than 1.5 × IQR below Q1 or above Q3 are often flagged as outliers in box plots.
The five-number summary condenses a variable into five numbers that describe its distribution: the minimum, Q1, the median (Q2), Q3, and the maximum. Together they capture both location and spread, and they are exactly what a box plot draws. The box spans Q1 to Q3 (with a line at the median), and the “whiskers” extend toward the minimum and maximum (often trimmed to 1.5 × IQR, with outliers plotted as dots beyond them).
import numpy as np
values = np.array([3, 7, 8, 9, 11, 12, 15, 18, 22, 40])
q1, med, q3 = np.percentile(values, [25, 50, 75])
print("Min:", values.min())
print("Q1:", q1)
print("Median:", med)
print("Q3:", q3)
print("Max:", values.max())So far we have described single variables. Covariance measures how two variables move together. It is the average of the products of each variable's deviation from its own mean:
cov(X, Y) = sum((xi - xbar)(yi - ybar)) / (n - 1)A positive covariance means that when one variable is above its mean, the other tends to be too (they rise and fall together). A negative covariance means they tend to move in opposite directions. The catch is that covariance has units (the product of the two variables' units), so it is hard to compare across different pairs.
Correlation fixes that by dividing covariance by the product of the two standard deviations. The result, Pearson's r, is unit-free and always lies between −1 and +1. An r near +1 means a strong positive linear relationship, near −1 a strong negative linear relationship, and near 0 little or no linear relationship.
import numpy as np
x = np.array([2, 4, 6, 8, 10])
y = np.array([1, 3, 5, 9, 11])
print("Covariance:", np.cov(x, y, ddof=1)[0, 1])
print("Correlation:", np.corrcoef(x, y)[0, 1])