📚 Trusted by students, educators & researchers since 2017.

Glossary

A

Alpha ($\alpha$): Also known as the significance level. It is the probability of rejecting the null hypothesis when it is actually true (making a Type I error). The standard default in most research is 0.05 (a 5% risk).

Alternative Hypothesis ($H_a$): The claim you are actively trying to prove in a hypothesis test. It usually states that there is a significant difference, effect, or relationship in your data.

ANOVA (Analysis of Variance): A parametric hypothesis test used to determine if there is a statistically significant difference between the averages of three or more independent groups.

Autocorrelation: The degree of similarity between a given time series and a lagged version of itself over successive time intervals.

B

Bayes’ Theorem: A mathematical formula used to update the probability of a hypothesis as more evidence or information becomes available.

Bias: A systematic error in data collection, sampling, or analysis that causes the results to inaccurately represent the true population.

Binomial Distribution: A probability distribution that models the number of successes in a fixed number of independent yes/no trials (e.g., flipping a coin 10 times and getting exactly 7 heads).

C

Central Limit Theorem (CLT): A foundational rule stating that if you take sufficiently large random samples from any population, the distribution of those sample means will form a normal bell curve—even if the original population is heavily skewed.

Chi-Square Test ($\chi^2$): A hypothesis test used to determine if there is a significant association between two categorical variables (e.g., comparing voting preferences across different age brackets).

Cluster Sampling: A method where the entire population is divided into groups (clusters), and a random selection of these entire clusters are chosen to be sampled.

Cohen’s d: A standardised measure of effect size. It measures the distance between two group means in terms of standard deviations, telling you how practically important a difference actually is.

Confidence Interval (CI): A calculated range of values that is likely to contain the true population parameter (like a mean or proportion). A 95% CI means you are 95% confident the true value falls within that specific range.

Confounding Variable: An unmeasured “third variable” that influences both the supposed cause and the supposed effect, creating a false association between them (e.g., ice cream sales and shark attacks are correlated, but “summer weather” is the confounding variable).

Continuous Data: Quantitative data that can take on any value within a range, including infinite fractions and decimals (e.g., height, weight, time).

Control Group: A baseline group in an experiment that does not receive the treatment or intervention being tested. Their results are compared against the experimental group.

Correlation: A statistical measure that describes the size and direction of a relationship between two or more variables. (Note: Correlation does not equal causation).

D

Degrees of Freedom ($df$): The number of independent values in a calculation that are free to vary without violating any given constraints. It is a crucial mathematical component for finding p-values in T-tests, ANOVA, and Chi-Square tests.

Dependent Variable: The outcome or variable you are trying to measure, predict, or explain. It “depends” on the changes made to the independent variable.

Descriptive Statistics: A branch of statistics focused entirely on summarising and organising data (e.g., mean, median, mode, standard deviation) rather than making predictions about a larger population.

Double-Blind Study: An experiment where neither the participants nor the researchers interacting with them know who is in the control group and who is in the experimental group, preventing placebo effects and observer bias.

E

Effect Size: A quantitative measure of the magnitude of a phenomenon or the strength of a relationship. While a p-value tells you if an effect exists, the effect size tells you how big it is.

Expected Value: The long-run average outcome of a random variable or probability scenario if you were to repeat the trial infinitely.

Exponential Smoothing: A time series forecasting method that applies decreasing, exponential weights to older data, allowing the forecast to react quickly to recent trend changes.

F

F-Statistic: The test statistic generated by an ANOVA or regression analysis. It is calculated by dividing the variance between groups by the variance within groups.

I

Independent Variable: The variable that is manipulated or categorized by the researcher to see if it causes a change in the dependent variable.

Interquartile Range (IQR): A measure of statistical dispersion. It represents the middle 50% of a dataset, calculated by subtracting the first quartile (25th percentile) from the third quartile (75th percentile).

L

Logistic Regression: A type of regression analysis used when the dependent variable is binary (e.g., predicting “pass/fail” or “yes/no” outcomes rather than a continuous number).

M

Mann-Whitney U Test: The non-parametric alternative to the independent two-sample t-test. It is used to compare the median differences between two independent groups when the data is not normally distributed.

Margin of Error: The maximum expected difference between the true population parameter and a sample estimate. It forms the “plus or minus” bounds of a confidence interval.

Mean ($\mu$ or $\bar{x}$): The mathematical average of a dataset, found by adding all numbers together and dividing by the total count.

Median: The exact middle value of a dataset when arranged in numerical order. It is highly resistant to extreme outliers.

Mode: The most frequently occurring value in a dataset.

Moving Average (SMA): A time series smoothing technique that calculates the unweighted average of the previous k data points to identify trends and filter out random noise.

Multicollinearity: A statistical headache in multiple regression that occurs when two or more independent variables are highly correlated with each other, making it difficult to determine which one is actually affecting the dependent variable.

N

Non-Parametric Test: A statistical test that does not assume the data follows a specific distribution (like a normal bell curve). These are ideal for skewed data, small sample sizes, or ordinal rankings.

Normal Distribution: A continuous probability distribution perfectly shaped like a symmetrical bell curve. In a normal distribution, the mean, median, and mode are all perfectly equal.

Null Hypothesis ($H_0$): The baseline assumption in a hypothesis test. It asserts that there is no true difference, no effect, or no relationship between the variables being studied.

P

P-Value: The probability of observing your exact sample data (or something even more extreme) assuming the null hypothesis is 100% true. A low p-value (typically under 0.05) indicates statistical significance.

Parametric Test: A statistical test that assumes the data follows a specific underlying distribution (usually a normal distribution). Examples include T-tests and ANOVA.

Permutations & Combinations: Mathematical methods used to calculate the total number of possible outcomes in a dataset. Combinations are used when order does not matter; permutations are used when order does matter.

Poisson Distribution: A probability distribution that models the number of times an event occurs within a specific, fixed interval of time or space (e.g., how many emails you receive between 9 AM and 10 AM).

Population: The entire, complete group of individuals or items that a researcher wants to draw conclusions about. (Because measuring a whole population is usually impossible, researchers use samples).

Power (Statistical Power): The probability that a test will correctly reject a false null hypothesis (i.e., successfully detect a real effect when one exists).

Principal Component Analysis (PCA): A multivariate dimensionality-reduction technique. It condenses massive datasets with dozens of variables into a few “principal components” while preserving as much of the original variance as possible.

R

R-Squared ($R^2$): Also known as the coefficient of determination. It represents the percentage of the variance in the dependent variable that can be explained by the independent variables in a regression model (ranging from 0 to 1, or 0% to 100%).

Regression Analysis: A statistical process for estimating the relationships among variables. Linear regression helps predict the value of a dependent variable based on the value of one or more independent variables.

Residual: In regression, the difference between the actual observed value and the value predicted by the regression line.

S

Sample: A smaller, representative subset drawn from a larger population, used to make inferences about that population.

Seasonality: Predictable, repeating fluctuations in time series data that occur at specific, regular intervals (like retail sales spiking every December).

Simple Random Sampling (SRS): The most basic sampling method where every single individual in a population has an equal, non-zero chance of being selected.

Skewness: A measure of the asymmetry of a distribution. A “right-skewed” distribution has a long tail stretching toward higher positive numbers, while a “left-skewed” distribution stretches toward lower numbers.

Standard Deviation ($\sigma$ or $s$): A measure of how spread out the numbers in a dataset are. A low standard deviation means the data is tightly clustered around the mean; a high standard deviation means the data is widely spread.

Standard Error: The estimated standard deviation of a sample statistic (like the sample mean). It tells you how far your sample mean is likely to be from the true population mean.

Stationarity: A property of a time series where its statistical properties (like mean and variance) remain constant over time. Most advanced time series models require data to be made stationary before forecasting.

Stratified Sampling: A method where the population is divided into distinct subgroups (strata) based on a specific characteristic (like age or income), and random samples are drawn proportionally from each subgroup.

T

T-Test: A parametric hypothesis test used to compare the averages of one or two groups. It is typically used when the population standard deviation is unknown and the sample size is relatively small.

Type I Error: A “false positive.” Concluding there is a statistically significant effect or difference when, in reality, there is not.

Type II Error: A “false negative.” Failing to detect a statistically significant effect or difference when one actually exists in reality.

V

Variance ($\sigma^2$ or $s^2$): The average of the squared differences from the mean. It is the mathematical precursor to standard deviation.

W

Wilcoxon Signed-Rank Test: The non-parametric alternative to the paired t-test. It is used to compare two related or matched samples (like pre-test/post-test scores) to assess whether their population mean ranks differ.

Z

Z-Score: A measurement of how many standard deviations a raw data point is away from the mean. A Z-score of 0 means the score is exactly average; a Z-score of 1.5 means it is 1.5 standard deviations above average.

Z-Test: A parametric hypothesis test used to compare group means or proportions. Unlike a t-test, it requires either a massive sample size or a known population standard deviation.