Data Analysis
This chapter covers turning prepared data into insight. You will learn the four levels of analytics, common descriptive statistics, the difference between correlation and causation, and the basics of hypothesis testing.
Levels of analytics
Analytics is often described in four increasing levels. Descriptive analytics summarizes what happened, diagnostic analytics explains why it happened, predictive analytics forecasts what is likely to happen, and prescriptive analytics recommends what action to take. Recognizing which level a question calls for guides the methods and tools an analyst applies.
Descriptive statistics
Central-tendency measures summarize a typical value: the mean is the arithmetic average, the median is the middle value, and the mode is the most frequent value. The median is more resistant to extreme outliers, so it better represents skewed data. Dispersion measures such as the range and standard deviation describe how spread out values are, with a larger standard deviation indicating greater variability.
Correlation versus causation
Correlation measures how strongly two variables move together, but a correlation does not by itself prove that one variable causes the other. Both may be driven by a hidden third factor, or the relationship may be coincidental. Establishing causation requires stronger evidence, such as a controlled experiment, and confusing the two leads to poor decisions.
Hypothesis testing
Hypothesis testing evaluates a claim about data. The analyst states a null hypothesis and an alternative, chooses a significance level (alpha, often 0.05), and computes a p-value. If the p-value is below alpha, the result is statistically significant and the null hypothesis is rejected; otherwise there is insufficient evidence to reject it. Testing never proves the null true, it only fails to reject it.