**Authors**
Jonathan A. Poritz

**License**
CC-BY-SA-4.0

Lies, Damned Lies, or Statistics: How to Tell the Truth with Statistics Jonathan A. Poritz Department of Mathematics and Physics Colorado State University, Pueblo 2200 Bonforte Blvd. Pueblo, CO 81001, USA E-mail: jonathan@poritz.net Web: poritz.net/jonathan 13 M AY 2017 23:04MDT Release Notes This is a first draft of a free (as in speech, not as in beer, [Sta02]) (although it is free as in beer as well) textbook for a one-semester, undergraduate statistics course. It was used for Math 156 at Colorado State University–Pueblo in the spring semester of 2017. Thanks are hereby offered to the students in that class who offered many useful sugges- tions and found numerous typos. In particular, Julie Berogan has an eagle eye, and found a nearly uncountably infinite number of mistakes, both small and large – thank you! This work is released under a CC BY-SA 4.0 license, which allows anyone who is interested to share (copy and redistribute in any medium or format) and adapt (remix, transform, and build upon this work for any purpose, even commercially). These rights cannot be revoked, so long as users follow the license terms, which require attribution (giving appropriate credit, linking to the license, and indicating if changes were made) to be given and share-alike (if you remix or trans- form this work, you must distribute your contributions under the same license as this one) imposed. See creativecommons.org/licenses/by-sa/4.0 for all the details. This version: 13 May 2017 23:04MDT. Jonathan A. Poritz Spring Semester, 2017 Pueblo, CO, USA iii Contents Release Notes iii Preface ix Part 1. Descriptive Statistics 1 Chapter 1. One-Variable Statistics: Basics 5 1.1. Terminology: Individuals/Population/Variables/Samples 5 1.2. Visual Representation of Data, I: Categorical Variables 7 1.2.1. Bar Charts I: Frequency Charts 7 1.2.2. Bar Charts II: Relative Frequency Charts 7 1.2.3. Bar Charts III: Cautions 8 1.2.4. Pie Charts 9 1.3. Visual Representation of Data, II: Quantitative Variables 11 1.3.1. Stem-and-leaf Plots 11 1.3.2. [Frequency] Histograms 12 1.3.3. [Relative Frequency] Histograms 14 1.3.4. How to Talk About Histograms 15 1.4. Numerical Descriptions of Data, I: Measures of the Center 17 1.4.1. Mode 17 1.4.2. Mean 18 1.4.3. Median 18 1.4.4. Strengths and Weaknesses of These Measures of Central Tendency 19 1.5. Numerical Descriptions of Data, II: Measures of Spread 22 1.5.1. Range 22 1.5.2. Quartiles and the IQR 22 1.5.3. Variance and Standard Deviation 23 1.5.4. Strengths and Weaknesses of These Measures of Spread 25 1.5.5. A Formal Definition of Outliers – the 1.5 IQR Rule 25 1.5.6. The Five-Number Summary and Boxplots 27 Exercises 30 Chapter 2. Bi-variate Statistics: Basics 33 2.1. Terminology: Explanatory/Response or Independent/Dependent 33 v vi CONTENTS 2.2. Scatterplots 35 2.3. Correlation 36 Exercises 38 Chapter 3. Linear Regression 39 3.1. The Least Squares Regression Line 39 3.2. Applications and Interpretations of LSRLs 43 3.3. Cautions 45 3.3.1. Sensitivity to Outliers 45 3.3.2. Causation 46 3.3.3. Extrapolation 47 3.3.4. Simpson’s Paradox 47 Exercises 49 Part 2. Good Data 51 Chapter 4. Probability Theory 53 4.1. Definitions for Probability 55 4.1.1. Sample Spaces, Set Operations, and Probability Models 55 4.1.2. Venn Diagrams 57 4.1.3. Finite Probability Models 63 4.2. Conditional Probability 66 4.3. Random Variables 69 4.3.1. Definition and First Examples 69 4.3.2. Distributions for Discrete RVs 70 4.3.3. Expectation for Discrete RVs 72 4.3.4. Density Functions for Continuous RVs 73 4.3.5. The Normal Distribution 77 Exercises 87 Chapter 5. Bringing Home the Data 91 5.1. Studies of a Population Parameter 93 5.2. Studies of Causality 99 5.2.1. Control Groups 99 5.2.2. Human-Subject Experiments: The Placebo Effect 100 5.2.3. Blinding 101 5.2.4. Combining it all: RCTs 101 5.2.5. Confounded Lurking Variables 102 5.3. Experimental Ethics 104 5.3.1. “Do No Harm” 104 CONTENTS vii 5.3.2. Informed Consent 105 5.3.3. Confidentiality 105 5.3.4. External Oversight [IRB] 105 Exercises 107 Part 3. Inferential Statistics 109 Chapter 6. Basic Inferences 111 6.1. The Central Limit Theorem 112 6.2. Basic Confidence Intervals 114 6.2.1. Cautions 116 6.3. Basic Hypothesis Testing 117 6.3.1. The Formal Steps of Hypothesis Testing 118 6.3.2. How Small is Small Enough, for p-values? 119 6.3.3. Calculations for Hypothesis Testing of Population Means 121 6.3.4. Cautions 123 Exercises 125 Bibliography 127 Index 129 Preface Mark Twain’s autobiography [TNA10] modestly questions his own reporting of the numbers of hours per day he sat down to write, and of the number of words he wrote in that time, saying Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies, and statistics.” [emphasis added] Here Twain gives credit for this pithy tripartite classification of lies to Benjamin Dis- raeli, who was Prime Minister of the United Kingdom in 1868 (under Queen Victoria), although modern scholars find no evidence that Disraeli was the actual originator of the phrase. But whoever actually deserves credit for the phrase, it does seem that statistics are often used to conceal the truth, rather than to reveal it. So much so, for example, that the wonderful book How to Lie with Statistics [Huf93], by Darrell Huff, gives many, many examples of misused statistics, and yet merely scratches the surface. We contend, however, that statistics are not a type of lie, but rather, when used carefully, are an alternative to lying. For this reason, we use “or” in the title of this book, where Twain/Disraeli used “and,” to underline how we are thinking of statistics, correctly applied, as standing in opposition to lies and damned lies. But why use such a complicated method of telling the truth as statistics, rather than, say, telling a good story or painting a moving picture? The answer, we believe, is simply that there are many concrete, specific questions that humans have about the world which are best answered by carefully collecting some data and using a modest amount of mathematics and a fair bit of logic to analyze them. The thing about the Scientific Method is that it just seems to work. So why not learn how to use it? Learning better techniques of critical thinking seems particularly important at this mo- ment of history when our politics in the United States (and elsewhere) are so divisive, and different parties cannot agree about the most basic facts. A lot of commentators from all parts of the political spectrum have speculated about the impact of so-called fake news on the outcomes of recent recent elections and other political debates. It is therefore the goal ix x PREFACE of this book to help you learn How to Tell the Truth with Statistics and, therefore, how to tell when others are telling the truth ... or are faking their “news.” Part 1 Descriptive Statistics The first instinct of the scientist should be to organize carefully a question of interest, and to collect some data about this question. How to collect good data is a real and im- portant issue, but one we discuss later. Let us instead assume for the moment that we have some data, good or bad, and first consider what to do with them1. In particular, we want to describe them, both graphically and with numbers that summarize some of their features. We will start by making some basic definitions of terminology – words like individual, population, variable, mean, median, etc. – which it will be important for the student to understand carefully and completely. So let’s briefly discuss what a definition is, in mathematics. Mathematical definitions should be perfectly precise because they do not describe something which is observed out there in the world, since such descriptive definitions might have fuzzy edges. In biology, for example, whether a virus is considered “alive” could be subject to some debate: viruses have some of the characteristics of life, but not others. This makes a mathematician nervous. When we look at math, however, we should always know exactly which objects satisfy some definition and which do not. For example, an even number is a whole number which is two times some other whole number. We can always tell whether some number n is even, then, by simply checking if there is some other number k for which the arithmetic statement n = 2k is true: if so, n is even, if not, n is not even. If you claim a number n is even, you need just state what is the corresponding k; if claim it is not even, you have to somehow give a convincing, detailed explanation (dare we call it a “proof”) that such a k simply does not exist. So it is important to learn mathematical definitions carefully, to know what the criteria are for a definition, to know examples that satisfy some definition and other examples which do not. Note, finally, that in statistics, since we are using mathematics in the real world, there will be some terms (like individual and population) which will not be exclusively in the mathematical realm and will therefore have less perfectly mathematical definitions. Never- theless, students should try to be as clear and precise as possible. The material in this Part is naturally broken into two cases, depending upon whether we measure a single thing about a collection of individuals or we make several measurements. The first case is called one-variable statistics, and will be our first major topic. The second case could potentially go as far as multi-variable statistics, but we will mostly talk about situations where we make two measurements, our second major topic. In this case of bivariate statistics, we will not only describe each variable separately (both graphically 1The word “data” is really a plural, corresponding to the singular “datum.” We will try to remember to use plural forms when we talk about “data,” but there will be no penalty for (purely grammatical) failure to do so. 3 and numerically), but we will also describe their relationship, graphically and numerically as well. CHAPTER 1 One-Variable Statistics: Basics 1.1. Terminology: Individuals/Population/Variables/Samples Oddly enough, it is often a lack of clarity about who [or what] you are looking at which makes a lie out of statistics. Here are the terms, then, to keep straight: D EFINITION 1.1.1. The units which are the objects of a statistical study are called the individuals in that study, while the collection of all such individuals is called the popula- tion of the study. Note that while the term “individuals” sounds like it is talking about people, the indi- viduals in a study could be things, even abstract things like events. E XAMPLE 1.1.2. The individuals in a study about a democratic election might be the voters. But if you are going to make an accurate prediction of who will win the election, it is important to be more precise about what exactly is the population of all of those individuals [voters] that you intend to study, but it all eligible voters, all registered voters, the people who actually voted, etc. E XAMPLE 1.1.3. If you want to study if a coin is “fair” or not, you would flip it re- peatedly. The individuals would then be flips of that coin, and the population might be something like all the flips ever done in the past and all that will every be done in the fu- ture. These individuals are quite abstract, and in fact it is impossible ever to get your hands on all of them (the ones in the future, for example). E XAMPLE 1.1.4. Suppose we’re interested in studying whether doing more homework helps students do better in their studies. So shouldn’t the individuals be the students? Well, which students? How about we look only at college students. Which college students? OK, how about students at 4-year colleges and universities in the United States, over the last five years – after all, things might be different in other countries and other historical periods. Wait, a particular student might sometimes do a lot of homework and sometimes do very little. And what exactly does “do better in their studies” mean? So maybe we should look at each student in each class they take, then we can look at the homework they did for that class and the success they had in it. Therefore, the individuals in this study would be individual experiences that students in US 4-year colleges and universities had in the last five years, and population of the study 5 6 1. ONE-VARIABLE STATISTICS: BASICS would essentially be the collection of all the names on all class rosters of courses in the last five years at all US 4-year colleges and universities. When doing an actual scientific study, we are usually not interested so much in the individuals themselves, but rather in D EFINITION 1.1.5. A variable in a statistical study is the answer of a question the researcher is asking about each individual. There are two types: • A categorical variable is one whose values have a finite number of possibilities. • A quantitative variable is one whose values are numbers (so, potentially an infi- nite number of possibilities). The variable is something which (as the name says) varies, in the sense that it can have a different value for each individual in the population (although that is not necessary). E XAMPLE 1.1.6. In Example 1.1.2, the variable most likely would be who they voted for, a categorical variable with only possible values “Mickey Mouse” or “Daffy Duck” (or whoever the names on the ballot were). E XAMPLE 1.1.7. In Example 1.1.3, the variable most likely would be what face of the coin was facing up after the flip, a categorical variable with values “heads” and “tails.” E XAMPLE 1.1.8. There are several variables we might use in Example 1.1.4. One might be how many homework problems did the student do in that course. Another could be how many hours total did the student spend doing homework over that whole semester, for that course. Both of those would be quantitative variables. A categorical variable for the same population would be what letter grade did the stu- dent get in the course, which has possible values A, A-, B+, . . . , D-, F. In many [most?] interesting studies, the population is too large for it to be practical to go observe the values of some interesting variable. Sometimes it is not just impractical, but actually impossible – think of the example we gave of all the flips of the coin, even in the ones in the future. So instead, we often work with D EFINITION 1.1.9. A sample is a subset of a population under study. Often we use the variable N to indicate the size of a whole population and the variable n for the size of a sample; as we have said, usually n < N. Later we shall discuss how to pick a good sample, and how much we can learn about a population from looking at the values of a variable of interest only for the individuals in a sample. For the rest of this chapter, however, let’s just consider what to do with these sample values. 1.2. VISUAL REPRESENTATION OF DATA, I 7 1.2. Visual Representation of Data, I: Categorical Variables Suppose we have a population and variable in which we are interested. We get a sample, which could be large or small, and look at the values of the our variable for the individuals in that sample. We shall informally refer to this collection of values as a dataset. In this section, we suppose also that the variable we are looking at is categorical. Then we can summarize the dataset by telling which categorical values did we see for the indi- viduals in the sample, and how often we saw those values. There are two ways we can make pictures of this information: bar charts and pie charts. 1.2.1. Bar Charts I: Frequency Charts. We can take the values which we saw for individuals in the sample along the x-axis of a graph, and over each such label make a box whose height indicates how many individuals had that value – the frequency of occurrence of that value. This is called a bar chart. As with all graphs, you should always label all axes. The x-axis will be labeled with some description of the variable in question, the y-axis label will always be “frequency” (or some synonym like “count” or “number of times”). E XAMPLE 1.2.1. In Example 1.1.7, suppose we took a sample of consisting of the next 10 flips of our coin. Suppose further that 4 of the flips came up heads – write it as “H” – and 6 came up tails, T. Then the corresponding bar chart would look like 1.2.2. Bar Charts II: Relative Frequency Charts. There is a variant of the above kind of bar chart which actually looks nearly the same but changes the labels on the y-axis. That is, instead of making the height of each bar be how many times each categorical value occurred, we could make it be what fraction of the sample had that categorical value – the relative frequency. This fraction is often displayed as a percentage. 8 1. ONE-VARIABLE STATISTICS: BASICS E XAMPLE 1.2.2. The relative frequency version of the above bar chart in Example 1.2.1 would look like 1.2.3. Bar Charts III: Cautions. Notice that with bar charts (of either frequency or relative frequency) the variable values along the x-axis can appear in any order whatso- ever. This means that any conclusion you draw from looking at the bar chart must not depend upon that order. For example, it would be foolish to say that the graph in the above Example 1.2.1 “shows and increasing trend,” since it would make just as much sense to put the bars in the other order and then “show a decreasing trend” – both are meaningless. For relative frequency bar charts, in particular, note that the total of the heights of all the bars must be 1 (or 100%). If it is more, something is weird; if it is less, some data has been lost. Finally, it makes sense for both kinds of bar charts for the y-axis to run from the logical minimum to maximum. For frequency charts, this means it should go from 0 to n (the sample size). For relative frequency charts, it should go from 0 to 1 (or 100%). Skimping on how much of this appropriate y-axis is used is a common trick to lie with statistics. E XAMPLE 1.2.3. The coin we looked at in Example 1.2.1 and Example 1.2.2 could well be a fair coin – it didn’t show exactly half heads and half tails, but it was pretty close. Someone who was trying to claim, deceptively, that the coin was not fair might have shown only a portion of the y axis, as 1.2. VISUAL REPRESENTATION OF DATA, I 9 This is actually, in a strictly technical sense, a correct graph. But, looking at it, one might conclude that T seems to occur more than twice as often as H, so the coin is probably not fair ... until a careful examination of the y-axis shows that even though the bar for T is more than twice as high as the bar for H, that is an artifact of how much of the y-axis is being shown. In summary, bar charts actually don’t have all that much use in sophisticated statistics, but are extremely common in the popular press (and on web sites and so on). 1.2.4. Pie Charts. Another way to make a picture with categorical data is to use the fractions from a relative frequency bar chart, but not for the heights of bars, instead for the sizes of wedges of a pie. E XAMPLE 1.2.4. Here’s a pie chart with the relative frequency data from Example 1.2.2. 10 1. ONE-VARIABLE STATISTICS: BASICS Pie charts are widely used, but actually they are almost never a good choice. In fact, do an Internet search for the phrase “pie charts are bad” and there will be nearly 3000 hits. Many of the arguments are quite insightful. When you see a pie chart, it is either an attempt (misguided, though) by someone to be folksy and friendly, or it is a sign that the author is quite unsophisticated with data visualization, or, worst of all, it might be a sign that the author is trying to use mathematical methods in a deceptive way. In addition, all of the cautions we mentioned above for bar charts of categorical data apply, mostly in exactly the same way, for pie charts. 1.3. VISUAL REPRESENTATION OF DATA, II 11 1.3. Visual Representation of Data, II: Quantitative Variables Now suppose we have a population and quantitative variable in which we are interested. We get a sample, which could be large or small, and look at the values of the our variable for the individuals in that sample. There are two ways we tend to make pictures of datasets like this: stem-and-leaf plots and histograms. 1.3.1. Stem-and-leaf Plots. One somewhat old-fashioned way to handle a modest amount of quantitative data produces something between simply a list of all the data val- ues and a graph. It’s not a bad technique to know about in case one has to write down a dataset by hand, but very tedious – and quite unnecessary, if one uses modern electronic tools instead – if the dataset has more than a couple dozen values. The easiest case of this technique is where the data are all whole numbers in the range 0 − 99. In that case, one can take off the tens place of each number – call it the stem – and put it on the left side of a vertical bar, and then line up all the ones places – each is a leaf – to the right of that stem. The whole thing is called a stem-and-leaf plot or, sometimes, just a stemplot. It’s important not to skip any stems which are in the middle of the dataset, even if there are no corresponding leaves. It is also a good idea to allow repeated leaves, if there are repeated numbers in the dataset, so that the length of the row of leaves will give a good representation of how much data is in that general group of data values. E XAMPLE 1.3.1. Here is a list of the scores of 30 students on a statistics test: 86 80 25 77 73 76 88 90 69 93 90 83 70 73 73 70 90 83 71 95 40 58 68 69 100 78 87 25 92 74 As we said, using the tens place (and the hundreds place as well, for the data value 100) as the stem and the ones place as the leaf, we get TABLE 1.3.1.1. Stem-and-leaf plot of students’ scores, Key: 1|7 = 17 Stem Leaf 10 0 9 000235 8 033678 7 0013334678 6 899 5 8 4 0 3 2 55 12 1. ONE-VARIABLE STATISTICS: BASICS One nice feature stem-and-leaf plots have is that they contain all of the data values, they do not lose anything (unlike our next visualization method, for example). 1.3.2. [Frequency] Histograms. The most important visual representation of quanti- tative data is a histogram. Histograms actually look a lot like a stem-and-leaf plot, except turned on its side and with the row of numbers turned into a vertical bar, like a bar graph. The height of each of these bars would be how many Another way of saying that is that we would be making bars whose heights were deter- mined by how many scores were in each group of ten. Note there is still a question of into which bar a value right on the edge would count: e.g., does the data value 50 count in the bar to the left of that number, or the bar to the right? It doesn’t actually matter which side, but it is important to state which choice is being made. E XAMPLE 1.3.2. Continuing with the score data in Example 1.3.1 and putting all data values x satisfying 20 ≤ x < 30 in the first bar, values x satisfying 30 ≤ x < 40 in the second, values x satisfying 40 ≤ x < 50 in the second, etc. – that is, put data values on the edges in the bar to the right – we get the figure Actually, there is no reason that the bars always have to be ten units wide: it is important that they are all the same size and that how they handle the edge cases (whether the left or right bar gets a data value on edge), but they could be any size. We call the successive ranges of the x coordinates which get put together for each bar the called bins or classes, and it is up to the statistician to chose whichever bins – where they start and how wide they are – shows the data best. Typically, the smaller the bin size, the more variation (precision) can be seen in the bars ... but sometimes there is so much variation that the result seems to have a lot of random jumps up and down, like static on the radio. On the other hand, using a large bin size makes 1.3. VISUAL REPRESENTATION OF DATA, II 13 the picture smoother ... but sometimes, it is so smooth that very little information is left. Some of this is shown in the following E XAMPLE 1.3.3. Continuing with the score data in Example 1.3.1 and now using the bins with x satisfying 10 ≤ x < 12, then 12 ≤ x < 14, etc., we get the histogram with bins of width 2: If we use the bins with x satisfying 10 ≤ x < 15, then 15 ≤ x < 20, etc., we get the histogram with bins of width 5: If we use the bins with x satisfying 20 ≤ x < 40, then 40 ≤ x < 60, etc., we get the histogram with bins of width 20: 14 1. ONE-VARIABLE STATISTICS: BASICS Finally, if we use the bins with x satisfying 0 ≤ x < 50, then 50 ≤ x < 100, and then 100 ≤ x < 150, we get the histogram with bins of width 50: 1.3.3. [Relative Frequency] Histograms. Just as we could have bar charts with abso- lute (§1.2.1) or relative (§1.2.2) frequencies, we can do the same for histograms. Above, in §1.3.2, we made absolute frequency histograms. If, instead, we divide each of the counts used to determine the heights of the bars by the total sample size, we will get fractions or percents – relative frequencies. We should then change the label on the y-axis and the tick-marks numbers on the y-axis, but otherwise the graph will look exactly the same (as it did with relative frequency bar charts compared with absolute frequency bar chars). E XAMPLE 1.3.4. Let’s make the relative frequency histogram corresponding to the absolute frequency histogram in Example 1.3.2, based on the data from Example 1.3.1 – all we have to do is change the numbers used to make heights of the bars in the graph by 1.3. VISUAL REPRESENTATION OF DATA, II 15 dividing them by the sample size, 30, and then also change the y-axis label and tick mark numbers. 1.3.4. How to Talk About Histograms. Histograms of course tell us what the data values are – the location along the x value of a bar is the value of the variable – and how many of them have each particular value – the height of the bar tells how many data values are in that bin. This is also given a technical name D EFINITION 1.3.5. Given a variable defined on a population, or at least on a sample, the distribution of that variable is a list of all the values the variable actually takes on and how many times it takes on these values. The reason we like the visual version of a distribution, its histogram, is that our visual intuition can then help us answer general, qualitative questions about what those data must be telling us. The first questions we usually want to answer quickly about the data are • What is the shape of the histogram? • Where is its center? • How much variability [also called spread] does it show? When we talk about the general shape of a histogram, we often use the terms D EFINITION 1.3.6. A histogram is symmetric if the left half is (approximately) the mirror image of the right half. We say a histogram is skewed left if the tail on the left side is longer than on the right. In other words, left skew is when the left half of the histogram – half in the sense that the total of the bars in this left part is half of the size of the dataset – extends farther to the left than the right does to the right. Conversely, the histogram is skewed right if the right half extends farther to the right than the left does to the left. 16 1. ONE-VARIABLE STATISTICS: BASICS If the shape of the histogram has one significant peak, then we say it is unimodal, while if it has several such, we say it is multimodal. It is often easy to point to where the center of a distribution looks like it lies, but it is hard to be precise. It is particularly difficult if the histogram is “noisy,” maybe multimodal. Similarly, looking at a histogram, it is often easy to say it is “quite spread out” or “very concentrated in the center,” but it is then hard to go beyond this general sense. Precision in our discussion of the center and spread of a dataset will only be possible in the next section, when we work with numerical measures of these features. 1.4. NUMERICAL DESCRIPTIONS OF DATA, I 17 1.4. Numerical Descriptions of Data, I: Measures of the Center Oddly enough, there are several measures of central tendency, as ways to define the middle of a dataset are called. There is different work to be done to calculate each of them, and they have different uses, strengths, and weaknesses. For this whole section we will assume we have collected n numerical values, the values of our quantitative variable for the sample we were able to study. When we write formulæ with these values, we can’t give them variable names that look like a, b, c, . . . , because we don’t know where to stop (and what would we do if n were more than 26?). Instead, we’ll use the variables x1 , x2 , . . . , xn to represent the data values. One more very convenient bit of notation, once we have started writing an unknown number (n) of numbers x1 , x2 , . . . , xn , is a way of writing their sum: D EFINITION 1.4.1. If we have n numbers which we write x1 , . . . , xn , then we use the P P shorthand summation notation xi to represent the sum xi = x1 + · · · + xn . 1 E XAMPLE 1.4.2. If our dataset were {1, 2, 17, −3.1415, 3/4}, then n would be 5 and the variables x1 , . . . , x5 would be defined with values x1 = 1, x2 = 2, x3 = 17, x4 = −3.1415, and x5 = 3/4. P In addition2, we would have xi = x1 +x2 +x3 +x4 +x5 = 1+2+17−3.1415+3/4 = 17.6085. 1.4.1. Mode. Let’s first discuss probably the simplest measure of central tendency, and in fact one which was foreshadowed by terms like “unimodal.” D EFINITION 1.4.3. A mode of a dataset x1 , . . . , xn of n numbers is one of the values xi which occurs at least as often in the dataset as any other value. It would be nice to say this in a simpler way, something like “the mode is the value which occurs the most often in the dataset,” but there may not be a single such number. E XAMPLE 1.4.4. Continuing with the data from Example 1.3.1, it is easy to see, look- ing at the stem-and-leaf plot, that both 73 and 90 are modes. Note that in some of the histograms we made using these data and different bin widths, the bins containing 73 and 90 were of the same height, while in others they were of different heights. This is an example of how it can be quite hard to see on a histogram where the mode is... or where the modes are. 1 Pn Pn Sometimes you will see this written instead i=1 xi . Think of the “ i=1 ” as a little computer program which with i = 1, increases it one step at a time until it gets all the way to i = n, and adds up whatever is to P3 the right. So, for example, i=1 2i would be (2 ∗ 1) + (2 ∗ 2) + (2 ∗ 3), and so has the value 12. 2no pun intended 18 1. ONE-VARIABLE STATISTICS: BASICS 1.4.2. Mean. The next measure of central tendency, and certainly the one heard most often in the press, is simply the average. However, in statistics, this is given a different name. D EFINITION 1.4.5. The mean of a dataset x1 , . . . , xn of n numbers is given by the P formula ( xi ) /n. If the data come from a sample, we use the notation x for the sample mean. If {x1 , . . . , xn } is all of the data from an entire population, we use the notation µX [this is the Greek letter “mu,” pronounced “mew,” to rhyme with “new.”] for the population mean. E XAMPLE 1.4.6. Since we’ve already computed the sum of the data in Example 1.4.2 to be 17.6085 and there were 5 values in the dataset, the mean is x = 17.6085/5 = 3.5217. E XAMPLE 1.4.7. Again using the data from Example 1.3.1, we can calculate the mean P x = ( xi ) /n = 2246/30 = 74.8667. Notice that the mean in the two examples above was not one of the data values. This is true quite often. What that means is that the phrase “the average whatever,” as in “the average American family has X” or “the average student does Y ,” is not talking about any particular family, and we should not expect any particular family or student to have or do that thing. Someone with a statistical education should mentally edit every phrase like that they hear to be instead something like “the mean of the variable X on the population of all American families is ...,” or “the mean of the variable Y on the population of all students is ...,” or whatever. 1.4.3. Median. Our third measure of central tendency is not the result of arithmetic, but instead of putting the data values in increasing order. D EFINITION 1.4.8. Imagine that we have put the values of a dataset {x1 , . . . , xn } of n numbers in increasing (or at least non-decreasing) order, so that x1 ≤ x2 ≤ · · · ≤ xn . Then if n is odd, the median of the dataset is the middle value, x(n+1)/2 , while if n is even, the x +x median is the mean of the two middle numbers, n/2 2(n/2)+1 . E XAMPLE 1.4.9. Working with the data in Example 1.4.2, we must first put them in order, as {−3.1415, 3/4, 1, 2, 17}, so the median of this dataset is the middle value, 1. E XAMPLE 1.4.10. Now let us find the median of the data from Example 1.3.1. For- tunately, in that example, we made a stem-and-leaf plot and even put the leaves in order, so that starting at the bottom and going along the rows of leaves and then up to the next row, will give us all the values in order! Since there are 30 values, we count up to the 15th and 16th values, being 76 and 77, and from this we find that the median of the dataset is 76+77 2 = 76.5. 1.4. NUMERICAL DESCRIPTIONS OF DATA, I 19 1.4.4. Strengths and Weaknesses of These Measures of Central Tendency. The weakest of the three measures above is the mode. Yes, it is nice to know which value happened most often in a dataset (or which values all happened equally often and more of- ten then all other values). But this often does not necessarily tell us much about the over-all structure of the data. E XAMPLE 1.4.11. Suppose we had the data 86 80 25 77 73 76 100 90 67 93 94 83 72 75 79 70 91 82 71 95 40 58 68 69 100 78 87 25 92 74 with corresponding stem-and-leaf plot Stem Leaf 10 0 9 012345 8 023678 7 0123456789 6 789 5 8 4 0 3 2 55 This would have a histogram with bins of width 10 that looks exactly like the one in Ex- ample 1.3.2 – so the center of the histogram would seem, visually, still to be around the bar over the 80s – but now there is a unique mode of 25. What this example shows is that a small change in some of the data values, small enough not to change the histogram at all, can change the mode(s) drastically. It also shows that the location of the mode says very little about the data in general or its shape, the mode is based entirely on a possibly accidental coincidence of some values in the dataset, no matter if those values are in the “center” of the histogram or not. The mean has a similar problem: a small change in the data, in the sense of adding only one new data value, but one which is very far away from the others, can change the mean quite a bit. Here is an example. E XAMPLE 1.4.12. Suppose we take the data from Example 1.3.1 but change only one value – such as by changing the 100 to a 1000, perhaps by a simple typo of the data entry. P Then if we calculate the mean, we get x = ( xi ) /n = 3146/30 = 104.8667, which is quite different from the mean of original dataset. 20 1. ONE-VARIABLE STATISTICS: BASICS A data value which seems to be quite different from all (or the great majority of) the rest is called an outlier3 What we have just seen is that the mean is very sensitive to outliers. This is a serious defect, although otherwise it is easy to compute, to work with, and to prove theorems about. Finally, the median is somewhat tedious to compute, because the first step is to put all the data values in order, which can be very time-consuming. But, once that is done, throwing in an outlier tends to move the median only a little bit. Here is an example. E XAMPLE 1.4.13. If we do as in Example 1.4.12 and change the data value of 100 in the dataset of Example 1.3.1 to 1000, but leave all of the other data values unchanged, it does not change the median at all since the 1000 is the new largest value, and that does not change the two middle values at all. If instead we take the data of Example 1.3.1 and simply add another value, 1000, with- out taking away the 100, that does change the media: there are now an odd number of data values, so the median is the middle one after they are put in order, which is 78. So the median has changed by only half a point, from 77.5 to 78. And his would even be true if the value we were adding to the dataset were 1000000 and not just 1000! In other words, the median is very insensitive to outliers. Since, in practice, it is very easy for datasets to have a few random, bad values (typos, mechanical errors, etc.), which are often outliers, it is usually smarter to use the median than the mean. As one final point, note that as we mentioned in §1.4.2, the word “average,” the unso- phisticated version of “mean,” is often incorrectly used as a modifier of the individuals in some population being studied (as in “the average American ...”), rather than as a modifier of the variable in the study (“the average income...”), indicating a fundamental misunder- standing of what the mean means. If you look a little harder at this misunderstanding, though, perhaps it is based on the idea that we are looking for the center, the “typical” value of the variable. The mode might seem like a good way – it’s the most frequently occurring value. But we have seen how that is somewhat flawed. The mean might also seem like a good way – it’s the “average,” literally. But we’ve also seen problems with the mean. In fact, the median is probably closest to the intuitive idea of “the center of the data.” It is, after all, a value with the property that both above and below that value lie half of the data values. One last example to underline this idea: E XAMPLE 1.4.14. The period of economic difficulty for world markets in the late 2000s and early 2010s is sometimes called the Great Recession. Suppose a politician says that 3This is a very informal definition of an outlier. Below we will have an extremely precise one. 1.4. NUMERICAL DESCRIPTIONS OF DATA, I 21 we have come out of that time of troubles, and gives as proof the fact that the average family income has increased from the low value it had during the Great Recession back to the values it had before then, and perhaps is even higher than it was in 2005. It is possible that in fact people are better off, as the increase in this average – mean – seems to imply. But it is also possible that while the mean income has gone up, the median income is still low. This would happen if the histogram of incomes recently still has most of the tall bars down where the variable (family income) is low, but has a few, very high outliers. In short, if the super-rich have gotten even super-richer, that will make the mean (average) go up, even if most of the population has experienced stagnant or decreasing wages – but the median will tell what is happening to most of the population. So when a politician uses the evidence of the average (mean) as suggested here, it is possible they are trying to hide from the pubic the reality of what is happening to the rich and the not-so-rich. It is also possible that this politician is simply poorly educated in statistics and doesn’t realize what is going on. You be the judge ... but pay attention so you know what to ask about. The last thing we need to say about the strengths and weaknesses of our different mea- sures of central tendency is a way to use the weaknesses of the mean and median to our advantage. That is, since the mean is sensitive to outliers, and pulled in the direction of those outliers, while the median is not, we can use the difference between the two to tell us which way a histogram is skewed. FACT 1.4.15. If the mean of a dataset is larger than the median, then histograms of that dataset will be right-skewed. Similarly, if the mean is less than the median, histograms will be left-skewed. 22 1. ONE-VARIABLE STATISTICS: BASICS 1.5. Numerical Descriptions of Data, II: Measures of Spread 1.5.1. Range. The simplest – and least useful – measure of the spread of some data is literally how much space on the x-axis the histogram takes up. To define this, first a bit of convenient notation: D EFINITION 1.5.1. Suppose x1 , . . . , xn is some quantitative dataset. We shall write xmin for the smallest and xmax for the largest values in the dataset. With this, we can define our first measure of spread D EFINITION 1.5.2. Suppose x1 , . . . , xn is some quantitative dataset. The range of this data is the number xmax − xmin . E XAMPLE 1.5.3. Using again the statistics test scores data from Example 1.3.1, we can read off from the stem-and-leaf plot that xmin = 25 and xmax = 100, so the range is 75(= 100 − 25). E XAMPLE 1.5.4. Working now with the made-up data in Example 1.4.2, which was put into increasing order in Example 1.4.9, we can see that xmin = −3.1415 and xmax = 17, so the range is 20.1415(= 17 − (−3.1415)). The thing to notice here is that since the idea of outliers is that they are outside of the normal behavior of the dataset, if there are any outliers they will definitely be what value gets called xmin or xmax (or both). So the range is supremely sensitive to outliers: if there are any outliers, the range will be determined exactly by them, and not by what the typical data is doing. 1.5.2. Quartiles and the IQR. Let’s try to find a substitute for the range which is not so sensitive to outliers. We want to see how far apart not the maximum and minimum of the whole dataset are, but instead how far apart are the typical larger values in the dataset and the typical smaller values. How can we measure these typical larger and smaller? One way is to define these in terms of the typical – central – value of the upper half of the data and the typical value of the lower half of the data. Here is the definition we shall use for that concept: D EFINITION 1.5.5. Imagine that we have put the values of a dataset {x1 , . . . , xn } of n numbers in increasing (or at least non-decreasing) order, so that x1 ≤ x2 ≤ · · · ≤ xn . If n is odd, call the lower half data all the values {x1 , . . . , x(n−1)/2 } and the upper half data all the values {x(n+3)/2 , . . . , xn }; if n is even, the lower half data will be the values {x1 , . . . , xn/2 } and the upper half data all the values {x(n/2)+1 , . . . , xn }. Then the first quartile, written Q1 , is the median of the lower half data, and the third quartile, written Q3 , is the median of the upper half data. 1.5. NUMERICAL DESCRIPTIONS OF DATA, II 23 Note that the first quartile is halfway through the lower half of the data. In other words, it is a value such that one quarter of the data is smaller. Similarly, the third quartile is halfway through the upper half of the data, so it is a value such that three quarters of the data is small. Hence the names “first” and “third quartiles.” We can build a outlier-insensitive measure of spread out of the quartiles. D EFINITION 1.5.6. Given a quantitative dataset, its inter-quartile range or IQR is defined by IQR = Q3 − Q1 . E XAMPLE 1.5.7. Yet again working with the statistics test scores data from Exam- ple 1.3.1, we can count off the lower and upper half datasets from the stem-and-leaf plot, being respectively Lower = {25, 25, 40, 58, 68, 69, 69, 70, 70, 71, 73, 73, 73, 74, 76} and Upper = {77, 78, 80, 83, 83, 86, 87, 88, 90, 90, 90, 92, 93, 95, 100} . It follows that, for these data, Q1 = 70 and Q3 = 88, so IQR = 18(= 88 − 70). E XAMPLE 1.5.8. Working again with the made-up data in Example 1.4.2, which was put into increasing order in Example 1.4.9, we can see that the lower half data is {−3.1415, .75}, the upper half is {2, 17}, Q1 = −1.19575(= −3.1415+.75 2 ), Q3 = 9.5(= 2+172 ), and IQR = 10.69575(= 9.5 − (−1.19575)). 1.5.3. Variance and Standard Deviation. We’ve seen a crude measure of spread, like the crude measure “mode” of central tendency. We’ve also seen a better measure of spread, the IQR, which is insensitive to outliers like the median (and built out of medians). It seems that, to fill out the parallel triple of measures, there should be a measure of spread which is similar to the mean. Let’s try to build one. Suppose the data is sample data. Then how far a particular data value xi is from the sample mean x is just xi − x. So the mean displacement from the mean, the mean of xi − x, should be a good measure of variability, shouldn’t it? Unfortunately, it turns out that the mean of xi − x is always 0. This is because when xi > x, xi − x is positive, while when xi < x, xi − x is negative, and it turns out that the positives always exactly cancel the negatives (see if you can prove this algebraically, it’s not hard). We therefore need to make the numbers xi − x positive before taking their mean. One way to do this is to square them all. Then we take something which is almost the mean of these squared numbers to get another measure of spread or variability: D EFINITION 1.5.9. Given sample data x1 , . . . , xn from a sample of size n, the sample variance is defined as P 2 (xi − x)2 Sx = . n−1 24 1. ONE-VARIABLE STATISTICS: BASICS Out of this, we then define the sample standard deviation s P p 2 (xi − x)2 Sx = Sx = . n−1 Why do we take the square root in that sample standard deviation? The answer is that the measure we build should have the property that if all the numbers are made twice as big, then the measure of spread should also be twice as big. Or, for example, if we first started working with data measured in feet and then at some point decided to work in inches, the numbers would all be 12 times as big, and it would make sense if the measure of spread were also 12 times as big. The variance does not have this property: if the data are all doubled, the variance in- creases by a factor of 4. Or if the data are all multiplied by 12, the variance is multiplied by a factor of 144. If we take the square root of the variance, though, we get back to the nice property of doubling data doubles the measure of spread, etc. For this reason, while we have defined the variance on its own and some calculators, computers, and on-line tools will tell the variance whenever you ask them to computer 1-variable statistics, we will in this class only consider the variance a stepping stone on the way to the real measure of spread of data, the standard deviation. One last thing we should define in this section. For technical reasons that we shall not go into now, the definition of standard deviation is slightly different if we are working with population data and not sample data: D EFINITION 1.5.10. Given data x1 , . . . , xn from an entire population of size n, the population variance is defined as P 2 (xi − µX )2 σX = . n Out of this, we then define the population standard deviation s p P 2 (xi − µX )2 σX = σX = . n [This letter σ is the lower-case Greek letter sigma, whose upper case Σ you’ve seen elsewhere.] Now for some examples. Notice that to calculate these values, we shall always use an electronic tool like a calculator or a spreadsheet that has a built-in variance and standard deviation program – experience shows that it is nearly impossible to get all the calculations entered correctly into a non-statistical calculator, so we shall not even try. 1.5. NUMERICAL DESCRIPTIONS OF DATA, II 25 E XAMPLE 1.5.11. For the statistics test scores data from Example 1.3.1, entering them into a spreadsheet and using VAR.S and STDEV.S for the sample variance and standard deviation and VAR.P and STDEV.P for population variance and population standard de- viation, we get Sx2 = 331.98 Sx = 18.22 σX 2 = 330.92 σX = 17.91 E XAMPLE 1.5.12. Similarly, for the data in Example 1.4.2, we find in the same way that Sx2 = 60.60 Sx = 7.78 σX 2 = 48.48 σX = 6.96 1.5.4. Strengths and Weaknesses of These Measures of Spread. We have already said that the range is extremely sensitive to outliers. The IQR, however, is built up out of medians, used in different ways, so the IQR is insensitive to outliers. The variance, both sample and population, is built using a process quite like a mean, and in fact also has the mean itself in the defining formula. Since the standard deviation in both cases is simply the square root of the variance, it follows that the sample and population variances and standard deviations are all sensitive to outliers. This differing sensitivity and insensitivity to outliers is the main difference between the different measures of spread that we have discussed in this section. One other weakness, in a certain sense, of the IQR is that there are several differ- ent definitions in use of the quartiles, based upon whether the median value is included or not when dividing up the data. These are called, for example, QUARTILE.INC and QUARTILE.EXC on some spreadsheets. It can then be confusing which one to use. 1.5.5. A Formal Definition of Outliers – the 1.5 IQR Rule. So far, we have said that outliers are simply data that are atypical. We need a precise definition that can be carefully checked. What we will use is a formula (well, actually two formulæ) that describe that idea of an outlier being far away from the rest of data. Actually, since outliers should be far away either in being significantly bigger than the rest of the data or in being significantly smaller, we should take a value on the upper side of the rest of the data, and another on the lower side, as the starting points for this far away. 26 1. ONE-VARIABLE STATISTICS: BASICS We can’t pick the xmax and xmin as those starting points, since they will be the outliers themselves, as we have noticed. So we will use our earlier idea of a value which is typical for the larger part of the data, the quartile Q3 , and Q1 for the corresponding lower part of the data. Now we need to decide how far is far enough away from those quartiles to count as an outlier. If the data already has a lot of variation, then a new data value would have to be quite far in order for us to be sure that it is not out there just because of the variation already in the data. So our measure of far enough should be in terms of a measure of spread of the data. Looking at the last section, we see that only the IQR is a measure of spread which is insensitive to outliers – and we definitely don’t want to use a measure which is sensitive to the outliers, one which would have been affected by the very outliers we are trying to define. All this goes together in the following D EFINITION 1.5.13. [The 1.5 IQR Rule for Outliers] Starting with a quantitative dataset whose first and third quartiles are Q1 and Q3 and whose inter-quartile range is IQR, a data value x is [officially, from now on] called an outlier if x < Q1 − 1.5 IQR or x > Q3 + 1.5 IQR. Notice this means that x is not an outlier if it satisfies Q1 − 1.5 IQR ≤ x ≤ Q3 + 1.5 IQR. E XAMPLE 1.5.14. Let’s see if there were any outliers in the test score dataset from Example 1.3.1. We found the quartiles and IQR in Example 1.5.7, so from the 1.5 IQR Rule, a data value x will be an outlier if x < Q1 − 1.5 IQR = 70 − 1.5 · 18 = 43 or if x > Q3 + 1.5 IQR = 88 + 1.5 · 18 = 115 . Looking at the stemplot in Table 1.3.1, we conclude that the data values 25, 25, and 40 are the outliers in this dataset. E XAMPLE 1.5.15. Applying the same method to the data in Example 1.4.2, using the quartiles and IQR from Example 1.5.8, the condition for an outlier x is x < Q1 − 1.5 IQR = −1.19575 − 1.5 · 10.69575 = −17.239375 or x > Q3 + 1.5 IQR = 9.5 + 1.5 · 10.69575 = 25.543625 . Since none of the data values satisfy either of these conditions, there are no outliers in this dataset. 1.5. NUMERICAL DESCRIPTIONS OF DATA, II 27 1.5.6. The Five-Number Summary and Boxplots. We have seen that numerical sum- maries of quantitative data can be very useful for quickly understanding (some things about) the data. It is therefore convenient for a nice package of several of these D EFINITION 1.5.16. Given a quantitative dataset {x1 , . . . , xn }, the five-number sum- mary4 of this data is the set of values {xmin , Q1 , median, Q3 , xmax } E XAMPLE 1.5.17. Why not write down the five-number summary for the same test score data we saw in Example 1.3.1? We’ve already done most of the work, such as calcu- lating the min and max in Example 1.5.3, the quartiles in Example 1.5.7, and the median in Example 1.4.10, so the five-number summary is xmin = 25 Q1 = 70 median = 76.5 Q3 = 88 xmax = 100 E XAMPLE 1.5.18. And, for completeness, the five number summary for the made-up data in Example 1.4.2 is xmin = −3.1415 Q1 = −1.9575 median = 1 Q3 = 9.5 xmax = 17 where we got the min and max from Example 1.5.4, the median from Example 1.4.9, and the quartiles from Example 1.5.8. As we have seen already several times, it is nice to have a both a numeric and a graph- ical/visual version of everything. The graphical equivalent of the five-number summary is D EFINITION 1.5.19. Given some quantitative data, a boxplot [sometimes box-and- whisker plot] is a graphical depiction of the five-number summary, as follows: 4Which might write 5NΣary for short. 28 1. ONE-VARIABLE STATISTICS: BASICS • an axis is drawn, labelled with the variable of the study • tick marks and numbers are put on the axis, enough to allow the following visual features to be located numerically • a rectangle (the box) is drawn parallel to the axis, stretching from values Q1 to Q3 on the axis • an addition line is drawn, parallel to the sides of the box at locations xmin and xmax , at the axis coordinate of the median of the data • lines are drawn parallel to the axis from the middle of sides of the box at the locations xmin and xmax out to the axis coordinates xmin and xmax , where these whiskers terminate in “T”s. E XAMPLE 1.5.20. A boxplot for the test score data we started using in Example 1.3.1 is easy to make after we found the corresponding five-number summary in Example 1.5.17: Sometimes it is nice to make a version of the boxplot which is less sensitive to outliers. Since the endpoints of the whiskers are the only parts of the boxplot which are sensitive in this way, they are all we have to change: D EFINITION 1.5.21. Given some quantitative data, a boxplot showing outliers [some- times box-and-whisker plot showing outliers] is minor modification of the regular box- plot, as follows • the whiskers only extend as far as the largest and smallest non-outlier data values • dots are put along the lines of the whiskers at the axis coordinates of any outliers in the dataset E XAMPLE 1.5.22. A boxplot showing outliers for the test score data we started using in Example 1.3.1 is only a small modification of the one we just made in Example 1.5.20 1.5. NUMERICAL DESCRIPTIONS OF DATA, II 29 30 1. ONE-VARIABLE STATISTICS: BASICS Exercises E XERCISE 1.1. A product development manager at the campus bookstore wants to make sure that the backpacks being sold there are strong enough to carry the heavy books students carry around campus. The manager decides she will collect some data on how heavy are the bags/packs/suitcases students are carrying around at the moment, by stopping the next 100 people she meets at the center of campus and measuring. What are the individuals in this study? What is the population? Is there a sample – what is it? What is the variable? What kind of variable is this? E XERCISE 1.2. During a blood drive on campus, 300 donated blood. Of these, 136 had blood of type O, 120 had blood of type A, 32 of type B, and the rest of type AB. Answer the same questions as in the previous exercise for this new situation. Now make at least two visual representations of these data. E XERCISE 1.3. Go to the Wikipedia page for “Heights of Presidents and Presidential Candidates of the United States” and look only at the heights of the presidents themselves, in centimeters (cm). Make a histogram with these data using bins of width 5. Explain how you are handling the edge cases in your histogram. E XERCISE 1.4. Suppose you go to the supermarket every week for a year and buy a bag of flour, packaged by a major national flour brand, which is labelled as weighing 1kg. You take the bag home and weigh it on an extremely accurate scale that measures to the nearest 1/100th of a gram. After the 52 weeks of the year of flour buying, you make a histogram of the accurate weights of the bags. What do you think that histogram will look like? Will it be symmetric or skewed left or right (which one?), where will its center be, will it show a lot of variation/spread or only a little? Explain why you think each of the things you say. What about if you buy a 1kg loaf of bread from the local artisanal bakery – what would the histogram of the accurate weights of those loaves look like (same questions as for histogram of weights of the bags of flour)? If you said that those histograms were symmetric, can you think of a measurement you would make in a grocery store or bakery which would be skewed; and if you said the histograms for flour and loaf weights were skewed, can you think of one which would be symmetric? (Explain why, always, of course.) [If you think one of the two above histograms was skewed and one was symmetric (with explanation), you don’t need to come up with another one here.] EXERCISES 31 E XERCISE 1.5. Twenty sacks of grain weigh a total of 1003kg. What is the mean weight per sack? Can you determine the median weight per sack from the given information? If so, explain how. If not, give two examples of datasets with the same total weight be different medians. E XERCISE 1.6. For the dataset {6, −2, 6, 14, −3, 0, 1, 4, 3, 2, 5}, which we will call DS1 , find the mode(s), mean, and median. Define DS2 by adding 3 to each number in DS1 . What are the mode(s), mean, and median of DS2 ? Now define DS3 by subtracting 6 from each number in DS1 . What are the mode(s), mean, and median of DS3 ? Next, define DS4 by multiplying every number in DS1 by 2. What are the mode(s), mean, and median of DS4 ? Looking at your answers to the above calculations, how do you think the mode(s), mean, and median of datasets must change when you add, subtract, multiply or divide all the numbers by the same constant? Make a specific conjecture! E XERCISE 1.7. There is a very hard mathematics competition in which college students in the US and Canada can participate called the William Lowell Putnam Mathematical Competition. It consists of a six-hour long test with twelve problems, graded 0 to 10 on each problem, so the total score could be anything from 0 to 120. The median score last year on the Putnam exam was 0 (as it often is, actually). What does this tell you about the scores of the students who took it? Be as precise as you can. Can you tell what fraction (percentage) of students had a certain score or scores? Can you figure out what the quartiles must be? E XERCISE 1.8. Find the range, IQR, and standard deviation of the following sample dataset: DS1 = {0, 0, 0, 0, 0, .5, 1, 1, 1, 1, 1} . Now find the range, IQR, and standard deviation of the following sample data: DS2 = {0, .5, 1, 1, 1, 1, 1, 1, 1, 1, 1} . Next find the range, IQR, and standard deviation of the following sample data: DS3 = {0, 0, 0, 0, 0, 0, 0, 0, 0, .5, 1} . Finally, find the range, IQR, and standard deviation of sample data DS4 , consisting of 98 0s, one .5, and one 1 (so like DS3 except with 0 occurring 98 times instead of 9 time). 32 1. ONE-VARIABLE STATISTICS: BASICS E XERCISE 1.9. What must be true about a dataset if its range is 0? Give the most interesting example of a dataset with range of 0 and the property you just described that you can think of. What must be true about a dataset if its IQR is 0? Give the most interesting example of a dataset with IQR of 0 and the property you just described that you can think of. What must be true about a dataset if its standard deviation is 0? Give the most interest- ing example of a dataset with standard deviation of 0 and the property you just described that you can think of. E XERCISE 1.10. Here are some boxplots of test scores, out of 100, on a standardized test given in five different classes – the same test, different classes. For each of these plots, A − E, describe qualitatively (in the sense of §1.3.4) but in as much detail as you can, what must have been the histogram for the data behind this boxplot. Also sketch a possible such histogram, for each case. CHAPTER 2 Bi-variate Statistics: Basics 2.1. Terminology: Explanatory/Response or Independent/Dependent All of the discussion so far has been for studies which have a single variable. We may collect the values of this variable for a large population, or at least the largest sample we can afford to examine, and we may display the resulting data in a variety of graphical ways, and summarize it in a variety of numerical ways. But in the end all this work can only show a single characteristic of the individuals. If, instead, we want to study a relationship, we need to collect two (at least) variables and develop methods of descriptive statistics which show the relationships between the values of these variables. Relationships in data require at least two variables. While more complex relationships can involve more, in this chapter we will start the project of understanding bivariate data, data where we make two observations for each individual, where we have exactly two variables. If there is a relationship between the two variables we are studying, the most that we could hope for would be that that relationship is due to the fact that one of the variables causes the other. In this situation, we have special names for these variables D EFINITION 2.1.1. In a situation with bivariate data, if one variable can take on any value without (significant) constraint it is called the independent variable, while the sec- ond variable, whose value is (at least partially) controlled by the first, is called the depen- dent variable. Since the value of the dependent variable depends upon the value of the independent variable, we could also say that it is explained by the independent variable. Therefore the independent variable is also called the explanatory variable and the dependent variable is then called the response variable Whenever we have bivariate data and we have made a choice of which variable will be the independent and which the dependent, we write x for the independent and y for the dependent variable. E XAMPLE 2.1.2. Suppose we have a large warehouse of many different boxes of prod- ucts ready to ship to clients. Perhaps we have packed all the products in boxes which are perfect cubes, because they are stronger and it is easier to stack them efficiently. We could do a study where • the individuals would be the boxes of product; 33 34 2. BI-VARIATE STATISTICS: BASICS • the population would be all the boxes in our warehouse; • the independent variable would be, for a particular box, the length of its side in cm; • the dependent variable would be, for a particular box, the cost to the customer of buying that item, in US dollars. We might think that the size determines the cost, at least approximately, because the larger boxes contain larger products into which went more raw materials and more labor, so the items would be more expensive. So, at least roughly, the size may be anything, it is a free or independent choice, while the cost is (approximately) determined by the size, so the cost is dependent. Otherwise said, the size explains and the cost is the response. Hence the choice of those variables. E XAMPLE 2.1.3. Suppose we have exactly the same scenario as above, but now we want to make the different choice where • the dependent variable would be, for a particular box, the volume of that box. There is one quite important difference between the two examples above: in one case (the cost), knowing the length of the side of a box give us a hint about how much it costs (bigger boxes cost more, smaller boxes cost less) but this knowledge is imperfect (some- times a big box is cheap, sometimes a small box is expensive); while in the other case (the volume), knowing the length of the side of the box perfectly tells us the volume. In fact, there is a simple geometric formula that the volume V of a cube of side length s is given by V = s3 . This motivates a last preliminary definition D EFINITION 2.1.4. We say that the relationship between two variables is deterministic if knowing the value of one variable completely determines the value of the other. If, instead, knowing one value does not completely determine the other, we say the variables have a non-deterministic relationship. 2.2. SCATTERPLOTS 35 2.2. Scatterplots When we have bivariate data, the first thing we should always do is draw a graph of this data, to get some feeling about what the data is showing us and what statistical methods it makes sense to try to use. The way to do this is as follows D EFINITION 2.2.1. Given bivariate quantitative data, we make the scatterplot of this data as follows: Draw an x- and a y-axis, and label them with descriptions of the indepen- dent and dependent variables, respectively. Then, for each individual in the dataset, put a dot on the graph at location (x, y), if x is the value of that individual’s independent variable and y the value of its dependent variable. After making a scatterplot, we usually describe it qualitatively in three respects: D EFINITION 2.2.2. If the cloud of data points in a scatterplot generally lies near some curve, we say that the scatterplot has [approximately] that shape. A common shape we tend to find in scatterplots is that it is linear If there is no visible shape, we say the scatterplot is amorphous, or has no clear shape. D EFINITION 2.2.3. When a scatterplot has some visible shape – so that we do not describe it as amorphous – how close the cloud of data points is to that curve is called the strength of that association. In this context, a strong [linear, e.g.,] association means that the dots are close to the named curve [line, e.g.,], while a weak association means that the points do not lie particularly close to any of the named curves [line, e.g.,]. D EFINITION 2.2.4. In case a scatterplot has a fairly strong linear association, the di- rection of the association described whether the line is increasing or decreasing. We say the association is positive if the line is increasing and negative if it is decreasing. [Note that the words positive and negative here can be thought of as describing the slope of the line which we are saying is the underlying relationship in the scatterplot.] 36 2. BI-VARIATE STATISTICS: BASICS 2.3. Correlation As before (in §§1.4 and 1.5), when we moved from describing histograms with words (like symmetric) to describing them with numbers (like the mean), we now will build a numeric measure of the strength and direction of a linear association in a scatterplot. D EFINITION 2.3.1. Given bivariate quantitative data {(x1 , y1 ), . . . , (xn , yn )} the [Pear- son] correlation coefficient of this dataset is 1 X (xi − x) (yi − y) r= n−1 sx sy where sx and sy are the standard deviations of the x and y, respectively, datasets by them- selves. We collect some basic information about the correlation coefficient in the following FACT 2.3.2. For any bivariate quantitative dataset {(x1 , y1), . . . , (xn , yn )} with corre- lation coefficient r, we have (1) −1 ≤ r ≤ 1 is always true; (2) if |r| is near 1 – meaning that r is near ±1 – then the linear association between x and y is strong (3) if r is near 0 – meaning that r is positive or negative, but near 0 – then the linear association between x and y is weak (4) if r > 0 then the linear association between x and y is positive, while if r < 0 then the linear association between x and y is negative (5) r is the same no matter what units are used for the variables x and y – meaning that if we change the units in either variable, r will not change (6) r is the same no matter which variable is begin used as the explanatory and which as the response variable – meaning that if we switch the roles of the x and the y in our dataset, r will not change. It is also nice to have some examples of correlation coefficients, such as 2.3. CORRELATION 37 Many electronic tools which compute the correlation coefficient r of a dataset also report its square, r 2 . There reason is explained in the following FACT 2.3.3. If r is the correlation coefficient between two variables x and y in some quantitative dataset, then its square r 2 it the fraction (often described as a percentage) of the variation of y which is associated with variation in x. E XAMPLE 2.3.4. If the square of the correlation coefficient between the independent variable how many hours a week a student studies statistics and the dependent variable how many points the student gets on the statistics final exam is .64, then 64% of the variation in scores for that class is cause by variation in how much the students study. The remaining 36% of the variation in scores is due to other random factors like whether a student was coming down with a cold on the day of the final, or happened to sleep poorly the night before the final because of neighbors having a party, or some other issues different just from studying time. 38 2. BI-VARIATE STATISTICS: BASICS Exercises E XERCISE 2.1. Suppose you pick 50 random adults across the United States in January 2017 and measure how tall they are. For each of them, you also get accurate information about how tall their (biological) parents are. Now, using as your individuals these 50 adults and as the two variables their heights and the average of their parents’ heights, make a sketch of what you think the resulting scatterplot would look like. Explain why you made the choice you did of one variable to be the explanatory and the other the response variable. Tell what are the shape, strength, and direction you see in this scatterplot, if it shows a deterministic or non-deterministic association, and why you think those conclusions would be true if you were to do this exercise with real data. Is there any time or place other than right now in the United States where you think the data you would collect as above would result in a scatterplot that would look fairly different in some significant way? Explain! E XERCISE 2.2. It actually turns out that it is not true that the more a person works, the more they produce ... at least not always. Data on workers in a wide variety of industries show that working more hours produces more of that business’s product for a while, but then after too many hours of work, keeping on working makes for almost no additional production. Describe how you might collect data to investigate this relationship, by telling what individuals, population, sample, and variables you would use. Then, assuming the truth of the above statement about what other research in this area has found, make an example of a scatterplot that you think might result from your suggested data collection. E XERCISE 2.3. Make a scatterplot of the dataset consisting of the following pairs of measurements: {(8, 16), (9, 9), (10, 4), (11, 1), (12, 0), (13, 1), (14, 4), (15, 9), (16, 16)} . You can do this quite easily by hand (there are only nine points!). Feel free to use an electronic device to make the plot for you, if you have one you know how to use, but copy the resulting picture into the homework you hand in, either by hand or cut-and-paste into an electronic version. Describe the scatterplot, telling what are the shape, strength, and direction. What do you think would be the correlation coefficient of this dataset? As always, explain all of your reasoning! CHAPTER 3 Linear Regression Quick review of equations for lines: Recall the equation of a line is usually in the form y = mx + b, where x and y are variables and m and b are numbers. Some basic facts about lines: • If you are given a number for x, you can plug it in to the equation y = mx + b to get a number for y, which together give you a point with coordinates (x, y) that is on the line. • m is the slope, which tells how much the line goes up (increasing y) for every unit you move over to the right (increasing x) – we often say that the value of the slope is m = rise run . It can be – positive, if the line is tilted up, – negative, if the line is tilted down, – zero, if the line is horizontal, and – undefined, if the line is vertical. • You can calculate the slope by finding the coordinates (x1 , y1 ) and (x2 , y2 ) of any two points on the line and then m = xy22 −y−x1 1 . y2 −y1 • In particular, x2 − x1 = 1, then m = 1 = y2 − y1 – so if you look at how much the line goes up in each step of one unit to the right, that number will be the slope m (and if it goes down, the slope m will simply be negative). In other words, the slope answers the question “for each step to the right, how much does the line increase (or decrease)?” • b is the y-intercept, which tells the y-coordinate of the point where the line crosses the y-axis. Another way of saying that is that b is the y value of the line when the x is 0. 3.1. The Least Squares Regression Line Suppose we have some bivariate quantitative data {(x1 , y1), . . . , (xn , yn )} for which the correlation coefficient indicates some linear association. It is natural to want to write down explicitly the equation of the best line through the data – the question is what is this line. The most common meaning given to best in this search for the line is the line whose total square error is the smallest possible. We make this notion precise in two steps 39 40 3. LINEAR REGRESSION D EFINITION 3.1.1. Given a bivariate quantitative dataset {(x1 , y1 ), . . . , (xn , yn )} and a candidate line yb = mx + b passing through this dataset, a residual is the difference in y-coordinates of an actual data point (xi , yi ) and the line’s y value at the same x-coordinate. That is, if the y-coordinate of the line when x = xi is ybi = mxi + b, then the residual is the measure of error given by errori = yi − ybi . Note we use the convention here and elsewhere of writing yb for the y-coordinate on an approximating line, while the plain y variable is left for actual data values, like yi . Here is an example of what residuals look like Now we are in the position to state the D EFINITION 3.1.2. Given a bivariate quantitative dataset the least square regression line, almost always abbreviated to LSRL, is the line for which the sum of the squares of the residuals is the smallest possible. FACT 3.1.3. If a bivariate quantitative dataset {(x1 , y1 ), . . . , (xn , yn )} has LSRL given by yb = mx + b, then (1) The slope of the LSRL is given by m = r ssxy , where r is the correlation coefficient of the dataset. (2) The LSRL passes through the point (x, y). (3) It follows that the y-intercept of the LSRL is given by b = y − x m = y − x r ssxy . It is possible to find the (coefficients of the) LSRL using the above information, but it is often more convenient to use a calculator or other electronic tool. Such tools also make it very easy to graph the LSRL right on top of the scatterplot – although it is often fairly easy to sketch what the LSRL will likely look like by just making a good guess, using 3.1. THE LEAST SQUARES REGRESSION LINE 41 visual intuition, if the linear association is strong (as will be indicated by the correlation coefficient). E XAMPLE 3.1.4. Here is some data where the individuals are 23 students in a statistics class, the independent variable is the students’ total score on their homeworks, while the dependent variable is their final total course points, both out of 100. x : 65 65 50 53 59 92 86 84 29 y : 74 71 65 60 83 90 84 88 48 x : 29 9 64 31 69 10 57 81 81 y : 54 25 79 58 81 29 81 94 86 x : 80 70 60 62 59 y : 95 68 69 83 70 Here is the resulting scatterplot, made with LibreOffice Calc(a free equivalent of Mi- crosoft Excel) It seems pretty clear that there is quite a strong linear association between these two vari- ables, as is born out by the correlation coefficient, r = .935 (computed with LibreOffice Calc’s CORREL). Using then STDEV.S and AVERAGE, we find that the coefficients of the LSRL for this data, yb = mx + b are sy 18.701 m=r = .935 = .754 and b = y − x m = 71 − 58 · .754 = 26.976 sx 23.207 42 3. LINEAR REGRESSION We can also use LibreOffice Calc’s Insert Trend Line, with Show Equation, to get all this done automatically. Note that when LibreOffice Calc writes the equation of the LSRL, it uses f (x) in place of yb, as we would. 3.2. APPLICATIONS AND INTERPRETATIONS OF LSRLS 43 3.2. Applications and Interpretations of LSRLs Suppose that we have a bivariate quantitative dataset {(x1 , y1 ), . . . , (xn , yn )} and we have computed its correlation coefficient r and (the coefficients of) its LSRL yb = mx + b. What is this information good for? The main use of the LSRL is described in the following D EFINITION 3.2.1. Given a bivariate quantitative dataset and associated LSRL with equation yb = mx + b, the process of guessing that the value of the dependent variable in this relationship to have the value mx0 + b, for x0 any value for the independent variable which satisfies xmin ≤ x0 ≤ xmax , is called interpolation. The idea of interpolation is that we think the LSRL describes as well as possible the relationship between the independent and dependent variables, so that if we have a new x value, we’ll use the LSRL equation to predict what would be our best guess of what would be the corresponding y. Note we might have a new value of x because we simply lost part of our dataset and are trying to fill it in as best we can. Another reason might be that a new individual came along whose value of the independent variable, x0 , was typical of the rest of the dataset – so the the very least xmin ≤ x0 ≤ xmax – and we want to guess what will be the value of the dependent variable for this individual before we measure it. (Or maybe we cannot measure it for some reason.) A common (but naive) alternate approach to interpolation for a value x0 as above might be to find two values xi and xj in the dataset which were as close to x0 as possible, and on either side of it (so xi < x0 < xj ), and simply to guess that the y-value for x0 would be the average of yi and yj . This is not a terrible idea, but it is not as effective as using the LSRL as described above, since we use the entire dataset when we build the coefficients of the LSRL. So the LSRL will give, by the process of interpolation, the best guess for what should be that missing y-value based on everything we know, while the “average of yi and yj ” method only pays attention to those two nearest data points and thus may give a very bad guess for the corresponding y-value if those two points are not perfectly typical, if they have any randomness, any variation in their y-values which is not due to the variation of the x. It is thus always best to use interpolation as described above. E XAMPLE 3.2.2. Working with the statistics students’ homework and total course points data from Example 3.1.4, suppose the gradebook of the course instructor was somewhat corrupted and the instructor lost the final course points of the student Janet. If Janet’s homework points of 77 were not in the corrupted part of the gradebook, the instructor might use interpolation to guess what Janet’s total course point probably were. To do this, the instructor would have plugged in x = 77 into the equation of the LSRL, yb = mx + b to get the estimated total course points of .754 · 77 + 26.976 = 85.034. 44 3. LINEAR REGRESSION Another important use of the (coefficients of the) LSRL is to use the underlying mean- ings of the slope and y-intercept. For this, recall that in the equation y = mx + b, the slope m tells us how much the line goes up (or down, if the slope is negative) for each increase of the x by one unit, while the y-intercept b tells us what would be the y value where the line crosses the y-axis, so when the x has the value 0. In each particular situation that we have bivariate quantitative data and compute an LSRL, we can then use these interpretations to make statements about the relationship between the independent and dependent variables. E XAMPLE 3.2.3. Look one more time at the data on students’ homework and total course points in a statistics class from Example 3.1.4, and the the LSRL computed there. We said that the slope of the LSRL was m = .754 and the y-intercept was b = 26.976. In context, what this means, is that On average, each additional point of homework cor- responded to an increase of .754 total course points. We may hope that this is actually a causal relationship, that the extra work a student does to earn that additional point of homework score helps the student learn more statistics and therefore get .75 more total course points. But the mathematics here does not require that causation, it merely tells us the increase in x is associated with that much increase in y. Likewise, we can also conclude from the LSRL that In general, a student who did no homework at all would earn about 26.976 total course points. Again, we cannot conclude that doing no homework causes that terrible final course point total, only that there is an association. 3.3. CAUTIONS 45 3.3. Cautions 3.3.1. Sensitivity to Outliers. The correlation coefficient and the (coefficients of the) LSRL are built out of means and standard deviations and therefore the following fact is completely unsurprising FACT 3.3.1. The correlation coefficient and the (coefficients of the) LSRL are very sensitive to outliers. What perhaps is surprising here is that the outliers for bivariate data are a little different from those for 1-variable data. D EFINITION 3.3.2. An outlier for a bivariate quantitative dataset is one which is far away from the curve which has been identified as underlying the shape of the scatterplot of that data. In particular, a point (x, y) can be a bivariate outlier even if both x is not an outlier for the independent variable data considered alone and y is not an outlier for the dependent variable data alone. E XAMPLE 3.3.3. Suppose we add one more point (90, 30) to the dataset in Exam- ple 3.1.4. Neither the x- nor y-coordinates of this point are outliers with respect to their respective single-coordinate datasets, but it is nevertheless clearly a bivariate outlier, as can be seen in the new scatterplot In fact recomputing the correlation coefficient and LSRL, we find quite a change from what we found before, in Example 3.1.4: r = .704 [which used to be .935] 46 3. LINEAR REGRESSION and yb = .529x + 38.458 [which used to be .754x + 26.976] all because of one additional point! 3.3.2. Causation. The attentive reader will have noticed that we started our discussion of bivariate data by saying we hoped to study when one thing causes another. However, what we’ve actually done instead is find correlation between variables, which is quite a different thing. Now philosophers have discussed what exactly causation is for millennia, so certainly it is a subtle issue that we will not resolve here. In fact, careful statisticians usually dodge the complexities by talking about relationships, association, and, of course, the correlation coefficient, being careful always not to commit to causation – at least based only on an analysis of the statistical data. As just one example, where we spoke about the meaning of the square r 2 of the cor- relation coefficient (we called it Fact 2.3.3), we were careful to say that r 2 measures the variation of the dependent variable which is associated with the variation of the indepen- dent variable. A more reckless description would have been to say that one caused the other – but don’t fall into that trap! This would be a bad idea because (among other reasons) the correlation coefficient is symmetric in the choice of explanatory and response variables (meaning r is the same no matter which is chosen for which role), while any reasonable notion of causation is asymmetric. E.g., while the correlation is exactly the same very large value with either variable being x and which y, most people would say that smoking causes cancer and not the other way1! We do need to make one caution about this caution, however. If there is a causal rela- tionship between two variables that are being studied carefully, then there will be correla- tion. So, to quote the great data scientist Edward Tufte [Tuf06], Correlation is not causation but it sure is a hint. The first part of this quote (up to the “but”) is much more famous and, as a very first step, is a good slogan to live by. Those with a bit more statistical sophistication might instead learn this version, though. A more sophisticated-sounding version, again due to Tufte [Tuf06], is Empirically observed covariation is a necessary but not sufficient condi- tion for causality. 1Although in the 1950s a doctor (who later was found to be in the pay of the tobacco industry) did say that the clear statistical evidence of association between smoking and cancer might be a sign that cancer causes smoking (I know: crazy!). His theory was that people who have lung tissue which is more prone to developing cancer are more likely to start smoking because somehow the smoke makes that particular tissue feel better. Needless to say, this is not the accepted medical view, because lots of evidence goes against it. 3.3. CAUTIONS 47 3.3.3. Extrapolation. We have said that visual intuition often allows humans to sketch fairly good approximations of the LSRL on a scatterplot, so long as the correlation coeffi- cient tells us there is a strong linear association. If the diligent reader did that with the first scatterplot in Example 3.1.4, probably the resulting line looked much like the line which LibreOffice Calc produced – except humans usually sketch their line all the way to the left and right edges of the graphics box. Automatic tools like LibreOffice Calc do not do that, for a reason. D EFINITION 3.3.4. Given a bivariate quantitative dataset and associated LSRL with equation yb = mx + b, the process of guessing that the value of the dependent variable in this relationship to have the value mx0 + b, for x0 any value for the independent variable which does not satisfy xmin ≤ x0 ≤ xmax [so, instead, either x0 < xmin or x0 > xmax ], is called extrapolation. Extrapolation is considered a bad, or at least risky, practice. The idea is that we used the evidence in the dataset {(x1 , y1), . . . , (xn , yn )} to build the LSRL, but, by definition, all of this data lies in the interval on the x-axis from xmin to xmax . There is literally no evidence from this dataset about what the relationship between our chosen explanatory and response variables will be for x outside of this interval. So in the absence of strong reasons to believe that the precise linear relationship described by the LSRL will continue for more x’s, we should not assume that it does, and therefore we should not use the LSRL equation to guess values by extrapolation. The fact is, however, that often the best thing we can do with available information when we want to make predictions out into uncharted territory on the x-axis is extrapola- tion. So while it is perilous, it is reasonable to extrapolate, so long as you are clear about what exactly you are doing. E XAMPLE 3.3.5. Using again the statistics students’ homework and total course points data from Example 3.1.4, suppose the course instructor wanted to predict what would be the total course points for a student who had earned a perfect 100 points on their homework. Plugging into the LSRL, this would have yielded a guess of .754 · 100 + 26.976 = 102.376. Of course, this would have been impossible, since the maximum possible total course score was 100. Moreover, making this guess is an example of extrapolation, since the x value of 100 is beyond the largest x value of xmax = 92 in the dataset. Therefore we should not rely on this guess – as makes sense, since it is invalid by virtue of being larger than 100. 3.3.4. Simpson’s Paradox. Our last caution is not so much a way using the LSRL can go wrong, but instead a warning to be ready for something very counter-intuitive to happen – so counter-intuitive, in fact, that it is called a paradox. It usually seems reasonable that if some object is cut into two pieces, both of which have a certain property, then probably the whole object also has that same property. But 48 3. LINEAR REGRESSION if the object in question is a population and the property is has positive correlation, then maybe the unreasonable thing happens. D EFINITION 3.3.6. Suppose we have a population for which we have a bivariate quan- titative dataset. Suppose further that the population is broken into two (or more) sub- populations for all of which the correlation between the two variables is positive, but the correlation of the variables for the whole dataset is negative. Then this situation is called Simpson’s Paradox. [It’s also called Simpson’s Paradox if the role of positive and negative is reversed in our assumptions.] The bad news is that Simpson’s paradox can happen. E XAMPLE 3.3.7. Let P = {(0, 1), (1, 0), (9, 10), (10, 9)} be a bivariate dataset, which is broken into the two subpopulations P1 = {(0, 1), (1, 0)} and P2 = {(9, 10), (10, 9)}. Then the correlation coefficients of both P1 and P2 are r = −1, but the correlation of all of P is r = .9756. This is Simpson’s Paradox! Or, in applications, we can have situations like E XAMPLE 3.3.8. Suppose we collect data on two sections of a statistics course, in particular on how many hours per work the individual students study for the course and how they do in the course, measured by their total course points at the end of the semester. It is possible that there is a strong positive correlation between these variables for each section by itself, but there is a strong negative correlation when we put all the students into one dataset. In other words, it is possible that the rational advice, based on both individual sections, is study more and you will do better in the course, but that the rational advice based on all the student data put together is study less and you will do better. EXERCISES 49 Exercises E XERCISE 3.1. The age (x) and resting heart rate (RHR, y) were measured for nine men, yielding this dataset: x : 20 23 30 37 35 45 51 60 63 y : 72 71 73 74 74 73 75 75 77 Make a scatterplot of these data. Based on the scatterplot, what do you think the correlation coefficient r will be? Now compute r. Compute the LSRL for these data, write down its equation, and sketch it on top of your scatterplot. [You may, of course, do as much of this with electronic tools as you like. However, you should explain what tool you are using, how you used it, and what it must have been doing behind the scenes to get the results which it displayed and you are turning in.] E XERCISE 3.2. Continuing with the data and computations of the previous problem: What percentage of the variation in RHR is associated with variation in age? Write the following sentences with blanks filled in: “If I measured the RHR of a 55 year-old man, I would expect it to be . Making an estimate like this is called .” Just looking at the equation of the LSRL, what does it suggest should be the RHR of a newborn baby? Explain. Also explain what an estimate like yours for the RHR of a baby is called. This kind of estimate is considered a bad idea in many cases – explain why in general, and also use specifics from this particular case. E XERCISE 3.3. Write down a bivariate quantitative dataset for a population of only two individuals whose LSRL is yb = 2x − 1. What is the correlation coefficient of your dataset? Next, add one more point to the dataset in such a way that you don’t change the LSRL or correlation coefficient. Finally, can your find a dataset with the same LSRL but having a larger correlation coefficient than you just had? [Hint: fool around with modifications or additions to the datasets in you already found in this problem, using an electronic tool to do all the computational work. When you find a good one, write it down and explain what you thinking was as you searched for it.] Part 2 Good Data It is something of an aphorism among statisticians that The plural of anecdote is not data.2 The distinction being emphasized here is between the information we might get from a personal experience or a friend’s funny story – an anecdote – and the cold, hard, objective information on which we want to base our scientific investigations of the world – data. In this Part, our goal is to discuss aspects of getting good data. It may seem counter- intuitive, but the first step in that direction is to develop some of the foundations of prob- ability theory, the mathematical study of systems which are non-deterministic – random – but in a consistent way. The reason for this is that the easiest and most reliable way to ensure objectivity in data, to suppress personal choices which may result in biased infor- mation from which we cannot draw universal, scientific conclusions, is to collect your data randomly. Randomness is a tool which the scientist introduces intentionally and carefully, as barrier against bias, in the collection of high quality data. But this strategy only works if we can understand how to extract precise information even in the presence of randomness – hence the importance of studying probability theory. After a chapter on probability, we move on to a discussion of some fundamentals of experimental design – starting, not surprisingly, with randomization, but finishing with the gold standard for experiments (on humans, at least): randomized, placebo-controlled, double-blind experiments [RCTs]. Experiments whose subjects are not humans share some, but not all, of these design goals It turns out that, historically, a number of experiments with human subjects have had very questionable moral foundations, so it is very important to stop, as we do in the last chapter of this Part, to build a outline of experimental ethics. 2Itis hard to be certain of the true origins of this phrase. The political scientist Raymond Wolfinger is sometimes given credit [PB] – for a version without the “not,” actually. Sometime later, then, it became widespread with the “not.” CHAPTER 4 Probability Theory We want to imagine doing an experiment in which there is no way to predict what the outcome will be. Of course, if we stop our imagination there, there would be nothing we could say and no point in trying to do any further analysis: the outcome would just be whatever it wanted to be, with no pattern. So let us add the additional assumption that while we cannot predict what will happen any particular time we do the experiment, we can predict general trends, in the long run, if we repeat the experiment many times. To be more precise, we assume that, for any collection E of possible outcomes of the experiment there is a number p(E) such that, no matter who does the experiment, no matter when they do it, if they repeat the experiment many times, the fraction of times they would have seen any of the outcomes of E would be close to that number p(E). This is called the frequentist approach to the idea of probability. While it is not uni- versally accepted – the Bayesian alternative does in fact have many adherents – it has the virtue of being the most internally consistent way of building a foundation for probability. For that reason, we will follow the frequentist description of probability in this text. Before we jump into the mathematical formalities, we should motivate two pieces of what we just said. First, why talk about sets of outcomes of the experiment instead of talking about individual outcomes? The answer is that we are often interested in sets of outcomes, as we shall see later in this book, so it is nice to set up the machinery from the very start to work with such sets. Or, to give a particular concrete example, suppose you were playing a game of cards and could see your hand but not the other players’ hands. You might be very interested in how likely is it that your hand is a winning hand, i.e., what is the likelihood of the set of all possible configurations of all the rest of the cards in the deck and in your opponents’ hands for which what you have will be the winning hand? It is situations like this which motivate an approach based on sets of outcomes of the random experiment. Another question we might ask is: where does our uncertainty about the experimental results come from? From the beginnings of the scientific method through the turn of the 20th century, it was thought that this uncertainty came from our incomplete knowledge of the system on which we were experimenting. So if the experiment was, say, flipping a coin, the precise amount of force used to propel the coin up into the air, the precise angular motion imparted to the coin by its position just so on the thumbnail of the person doing 53 54 4. PROBABILITY THEORY the flipping, the precise drag that the coin felt as it tumbled through the air caused in part by eddies in the air currents coming from the flap of a butterfly’s wings in the Amazon rainforest – all of these things could significantly contribute to changing whether the coin would eventually come up heads or tails. Unless the coin-flipper was a robot operating in a vacuum, then, there would just be no way to know all of these physical details with enough accuracy to predict the toss. After the turn of the 20th century, matters got even worse (at least for physical deter- minists): a new theory of physics came along then, called Quantum Mechanics, according to which true randomness is built into the laws of the universe. For example, if you have a very dim light source, which produces the absolutely smallest possible “chunks” of light (called photons), and you shine it through first one polarizing filter and then see if it goes through a second filter at a 45◦ angle to the first, then half the photons will get through the second filter, but there is absolutely no way ever to predict whether any particular photon will get though or not. Quantum mechanics is full of very weird, non-intuitive ideas, but it is one of the most well-tested theories in the history of science, and it has passed every test. 4.1. DEFINITIONS FOR PROBABILITY 55 4.1. Definitions for Probability 4.1.1. Sample Spaces, Set Operations, and Probability Models. Let’s get right to the definitions. D EFINITION 4.1.1. Suppose we have a repeatable experiment we want to investigate probabilistically. The things that happen when we do the experiment, the results of running it, are called the [experimental] outcomes. The set of all outcomes is called the sample space of the experiment. We almost always use the symbol S for this sample space. E XAMPLE 4.1.2. Suppose the experiment we are doing is “flip a coin.” Then the sample space would be S = {H, T }. E XAMPLE 4.1.3. For the experiment “roll a [normal, six-sided] die,” the sample space would be S = {1, 2, 3, 4, 5, 6}. E XAMPLE 4.1.4. For the experiment “roll two dice,” the sample space would be S = {11, 12, 13, 14, 15, 16, 21, 22, 23, 24, 25, 26 31, 23, 33, 34, 35, 36 41, 42, 43, 44, 45, 46 51, 52, 53, 54, 55, 56 61, 62, 63, 64, 65, 66 where the notation “nm” means “1st roll resulted in an n, 2nd in an m.” E XAMPLE 4.1.5. Consider the experiment “flip a coin as many times as necessary to see the first Head.” This would have the infinite sample space S = {H, T H, T T H, T T T H, T T T T H, . . . } . E XAMPLE 4.1.6. Finally, suppose the experiment is “point a Geiger counter at a lump of radioactive material and see how long you have to wait until the next click.” Then the sample space S is the set of all positive real numbers, because potentially the waiting time could be any positive amount of time. As mentioned in the chapter introduction, we are more interested in D EFINITION 4.1.7. Given a repeatable experiment with sample space S, an event is any collection of [some, all, or none of the] outcomes in S; i.e., an event is any subset E of S, written E ⊂ S. 56 4. PROBABILITY THEORY There is one special set which is a subset of any other set, and therefore is an event in any sample space. D EFINITION 4.1.8. The set {} with no elements is called the empty set, for which we use the notation ∅. E XAMPLE 4.1.9. Looking at the sample space S = {H, T } in Example 4.1.2, it’s pretty clear that the following are all the subsets of S: ∅ {H} {T } S [= {H, T }] Two parts of that example are always true: ∅ and S are always subsets of any set S. Since we are going to be working a lot with events, which are subsets of a larger set, the sample space, it is nice to have a few basic terms from set theory: D EFINITION 4.1.10. Given a subset E ⊂ S of a larger set S, the complement of E, is the set E c = {all the elements of S which are not in E}. If we describe an event E in words as all outcomes satisfies some property X, the complementary event, consisting of all the outcomes not in E, can be described as all outcomes which don’t satisfy X. In other words, we often describe the event E c as the event “not E.” D EFINITION 4.1.11. Given two sets A and B, their union is the set A ∪ B = {all elements which are in A or B [or both]} . Now if event A is those outcomes having property X and B is those with property Y , the event A ∪ B, with all outcomes in A together with all outcomes in B can be described as all outcomes satisfying X or Y , thus we sometimes pronounce the event “A ∪ B” as “A or B.” D EFINITION 4.1.12. Given two sets A and B, their intersection is the set A ∩ B = {all elements which are in both A and B} . If, as before, event A consists of those outcomes having property X and B is those with property Y , the event A ∩ B will consist of those outcomes which satisfy both X and Y . In other words, “A ∩ B” can be described as “A and B.” Putting together the idea of intersection with the idea of that special subset ∅ of any set, we get the 4.1. DEFINITIONS FOR PROBABILITY 57 D EFINITION 4.1.13. Two sets A and B are called disjoint if A ∩ B = ∅. In other words, sets are disjoint if they have nothing in common. A exact synonym for disjoint that some authors prefer is mutually exclusive. We will use both terms interchangeably in this book. Now we are ready for the basic structure of probability. D EFINITION 4.1.14. Given a sample space S, a probability model on S is a choice of a real number P (E) for every event E ⊂ S which satisfies (1) For all events E, 0 ≤ P (E) ≤ 1. (2) P (∅) = 1 and P (S) = 1. (3) For all events E, P (E c ) = 1 − P (E). (4) If A and B are any two disjoint events, then P (A ∪ B) = P (A) + P (B). [This is called the addition rule for disjoint events.] 4.1.2. Venn Diagrams. Venn diagrams are a simple way to display subsets of a fixed set and to show the relationships between these subsets and even the results of various set operations (like complement, union, and intersection) on them. The primary use we will make of Venn diagrams is for events in a certain sample space, so we will use that terminology [even though the technique has much wider application]. To make a Venn Diagram, always start out by making a rectangle to represent the whole sample space: Within that rectangle, we make circles, ovals, or just blobs, to indicate that portion of the sample space which is some event E: 58 4. PROBABILITY THEORY Sometimes, if the outcomes in the sample space S and in the event A might be indicated in the different parts of the Venn diagram. So, if S = {a, b, c, d} and A = {a, b} ⊂ S, we might draw this as The complement E c of an event E is easy to show on a Venn diagram, since it is simply everything which is not in E: If the filled part here is E ... then the filled part here is E c This can actually be helpful in figuring out what must be in E c . In the example above with S = {a, b, c, d} and A = {a, b} ⊂ S, by looking at what is in the shaded exterior part for our picture of E c , we can see that for that A, we would get Ac = {c, d}. 4.1. DEFINITIONS FOR PROBABILITY 59 Moving now to set operations that work with two events, suppose we want to make a Venn diagram with events A and B. If we know these events are disjoint, then we would make the diagram as follows: while if they are known not to be disjoint, we would use instead this diagram: For example, it S = {a, b, c, d}, A = {a, b}, and B = {b, c}, we would have When in doubt, it is probably best to use the version with overlap, which then could simply not have any points in it (or could have zero probability, when we get to that, below). 60 4. PROBABILITY THEORY Venn diagrams are very good at showing unions, and intersection: If the filled part here is A and the filled part here is B then the filled part here is A ∪ B and the filled part here is A ∩ B Another nice thing to do with Venn diagrams is to use them as a visual aid for proba- bility computations. The basic idea is to make a diagram showing the various events sitting inside the usual rectangle, which stands for the sample space, and to put numbers in various parts of the diagram showing the probabilities of those events, or of the results of operations (unions, intersection, and complement) on those events. For example, if we are told that an event A has probability P (A) = .4, then we can immediately fill in the .4 as follows: 4.1. DEFINITIONS FOR PROBABILITY 61 But we can also put a number in the exterior of that circle which represents A, taking advantage of the fact that that exterior is Ac and the rule for probabilities of complements (point (3) in Definition 4.1.14) to conclude that the appropriate number is 1 − .4 = .6: We recommend that, in a Venn diagram showing probability values, you always put a number in the region exterior to all of the events [but inside the rectangle indicating the sample space, of course]. Complicating a little this process of putting probability numbers in the regions of a Venn diagram is the situation where we are giving for both an event and a subsetsubset, ⊂ of that event. This most often happens when we are told probabilities both of some events and of their intersection(s). Here is an example: E XAMPLE 4.1.15. Suppose we are told that we have two events A and B in the sample space S, which satisfy P (A) = .4, P (B) = .5, and P (A ∩ B) = .1. First of all, we know that A and B are not disjoint, since if they were disjoint, that would mean (by definition) that A ∩ B = ∅, and since P (∅) = 0 but P (A ∩ B) 6= 0, that is not possible. So we draw a Venn diagram that we’ve see before: However, it would be unwise simply to write those given numbers .4, .5, and .1 into the three central regions of this diagram. The reason is that the number .1 is the probability of 62 4. PROBABILITY THEORY A ∩ B, which is a part of A already, so if we simply write .4 in the rest of A, we would be counting that .1 for the A ∩ B twice. Therefore, before we write a number in the rest of A, outside of A ∩ B, we have to subtract the .1 for P (A ∩ B). That means that the number which goes in the rest of A should be .4 − .1 = .3. A similar reasoning tells us that the number in the part of B outside of A ∩ B, should be .5 − .1 = .4. That means the Venn diagram with all probabilities written in would be: The approach in the above example is our second important recommendation for who to put numbers in a Venn diagram showing probability values: always put a number in each region which corresponds to the probability of that smallest connected region containing the number, not any larger region. One last point we should make, using the same argument as in the above example. Suppose we have events A and B in a sample space S (again). Suppose we are not sure if A and B are disjoint, so we cannot use the addition rule for disjoint events to compute P (A ∪ B). But notice that the events A and Ac are disjoint, so that A ∩ B and Ac ∩ B are also disjoint and A = A ∩ S = A ∩ (B ∪ B c ) = (A ∩ B) ∪ (A ∩ B c ) is a decomposition of the event A into the two disjoint events A ∩ B and Ac ∩ B. From the addition rule for disjoint events, this means that P (A) = P (A ∩ B) + P (A ∩ B c ) . Similar reasoning tells us both that P (B) = P (A ∩ B) + P (Ac ∩ B) and that A ∪ B = (A ∩ B c ) ∪ (A ∩ B) ∪ (Ac ∩ B) is a decomposition of A ∪ B into disjoint pieces, so that P (A ∪ B) = P (A ∩ B c ) + P (A ∩ B) + P (Ac ∩ B) . 4.1. DEFINITIONS FOR PROBABILITY 63 Combining all of these equations, we conclude that P (A) + P (B) − P (A ∩ B) = P (A ∩ B) + P (A ∩ B c ) + P (A ∩ B) + P (Ac ∩ B) − P (A ∩ B) = P (A ∩ B c ) + P (A ∩ B) + P (Ac ∩ B) + P (A ∩ B) − P (A ∩ B) = P (A ∩ B c ) + P (A ∩ B) + P (Ac ∩ B) = P (A ∪ B) . This is important enough to state as a FACT 4.1.16. The Addition Rule for General Events If A and B are events in a sample space S then we have the addition rule for their probabilities P (A ∪ B) = P (A) + P (B) − P (A ∩ B) . This rule is true whether or not A and B are disjoint. 4.1.3. Finite Probability Models. Here is a nice situation in which we can easily cal- culate a lot of probabilities fairly easily: if the sample space S of some experiment is finite. So let’s suppose the sample space consists of just the outcomes S = {o1 , o2 , . . . , on }. For each of the outcomes, we can compute the probability: p1 =P ({o1}) p2 =P ({o2}) .. . pn =P ({on }) Let’s think about what the rules for probability models tell us about these numbers p1 , p2 , . . . , pn . First of all, since they are each the probability of an event, we see that 0 ≤p1 ≤ 1 0 ≤p2 ≤ 1 .. . 0 ≤pn ≤ 1 Furthermore, since S = {o1 , o2 , . . . , on } = {o1 } ∪ {o2 } ∪ · · · ∪ {on } and all of the events {o1 }, {o2 }, . . . , {on } are disjoint, by the addition rule for disjoint events we have 1 = P (S) = P ({o1 , o2 , . . . , on }) = P ({o1 } ∪ {o2 } ∪ · · · ∪ {on }) = P ({o1 }) + P ({o2}) + · · · + P ({on }) = p1 + p2 + · · · + pn . 64 4. PROBABILITY THEORY The final thing to notice about this situation of a finite sample space is that if E ⊂ S is any event, then E will be just a collection of some of the outcomes from {o1 , o2 , . . . , on } (maybe none, maybe all, maybe an intermediate number). Since, again, the events like {o1 } and {o2 } and so on are disjoint, we can compute P (E) = P ({the outcomes oj which make up E}) X = {the pj ’s for the outcomes in E} . In other words FACT 4.1.17. A probability model on a sample space S with a finite number, n, of outcomes, is nothing other than a choice of real numbers p1 , p2 , . . . , pn , all in the range from 0 to 1 and satisfying p1 + p2 + · · · + pn = 1. For such a choice of numbers, we can compute the probability of any event E ⊂ S as X P (E) = {the pj ’s corresponding to the outcomes oj which make up E} . E XAMPLE 4.1.18. For the coin flip of Example 4.1.2, there are only the two outcomes H and T for which we need to pick two probabilities, call them p and q. In fact, since the total must be 1, we know that p + q = 1 or, in other words, q = 1 − p. The the probabilities for all events (which we listed in Example 4.1.9) are P (∅) = 0 P ({H}) = p P ({T }) = q = 1 − p P ({H, T }) = p + q = 1 What we’ve described here is, potentially, a biased coin, since we are not assuming that p = q – the probabilities of getting a head and a tail are not assumed to be the same. The alternative is to assume that we have a fair coin, meaning that p = q. Note that in such a case, since p + q = 1, we have 2p = 1 and so p = 1/2. That is, the probability of a head (and, likewise, the probability of a tail) in a single throw of a fair coin is 1/2. E XAMPLE 4.1.19. As in the previous example, we can consider the die of Exam- ple 4.1.3 to a fair die, meaning that the individual face probabilities are all the same. Since they must also total to 1 (as we saw for all finite probability models), it follows that p1 = p2 = p3 = p4 = p5 = p6 = 1/6. We can then use this basic information and the formula (for P (E)) in Fact 4.1.17 to com- pute the probability of any event of interest, such as 1 1 1 3 1 P (“roll was even”) = P ({2, 4, 6}) = + + = = . 6 6 6 6 2 We should immortalize these last two examples with a 4.1. DEFINITIONS FOR PROBABILITY 65 D EFINITION 4.1.20. When we are talking about dice, coins, individuals for some task, or another small, practical, finite experiment, we use the term fair to indicate that the probabilities of all individual outcomes are equal (and therefore all equal to the the number 1/n, where n is the number of outcomes in the sample space). A more technical term for the same idea is equiprobable, while a more casual term which is often used for this in very informal settings is “at random” (such as “pick a card at random from this deck” or “pick a random patient from the study group to give the new treatment to...”). E XAMPLE 4.1.21. Suppose we look at the experiment of Example 4.1.4 and add the information that the two dice we are rolling are fair. This actually isn’t quite enough to figure out the probabilities, since we also have to assure that the fair rolling of the first die doesn’t in any way affect the rolling of the second die. This is technically the requirement that the two rolls be independent, but since we won’t investigate that carefully until §4.2, below, let us instead here simply say that we assume the two rolls are fair and are in fact completely uninfluenced by anything around them in the world including each other. What this means is that, in the long run, we would expect the first die to show a 1 th roughly 61 of the time, and in the very long run, the second die would show a 1 roughly 1 th 6 of those times. This means that the outcome of the “roll two dice” experiment should be 1 11 with probability 36 – and the same reasoning would show that all of the outcomes have that probability. In other words, this is an equiprobable sample space with 36 outcomes 1 each having probability 36 . Which in turn enables us to compute any probability we might like, such as P (“sum of the two rolls is 4”) = P ({13, 22, 31}) 1 1 1 = + + 36 36 36 3 = 36 1 = . 12 66 4. PROBABILITY THEORY 4.2. Conditional Probability We have described the whole foundation of the theory of probability as coming from imperfect knowledge, in the sense that we don’t know for sure if an event A will happen any particular time we do the experiment but we do know, in the long run, in what fraction of times A will happen. Or, at least, we claim that there is some number P (A) such that after running the experiment N times, out of which nA of these times are when A happened, P (A) is approximately nA /N (and this ratio gets closer and closer to P (A) as N gets bigger and bigger). But what if we have some knowledge? In particular, what happens if we know for sure that the event B has happened – will that influence our knowledge of whether A happens or not? As before, when there is randomness involved, we cannot tell for sure if A will happen, but we hope that, given the knowledge that B happened, we can make a more accurate guess about the probability of A. E XAMPLE 4.2.1. If you pick a person at random in a certain country on a particular date, you might be able to estimate the probability that the person had a certain height if you knew enough about the range of heights of the whole population of that country. [In fact, below we will make estimates of this kind.] That is, if we define the event A = “the random person is taller than 1.829 meters (6 feet)” then we might estimate P (A). But consider the event B = “the random person’s parents were both taller than 1.829 meters” . Because there is a genetic component to height, if you know that B happened, it would change your idea of how likely, given that knowledge, that A happened. Because genetics are not the only thing which determines a person’s height, you would not be certain that A happened, even given the knowledge of B. Let us use the frequentist approach to derive a formula for this kind of probability of A given that B is known to have happened. So think about doing the repeatable experiment many times, say N times. Out of all those times, some times B happens, say it happens nB times. Out of those times, the ones where B happened, sometimes A also happened. These are the cases where both A and B happened – or, converting this to a more mathematical descriptions, the times that A ∩ B happened – so we will write it nA∩B . We know that the probability of A happening in the cases where we know for sure that B happened is approximately nA∩B /nB . Let’s do that favorite trick of multiplying and dividing by the same number, so finding that the probability in which we are interested is 4.2. CONDITIONAL PROBABILITY 67 approximately , nA∩B nA∩B · N nA∩B N nA∩B nB . = = · = ≈ P (A ∩ B) P (B) nB N · nB N nB N N Which is why we make the D EFINITION 4.2.2. The conditional probability of the event A given the event B is P (A ∩ B) P (A|B) = . P (B) Here P (A|B) is pronounced the probability of A given B. Let’s do a simple E XAMPLE 4.2.3. Building off of Example 4.1.19, note that the probability of rolling a 2 is P ({2}) = 1/6 (as is the probability of rolling any other face – it’s a fair die). But suppose that you were told that the roll was even, which is the event {2, 4, 6}, and asked for the probability that the roll was a 2 given this prior knowledge. The answer would be P ({2} ∩ {2, 4, 6}) P ({2}) 1/6 P ({2} | {2, 4, 6}) = = = = 1/3 . P ({2, 4, 6}) P ({2, 4, 6}) 1/2 In other words, the probability of rolling a 2 on a fair die with no other information is 1/6, which the probability of rolling a 2 given that we rolled an even number is 1/3. So the probability doubled with the given information. Sometimes the probability changes even more than merely doubling: the probability that we rolled a 1 with no other knowledge is 1/6, while the probability that we rolled a 1 given that we rolled an even number is P ({1} ∩ {2, 4, 6}) P (∅) 0 P ({1} | {2, 4, 6}) = = = =0. P ({2, 4, 6}) P ({2, 4, 6}) 1/2 But, actually, sometimes the conditional probability for some event is the same as the unconditioned probability. In other words, sometimes knowing that B happened doesn’t change our estimate of the probability of A at all, they are no really related events, at least from the point of view of probability. This motivates the D EFINITION 4.2.4. Two events A and B are called independent if P (A | B) = P (A). Plugging the defining formula for P (A | B) into the definition of independent, it is easy to see that FACT 4.2.5. Events A and B are independent if and only if P (A ∩ B) = P (A) · P (B). E XAMPLE 4.2.6. Still using the situation of Example 4.1.19, we saw in Example 4.2.3 that the events {2} and {2, 3, 4} are not independent since P ({2}) = 1/6 6= 1/3 = P ({2} | {2, 4, 6}) 68 4. PROBABILITY THEORY nor are {1} and {2, 3, 4}, since P ({1}) = 1/6 6= 0 = P ({1} | {2, 4, 6}) . However, look at the events {1, 2} and {2, 4, 6}: P ({1, 2}) = P ({1}) + P ({2}) = 1/6 + 1/6 = 1/3 1/6 = 1/2 P ({1}) = P ({2, 4, 6}) P ({1, 2} ∩ {2, 4, 6}) = P ({2, 4, 6}) = P ({1, 2} | {2, 4, 6}) which means that they are independent! E XAMPLE 4.2.7. We can now fully explain what was going on in Example 4.1.21. The two fair dice were supposed to be rolled in a way that the first roll had no effect on the second – this exactly means that the dice were rolled independently. As we saw, this then 1 means that each individual outcome of sample space S had probability 36 . But the first roll having any particular value is independent of the second roll having another, e.g., if A = {11, 12, 13, 14, 15, 16} is the event in that sample space of getting a 1 on the first roll and B = {14, 24, 34, 44, 54, 64} is the event of getting a 4 on the second roll, then events A and B are independent, as we check by using Fact 4.2.5: P (A ∩ B) = P ({14}) 1 = 36 1 1 = · 6 6 6 6 = · 36 36 = P (A) · P (B) . On the other hand, the event “the sum of the rolls is 4,” which is C = {13, 22, 31} as a 1 set, is not independent of the value of the first roll, since P (A ∩ C) = P ({13}) = 36 but 6 3 1 1 1 P (A) · P (C) = 36 · 36 = 6 · 12 = 72 . 4.3. RANDOM VARIABLES 69 4.3. Random Variables 4.3.1. Definition and First Examples. Suppose we are doing a random experiment and there is some consequence of the result in which we are interested that can be measured by a number. The experiment might be playing a game of chance and the result could be how much you win or lose depending upon the outcome, or the experiment could be which part of the drives’ manual you randomly choose to study and the result how many points we get on the driver’s license test we make the next day, or the experiment might be giving a new drug to a random patient in medical study and the result would be some medical measurement you make after treatment (blood pressure, white blood cell count, whatever), etc. There is a name for this situation in mathematics D EFINITION 4.3.1. A choice of a number for each outcome of a random experiment is called a random variable [RV]. If the values an RV takes can be counted, because they are either finite or countably infinite1 in number, the RV is called discrete; if, instead, the RV takes on all the values in an interval of real numbers, the RV is called continuous. We usually use capital letters to denote RVs and the corresponding lowercase letter to indicate a particular numerical value the RV might have, like X and x. E XAMPLE 4.3.2. Suppose we play a silly game where you pay me $5 to play, then I flip a fair coin and I give you $10 if the coin comes up heads and $0 if it comes up tails. Then your net winnings, which would be +$5 or -$5 each time you play, are a random variable. Having only two possible values, this RV is certainly discrete. E XAMPLE 4.3.3. Weather phenomena vary so much, due to such small effects – such as the famous butterfly flapping its wings in the Amazon rain forest causing a hurricane in North America – that they appear to be a random phenomenon. Therefore, observing the temperature at some weather station is a continuous random variable whose value can be any real number in some range like −100 to 100 (we’re doing science, so we use ◦ C). E XAMPLE 4.3.4. Suppose we look at the “roll two fair dice independently” experiment from Example 4.2.7 and Example 4.1.21, which was based on the probability model in Example 4.1.21 and sample space in Example 4.1.4. Let us consider in this situation the random variable X whose value for some pair of dice rolls is the sum of the two numbers showing on the dice. So, for example, X(11) = 2, X(12) = 3, etc. 1There many kinds of infinity in mathematics – in fact, an infinite number of them. The smallest is an infinity that can be counted, like the whole numbers. But then there are many larger infinities, describing sets that are too big even to be counted, like the set of all real numbers. 70 4. PROBABILITY THEORY In fact, let’s make a table of all the values of X: X(11) = 2 X(21) = X(12) = 3 X(31) = X(22) = X(13) = 4 X(41) = X(32) = X(23) = X(14) = 5 X(51) = X(42) = X(33) = X(24) = X(15) = 6 X(61) = X(52) = X(43) = X(34) = X(25) = X(16) = 7 X(62) = X(53) = X(44) = X(35) = X(26) = 8 X(63) = X(54) = X(45) = X(36) = 9 X(64) = X(55) = X(46) = 10 X(65) = X(56) = 11 X(66) = 12 4.3.2. Distributions for Discrete RVs. The first thing we do with a random variable, usually, is talk about the probabilities associate with it. D EFINITION 4.3.5. Given a discrete RV X, its distribution is a list of all of the values X takes on, together with the probability of it taking that value. [Note this is quite similar to Definition 1.3.5 – because it is essentially the same thing.] E XAMPLE 4.3.6. Let’s look at the RV, which we will call X, in the silly betting game of Example 4.3.2. As we noticed when we first defined that game, there are two possible values for this RV, $5 and -$5. We can actually think of “X = 5” as describing an event, consisting of the set of all outcomes of the coin-flipping experiment which give you a net gain of $5. Likewise, “X = −5” describes the event consisting of the set of all outcomes which give you a net gain of -$5. These events are as follows: Set of outcomes o x such that X(o) = x 5 {H} −5 {T } Since it is a fair coin so the probabilities of these events are known (and very simple), we conclude that the distribution of this RV is the table x P (X = x) 5 1/2 −5 1/2 4.3. RANDOM VARIABLES 71 E XAMPLE 4.3.7. What about the X = ”sum of the face values” RV on the “roll two fair dice, independently” random experiment from Example 4.3.4? We have actually already done most of the work, finding out what values the RV can take and which outcomes cause each of those values. To summarize what we found: Set of outcomes o x such that X(o) = x 2 {11} 3 {21, 12} 4 {31, 22, 13} 5 {41, 32, 23, 14} 6 {51, 42, 33, 24, 15} 7 {61, 52, 43, 34, 25, 16} 8 {62, 53, 44, 35, 26} 9 {63, 54, 45, 36} 10 {64, 55, 46} 11 {65, 56} 12 {66} But we have seen that this is an equiprobable situation, where the probability of any event A contain n outcomes is P (A) = n · 1/36, so we can instantly fill in the distribution table for this RV as x P (X = x) 1 2 36 2 1 3 36 = 18 3 1 4 36 = 12 4 1 5 36 = 6 5 6 36 6 1 7 36 = 6 5 8 36 4 1 9 36 = 6 3 1 10 36 = 12 2 1 11 36 = 18 1 12 36 One thing to notice about distributions is that if we make a preliminary table, as we just did, of the events consisting of all outcomes which give a particular value when plugged into the RV, then we will have a collection of disjoint events which exhausts all of the sam- ple space. What this means is that the sum of the probability values in the distribution table of an RV is the probability of the whole sample space of that RV’s experiment. Therefore 72 4. PROBABILITY THEORY FACT 4.3.8. The sum of the probabilities in a distribution table for a random variable must always equal 1. It is quite a good idea, whenever you write down a distribution, to check that this Fact is true in your distribution table, simply as a sanity check against simple arithmetic errors. 4.3.3. Expectation for Discrete RVs. Since we cannot predict what exactly will be the outcome each time we perform a random experiment, we cannot predict with precision what will be the value of an RV on that experiment, each time. But, as we did with the basic idea of probability, maybe we can at least learn something from the long-term trends. It turns out that it is relatively easy to figure out the mean value of an RV over a large number of runs of the experiment. Say X is a discrete RV, for which the distribution tells us that X takes the values x1 , . . . , xn , each with corresponding probability p1 , . . . , pn . Then the frequentist view of probability says that the probability pi that X = xi is (approximately) ni /N, where ni is the number of times X = xi out of a large number N of runs of the experiment. But if pi = ni /N then, multiplying both sides by N, ni = pi N . That means that, out of the N runs of the experiment, X will have the value x1 in p1 N runs, the value x2 in p2 N runs, etc. So the sum of X over those N runs will be (p1 N)x1 + (p2 N)x2 + · · · + (pn N)xn . Therefore the mean value of X over these N runs will be the total divided by N, which is p1 x1 + · · · + pn xn . This motivates the definition D EFINITION 4.3.9. Given a discrete RV X which takes on the values x1 , . . . , xn with probabilities p1 , . . . , pn , the expectation [sometimes also called the expected value] of X is the value X E(X) = pi xi . By what we saw just before this definition, we have the following FACT 4.3.10. The expectation of a discrete RV is the mean of its values over many runs of the experiment. Note: The attentive reader will have noticed that we dealt above only with the case of a finite RV, not the case of a countably infinite one. It turns out that all of the above works quite well in that more complex case as well, so long as one is comfortable with a bit of mathematical technology called “summing an infinite series.” We do not assume such a 4.3. RANDOM VARIABLES 73 comfort level in our readers at this time, so we shall pass over the details of expectations of infinite, discrete RVs. E XAMPLE 4.3.11. Let’s compute the expectation of net profit RV X in the silly bet- ting game of Example 4.3.2, whose distribution we computed in Example 4.3.6. Plugging straight into the definition, we see X 1 1 E(X) = pi xi = · 5 + · (−5) = 2.5 − 2.5 = 0 . 2 2 In other words, your average net gain playing this silly game many times will be zero. Note that does not mean anything like “if you lose enough times in a row, the chances of starting to win again will go up,” as many gamblers seem to believe, it just means that, in the very long run, we can expect the average winnings to be approximately zero – but no one knows how long that run has to be before the balancing of wins and losses happens2. A more interesting example is E XAMPLE 4.3.12. In Example 4.3.7 we computed the distribution of the random vari- able X = “sum of the face values” on the “roll two fair dice, independently” random ex- periment from Example 4.3.4. It is therefore easy to plug the values of the probabilities and RV values from the distribution table into the formula for expectation, to get X E(X) = pi xi 1 2 3 4 5 6 5 4 3 = ·2+ ·3+ ·4+ ·5+ ·6+ ·7+ ·8+ ·9+ · 10 36 36 36 36 36 36 36 36 36 2 1 + · 11 + · 12 36 36 2 · 1 + 3 · 2 + 4 · 3 + 5 · 4 + 6 · 5 + 7 · 6 + 8 · 5 + 9 · 4 + 10 · 3 + 11 · 2 + 12 · 1 = 36 =7 So if you roll two fair dice independently and add the numbers which come up, then do this process many times and take the average, in the long run that average will be the value 7. 4.3.4. Density Functions for Continuous RVs. What about continuous random vari- ables? Definition 4.3.5 of distribution explicitly excluded the case of continuous RVs, so does that mean we cannot do probability calculations in that case? There is, when we think about it, something of a problem here. A distribution is sup- posed to be a list of possible values of the RV and the probability of each such value. But if some continuous RV has values which are an interval of real numbers, there is just no way to list all such numbers – it has been known since the late 1800s that there is no way to make a list like that (see [Wik17a], for a description of a very pretty proof of this fact). In 2 In fact, in a very precise sense which we will not discuss in this book, the longer you play a game like this, the more you can expect there will be short-term, but very large, wins and losses. 74 4. PROBABILITY THEORY addition, the chance of some random process producing a real number that is exactly equal to some particular value really is zero: for two real numbers to be precisely equal requires infinite accuracy ... think of all of those decimal digits, marching off in orderly rows to infinity, which must match between the two numbers. Rather than a distribution, we do the following: D EFINITION 4.3.13. Let X be a continuous random variable whose values are the real interval [xmin , xmax ], where either xmin or xmax or both may be ∞. A [probability] density function for X is a function f (x) defined for x in [xmin , xmax ], meaning it is a curve with one y value for each x in that interval, with the property that ( the area in the xy-plane above the x-axis, below P (a < X < b) = . the curve y = f (x) and between x = a and x = b. Graphically, what is going on here is Because of what we know about probabilities, the following is true (and fairly easy to prove): FACT 4.3.14. Suppose f (x) is a density function for the continuous RV X defined on the real interval [xmin , xmax ]. Then • For all x in [xmin , xmax ], f (x) ≥ 0. • The total area under the curve y = f (x), above the x-axis, and between x = xmin and x = xmax is 1. If we want the idea of picking a real number on the interval [xmin , xmax ] at random, where at random means that all numbers have the same chance of being picked (along the lines of fair in Definition 4.1.20, the height of the density function must be the same at all x. In other words, the density function f (x) must be a constant c. In fact, because of the 1 above Fact 4.3.14, that constant must have the value xmax −x min . There is a name for this: 4.3. RANDOM VARIABLES 75 D EFINITION 4.3.15. The uniform distribution on [xmin , xmax ] is the distribution for the continuous RV whose values are the interval [xmin , xmax ] and whose density function 1 is the constant function f (x) = xmax −x min . E XAMPLE 4.3.16. Suppose you take a bus to school every day and because of a chaotic home life (and, let’s face it, you don’t like mornings), you get to the bus stop at a pretty nearly perfectly random time. The bus also doesn’t stick perfectly to its schedule – but it is guaranteed to come at least every 30 minutes. What this adds up to is the idea that your waiting time at the bus stop is a uniformly distributed RV on the interval [0, 30]. If you wonder one morning how likely it then is that you will wait for less than 10 minutes, you can simply compute the area of the rectangle whose base is the interval [0, 10] 1 on the x-axis and whose height is 30 , which will be 1 1 P (0 < X < 10) = base · height = 10 · = . 30 3 A picture which should clarify this is where the area of the shaded region represents the probability of having a waiting time from 0 to 10 minutes. One technical thing that can be confusing about continuous RVs and their density func- tions is the question of whether we should write P (a < X < b) or P (a ≤ X ≤ b). But if you think about it, we really have three possible events here: A = {outcomes such that X = a}, M = {outcomes such that a < X < b}, and B = {outcomes such that X = b} . Since X always takes on exactly one value for any particular outcome, there is no overlap between these events: they are all disjoint. That means that P (A ∪ M ∪ B) = P (A) + P (M) + P (B) = P (M) where the last equality is because, as we said above, the probability of a continuous RV taking on exactly one particular value, as it would in events A and B, is 0. The same would be true if we added merely one endpoint of the interval (a, b). To summarize: 76 4. PROBABILITY THEORY FACT 4.3.17. If X is a continuous RV with values forming the interval [xmin , xmax ] and a and b are in this interval, then P (a < X < b) = P (a < X ≤ b) = P (a ≤ X < b) = P (a ≤ X ≤ b) . As a consequence of this fact, some authors write probability formulæ about continuous RVs with “ < ” and some with “ ≤ ” and it makes no difference. Let’s do a slightly more interesting example than the uniform distribution: E XAMPLE 4.3.18. Suppose you repeatedly throw darts at a dartboard. You’re not a machine, so the darts hit in different places every time and you think of this as a repeatable random experiment whose outcomes are the locations of the dart on the board. You’re interested in the probabilities of getting close to the center of the board, so you decide for each experimental outcome (location of a dart you threw) to measure its distance to the center – this will be your RV X. Being good at this game, you hit near the center more than near the edge and you never completely miss the board, whose radius is 10cm– so X is more likely to be near 0 than near 10, and it is never greater than 10. What this means is that the RV has values forming the interval [0, 10] and the density function, defined on the same interval, should have its maximum value at x = 0 and should go down to the value 0 when x = 10. You decide to model this situation with the simplest density function you can think of that has the properties we just noticed: a straight line from the highest point of the density function when x = 0 down to the point (10, 0). The figure that will result will be a triangle, and since the total area must be 1 and the base is 10 units long, the height must be .2 units. [To get that, we solved the equation 1 = 12 bh = 21 10h = 5h for h.] So the graph must be 1 and the equation of this linear density function would be y = − 50 x + .2 [why? – think about the slope and y-intercept!]. To the extent that you trust this model, you can now calculate the probabilities of events like, for example, “hitting the board within that center bull’s-eye of radius 1.5cm,” which probability would be the area of the shaded region in this graph: 4.3. RANDOM VARIABLES 77 The upper-right corner of this shaded region is at x-coordinate 1.5 and is on the line, so its 1 y-coordinate is − 50 1.5 + .2 = .17 . Since the region is a trapezoid, its area is the distance between the two parallel sides times the average of the lengths of the other two sides, giving .2 + .17 P (0 < X < 1.5) = 1.5 · = .2775 . 2 In other words, the probability of hitting the bull’s-eye, assuming this model of your dart- throwing prowess, is about 28%. If you don’t remember the formula for the area of a trapezoid, you can do this problem another way: compute the probability of the complementary event, and then take one minus that number. The reason to do this would be that the complementary event corresponds to the shaded region here which is a triangle! Since we surely do remember the formula for the area of a triangle, we find that 1 1 P (1.5 < X < 10) = bh = .17 · 8.5 = .7225 2 2 and therefore P (0 < X < 1.5) = 1 − P (1.5 < X < 10) = 1 − .7225 = .2775. [It’s nice that we got the same number this way, too!] 4.3.5. The Normal Distribution. We’ve seen some examples of continuous RVs, but we have yet to meet the most important one of all. D EFINITION 4.3.19. The Normal distribution with mean µX and standard deviation σX is the continuous RV which takes on all real values and is governed by the probability density function (x−µX )2 1 − ρ(x) = √ 2 e 2σX 2 . 2σX π 78 4. PROBABILITY THEORY If X is a random variable which follows this distribution, then we say that X is Normally distributed with mean µX and standard deviation σX or, in symbols, X is N(µX , σX ). [More technical works also call this the Gaussian distribution, named after the great mathematician Carl Friedrich Gauss. But we will not use that term again in this book after this sentence ends.] The good news about this complicated formula is that we don’t really have to do any- thing with it. We will collect some properties of the Normal distribution which have been derived from this formula, but these properties are useful enough, and other tools such as modern calculators and computers which can find specific areas we need under the graph of y = ρ(x), that we won’t need to work directly with the above formula for ρ(x) again. It is nice to know that N(µX , σX ) does correspond to a specific, known density function, though, isn’t it? It helps to start with an image of what the Normal distribution looks like. Here is the density function for µX = 17 and σX = 3: Now let’s collect some of these useful facts about the Normal distributions. FACT 4.3.20. The density function ρ for the Normal distribution N(µX , σX ) is a positive function for all values of x and the total area under the curve y = ρ(x) is 1. This simply means that ρ is a good candidate for the probability density function for some continuous RV. FACT 4.3.21. The density function ρ for the Normal distribution N(µX , σX ) is unimodal with maximum at x-coordinate µX . 4.3. RANDOM VARIABLES 79 This means that N(µX , σX ) is a possible model for an RV X which tends to have one main, central value, and less often has other values farther away. That center is at the location given by the parameter µX , so wherever we want to put the center of our model for X, we just use that for µX . FACT 4.3.22. The density function ρ for the Normal distribution N(µX , σX ) is is sym- metric when reflected across the line x = µX . This means that the amount X misses its center, µX , tends to be about the same when it misses above µX and when it misses below µX . This would correspond to situations were you hit as much to the right as to the left of the center of a dartboard. Or when randomly picked people are as likely to be taller than the average height as they are to be shorter. Or when the time it takes a student to finish a standardized test is as likely to be less than the average as it is to be more than the average. Or in many, many other useful situations. FACT 4.3.23. The density function ρ for the Normal distribution N(µX , σX ) has has tails in both directions which are quite thin, in fact get extremely thin as x → ±∞, but never go all the way to 0. This means that N(µX , σX ) models situations where the amount X deviates from its average has no particular cut-off in the positive or negative direction. So you are throwing darts at a dart board, for example, and there is no way to know how far your dart may hit to the right or left of the center, maybe even way off the board and down the hall – although that may be very unlikely. Or perhaps the time it takes to complete some task is usually a certain amount, but every once and a while it might take much more time, so much more that there is really no natural limit you might know ahead of time. At the same time, those tails of the Normal distribution are so thin, for values far away from µX , that it can be a good model even for a situation where there is a natural limit to the values of X above or below µX . For example, heights of adult males (in inches) in the United States are fairly well approximated by N(69, 2.8), even though heights can never be less than 0 and N(69, 2.8) has an infinitely long tail to the left – because while that tail is non-zero all the way as x → −∞, it is very, very thin. All of the above Facts are clearly true on the first graph we saw of a Normal distribution density function. FACT 4.3.24. The graph of the density function ρ for the Normal distribution N(µX , σX ) has a taller and narrower peak if σX is smaller, and a lower and wider peak if σX is larger. This allows the statistician to adjust how much variation there typically is in a normally distributed RV: By making σX small, we are saying that an RV X which is N(µX , σX ) is very likely to have values quite close to its center, µX . If we make σX large, however, X is more likely to have values all over the place – still, centered at µX , but more likely to wander farther away. 80 4. PROBABILITY THEORY Let’s make a few versions of the graph we saw for ρ when µX was 17 and σX was 3, but now with different values of σX . First, if σX = 1, we get If, instead, σX = 5, then we get Finally, let’s superimpose all of the above density functions on each other, for one, combined graph: 4.3. RANDOM VARIABLES 81 This variety of Normal distributions (one for each µX and σX ) is a bit bewildering, so traditionally, we concentrate on one particularly nice one. D EFINITION 4.3.25. The Normal distribution with mean µX = 0 and standard deviation σX = 1 is called the standard Normal distribution and an RV [often written with the variable Z] that is N(0, 1) is described as a standard Normal RV. Here is what the standard Normal probability density function looks like: 82 4. PROBABILITY THEORY One nice thing about the standard Normal is that all other Normal distributions can be related to the standard. FACT 4.3.26. If X is N(µX , σX ), then Z = (X − µX )/σX is standard Normal. This has a name. D EFINITION 4.3.27. The process of replacing a random variable X which is N(µX , σX ) with the standard normal RV Z = (X − µX )/σX is called standardizing a Normal RV. It used to be that standardization was an important step in solving problems with Nor- mal RVs. A problem would be posed with information about some data that was modelled by a Normal RV with given mean µX and standardization σX . Then questions about proba- bilities for that data could be answered by standardizing the RV and looking up values in a single table of areas under the standard Normal curve. Today, with electronic tools such as statistical calculators and computers, the standard- ization step is not really necessary. E XAMPLE 4.3.28. As we noted above, the heights of adult men in the United States, when measured in inches, give a RV X which is N(69, 2.8). What percentage of the population, then, is taller than 6 feet? First of all, the frequentist point of view on probability tells us that what we are inter- ested in is the probability that a randomly chosen adult American male will be taller than 6 feet – that will be the same as the percentage of the population this tall. In other words, we must find the probability that X > 72, since in inches, 6 feet becomes 72. As X is a continuous RV, we must find the area under its density curve, which is the ρ for N(69, 2.8), between 72 and ∞. That ∞ is a little intimidating, but since the tails of the Normal distribution are very thin, we can stop measuring area when x is some large number and we will have missed only a very tiny amount of area, so we will have a very good approximation. Let’s therefore find the area under ρ from x = 72 up to x = 1000. This can be done in many ways: • With a wide array of online tools – just search for “online normal probability calculator.” One of these yields the value .142. • With a TI-8x calculator, by typing normalcdf(72, 1000, 69, 2.8) which yields the value .1419884174. The general syntax here is normalcdf(a, b, µX , σX ) to find P (a < X < b) when X is N(µX , σX ). Note you get normalcdf by typing 2ND → V ARS → 2 4.3. RANDOM VARIABLES 83 • Spreadsheets like LibreOffice Calc and Microsoft Excel will compute this by putting the following in a cell =1-NORM.DIST(72, 69, 2.8, 1) giving the value 0.1419883859. Here we are using the command NORM.DIST(b, µX , σX , 1) which computes the area under the density function for N(µX , σX ) from −∞ to b. [The last input of “1” to NORM.DIST just tells it that we want to compute the area under the curve. If we used “0” instead, it would simple tell us the particular value of ρ(b), which is of very direct little use in probability calculations.] Therefore, by doing 1 − NORM.DIST (72, 69, 2.8, 1), we are taking the total area of 1 and subtracting the area to the left of 72, yielding the area to the right, as we wanted. Therefore, if you want the area between a and b on an N(µX , σX ) RV using a spreadsheet, you would put =NORM.DIST(b, µX , σX , 1) - NORM.DIST(a, µX , σX , 1) in a cell. While standardizing a non-standard Normal RV and then looking up values in a table is an old-fashioned method that is tedious and no longer really needed, one old technique still comes in handy some times. It is based on the following: FACT 4.3.29. The 68-95-99.7 Rule: Let X be an N(µX , σX ) RV. Then some special values of the area under the graph of the density curve ρ for X are nice to know: • The area under the graph of ρ from x = µX − σX to x = µX + σX , also known as P (µX − σX < X < µX + σX ), is .68 . • The area under the graph of ρ from x = µX − 2σX to x = µX + 2σX , also known as P (µX − 2σX < X < µX + 2σX ), is .95 . • The area under the graph of ρ from x = µX − 3σX to x = µX + 3σX , also known as P (µX − 3σX < X < µX + 3σX ), is .997 . This is also called The Empirical Rule by some authors. Visually3: 3 By Dan Kernler - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index. php?curid=36506025 . 84 4. PROBABILITY THEORY In order to use the 68-95-99.7 Rule in understanding a particular situation, it is helpful to keep an eye out for the numbers that it talks about. Therefore, when looking at a problem, one should notice if the numbers µX + σX , µX − σX , µX + 2σX , µX − 2σX , µX + 3σX , or µX − 3σX are ever mentioned. If so, perhaps this Rule can help. E XAMPLE 4.3.30. In Example 4.3.28, we needed to compute P (X > 72) where X was known to be N(69, 2.8). Is 72 one of the numbers for which we should be looking, to use the Rule? Well, it’s greater than µX = 69, so we could hope that it was µX + σX , µX + 2σX , or µX + 3σX . But values are µX + σX = 69 + 2.8 = 71.8, µX + 2σX = 69 + 5.6 = 74.6, and µX + 3σX = 69 + 8.4 = 77.4, none of which is what we need. Well, it is true that 72 ≈ 71.8, so we could use that fact and accept that we are only getting an approximate answer – an odd choice, given the availability of tools which will give us extremely precise answers, but let’s just go with it for a minute. Let’s see, the above Rule tells us that P (66.2 < X < 71.8) = P (µX − σX < X < µX + σX ) = .68 . Now since the total area under any density curve is 1, P (X < 66.2 or X > 71.8) = 1 − P (66.2 < X < 71.8) = 1 − .68 = .32 . 4.3. RANDOM VARIABLES 85 Since the event “X < 66.2” is disjoint from the event “X > 71.8” (X only takes on one value at a time, so it cannot be simultaneously less than 66.2 and greater than 71.8), we can use the simple rule for addition of probabilities: .32 = P (X < 66.2 or X > 71.8) = P (X < 66.2) + P (X > 71.8) . Now, since the density function of the Normal distribution is symmetric around the line x = µX , the two terms on the right in the above equation are equal, which means that 1 1 P (X > 71.8) = (P (X < 66.2) + P (X > 71.8)) = .32 = .16 . 2 2 It might help to visualize the symmetry here as the equality of the two shaded areas in the following graph Now, using the fact that 72 ≈ 71.8, we may say that P (X > 72) ≈ P (X > 71.8) = .16 which, since we know that in fact P (X > 72) = .1419883859, is not a completely terrible approximation. E XAMPLE 4.3.31. Let’s do one more computation in the context of the heights of adult American males, as in the immediately above Example 4.3.30, but now one in which the 68-95-99.7 Rule gives a more precise answer. So say we are asked this time what proportion of adult American men are shorter than 63.4 inches. Why that height, in particular? Well, it’s how tall archaeologists have deter- mined King Tut was in life. [No, that’s made up. It’s just a good number for this problem.] Again, looking through the values µX ± σX , µX ± 2σX , and µX ± 3σX , we notice that 63.4 = 69 − 5.6 = µX − 2σX . Therefore, to answer what fraction of adult American males are shorter than 63.4 inches amounts to asking what is the value of P (X < µX − 2σX ). 86 4. PROBABILITY THEORY What we know about µX ± 2σX is that the probability of X being between those two values is P (µX − 2σX < X < µX + 2σX ) = .95 . As in the previous Example, the complementary event to “µX − 2σX < X < µX + 2σX ,” which will have probability .05, consists of two pieces “X < µX − 2σX ” and “X > µX + 2σX ,” which have the same area by symmetry. Therefore P (X < 63.4) = P (X < µX − 2σX ) 1 = [P (X < µX − 2σX ) + P (X > µX + 2σX )] 2 1 = P (X < µX − 2σX or X > µX + 2σX ) since they’re disjoint 2 1 = P ((µX − 2σX < X < µX + 2σX )c ) 2 1 = [1 − P (µX − 2σX < X < µX + 2σX )] by prob. for complements 2 1 = .05 2 = .025 Just the way finding the particular X values µX ± σX , µX ± 2σX , and µX ± 3σX in a particular situation would tell us the 68-95-99.7 Rule might be useful, so also would finding the probability values .68, .95, 99.7, or their complements .32, .05, or .003, – or even half of one of those numbers, using the symmetry. E XAMPLE 4.3.32. Continuing with the scenario of Example 4.3.30, let us now figure out what is the height above which there will only be .15% of the population. Notice that .15%, or the proportion .0015, is not one of the numbers in the 68-95-99.7 Rule, nor is it one of their complements – but it is half of one of the complements, being half of .003 . Now, .003 is the complementary probability to .997, which was the probability in the range µX ± 3σX . As we have seen already (twice), the complementary area to that in the region between µX ± 3σX consists of two thin tails which are of equal area, each of these areas being 12 (1−.997) = .0015 . This all means that the beginning of that upper tail, above which value lies .15% of the population, is the X value µX + 3σX = 68 + 3 · 2.8 = 77.4 . Therefore .15% of adult American males are taller than 77.4 inches. EXERCISES 87 Exercises E XERCISE 4.1. A basketball player shoots four free throws, and you write down the sequence of hits and misses. Write down the sample space for thinking of this whole thing as a random experiment. In another game, a basketball player shoots four free throws, and you write down the number of baskets she makes. Write down the sample space for this different random experiment. E XERCISE 4.2. You take a normal, six-sided die, paint over all the sides, and then write the letter A on all six sides. You then roll the die. What is the sample space of this experiment? Also, list all the possible events for this experiment. [Hint: it may help to look at Example 4.1.9.] Now you paint it over again, and write A on half the sides and B on the other half. Again, say what is the sample space and list all possible events. One more time you paint over the sides, then write A on one third of the faces, B on one third of the other faces, and C on the remaining third. Again, give the sample space and all events. Make a conjecture about how many events there will be if the sample space has n outcomes in it. E XERCISE 4.3. Describe a random experiment whose sample space will be the set of all points on the (standard, 2-dimensional, xy-) plane. E XERCISE 4.4. The most common last [family] name in the world seems to be Wang [or the variant Wong]. Approximately 1.3% of the global population has this last name. The most common first name in the world seems to be Mohammad [or one of several variants]. Some estimates suggest that perhaps as many as 2% of the global population has this first name. Can you tell, from the above information, what percentage of the world population has the name “Mohammad Wang?” If so, why and what would it be? If not, why not, and can you make any guess about what that percentage would be, anyway? [Hint: think of all the above percentages as probabilities, where the experiment is pick- ing a random person on Earth and asking their name. Carefully describe some events for this experiment, relevant to this problem, and say what their probabilities are. Tell how combining events will or will not compute the probability of the desired event, correspond- ing to the desired percentage.] [Note: don’t bet on the numbers given in this problem being too accurate – they might be, but there is a wide range of published values for them in public information from dif- ferent sources, so probably they are only a very crude approximation.] 88 4. PROBABILITY THEORY E XERCISE 4.5. Suppose that when people have kids, the chance of having a boy or a girl is the same. Suppose also that the sexes of successive children in the same family are independent. [Neither of these is exactly true in real life, but let’s pretend for this problem.] The Wang family has two children. If we think of the sexes of these children as the result of a random experiment, what is the sample space? Note that we’re interested in birth order as well, so that should be apparent from the sample space. What are the probabilities of each of the outcomes in your sample space? Why? Now suppose we know that at least one of the Wang children is a boy. Given this information, what is the probability that the Wangs have two boys? Suppose instead that we know that the Wangs’ older child is a boy. What is the proba- bility, given this different information, that both Wang children are boys? To solve this, clearly define events in words and with symbols, compute probabilities, and combine these to get the desired probability. Explain everything you do, of course. E XERCISE 4.6. Imagine you live on a street with a stop light at both ends of the block. You watch cars driving down the street and notice which ones have to stop at the 1st and/or 2nd light (or none). After counting cars and stops for a year, you have seen what a very large number – call it N – of cars did. Now imagine you decide to think about the experiment “pick a car on this street from the last year at random and notice at which light or lights it has to stop.” Let A be the event “the car had to stop at the 1st light” and B be the event “the car had to stop at the 2nd light.” What else would you have to count, over your year of data collection, to estimate the probabilities of A and of B? Pick some numbers for all of these variables and show what the probabilities would then be. Make a Venn diagram of this situation. Label each of the four connected regions of this diagram (the countries, if this were a map) with a number from 1 to 4 , then provide a key which gives, for each of these numbered regions, both a formula in terms of A, B, unions, intersections, and/or complements, and then also a description entirely in words which do not mention A or B or set operations at all. Then put a decimal number in each of the regions indicating the probability of the corresponding event. Wait – for one of the regions, you can’t fill in the probability yet, with the information you’ve collected so far. What else would you have had to count over the data-collection year to estimate this probability? Make up a number and show what the corresponding probability would then be, and add that number to your Venn diagram. Finally, using the probabilities you have chosen, are the events A and B independent? Why or why not? Explain in words what this means, in this context. EXERCISES 89 E XERCISE 4.7. Here is a table of the prizes for the EnergyCube Lottery: Prize Odds of winning $1,000,000 1 in 12,000,000 $50,000 1 in 1,000,000 $100 1 in 10,000 $7 1 in 300 $4 1 in 25 We want to transform the above into the [probability] distribution of a random variable X. First of all, let’s make X represent the net gain a Lottery player would have for the various outcomes of playing – note that the ticket to play costs $2. How would you modify the above numbers to take into account the ticket costs? Next, notice that the above table gives winning odds, not probabilities. How will you compute the probabilities from those odds? Recall that saying something has odds of “1 in n” means that it tends to happen about once out of n runs of the experiment. You might use the word frequentist somewhere in your answer here. Finally, something is missing from the above table of outcomes. What prize – actu- ally the most common one! – is missing from the table, and how will you figure out its probability? After giving all of the above explanations, now write down the full, formal, probability distribution for this “net gain in EnergyCube Lottery plays” random variable, X. In this problem, some of the numbers are quite small and will disappear entirely if you round them. So use a calculator or computer to compute everything here and keep as much accuracy as your device shows for each step of the calculation. E XERCISE 4.8. Continuing with the same scenario as in the previous Exercise 4.7, with the EnergyCube Lottery: What would be your expectation of the average gain per play of this Lottery? Explain fully, of course. So if you were to play every weekday for a school year (so: five days a week for the 15 weeks of each semester, two semesters in the year), how much would you expect to win or lose in total? Again, use as much accuracy as your computational device has, at every step of these calculations. E XERCISE 4.9. Last problem in the situation of the above Exercise 4.7 about the En- ergyCube Lottery: Suppose your friend plays the lottery and calls you to tell you that she won ... but her cell phone runs out of charge in the middle of the call, and you don’t know how much she won. Given the information that she won, what is the probability that she won more than $1,000? Continue to use as much numerical accuracy as you can. 90 4. PROBABILITY THEORY E XERCISE 4.10. Let’s make a modified version of Example 4.3.18. You are again throwing darts at a dartboard, but you notice that you are very left-handed so your throws pull to the right much more than they pull to the left. What this means is that it is not a very good model of your dart throws just to notice how far they are from the center of the dartboard, it would be better to notice the x-coordinate of where the dart hits, measuring (in cm) with the center of the board at x location 0. This will be your new choice of RV, which you will still call X. You throw repeatedly at the board, measure X, and find out that you never hit more than 10cm to the right of the center, while you are more accurate to the left and never hit more than 5cm in that direction. You do hit the middle (X = 0) the most often, and you guess that the probability decreases linearly to those edges where you never hit. Explain why your X is a continuous RV, and what its interval [xmin , xmax ] of values is. Now sketch the graph of the probability density function for X. [Hint: it will be a triangle, with one side along the interval of values [xmin , xmax ] on the x-axis, and its maxi- mum at the center of the dartboard.] Make sure that you put tick marks and numbers on the axes, enough so that the coordinates of the corners of the triangular graph can be seen eas- ily. [Another hint: it is a useful fact that the total area under the graph of any probability density function is 1.] What is the probability that your next throw will be in the bull’s-eye, whose radius, remember, is 1.5cm and which therefore stretches from x coordinate −1.5 to x-coordinate 1.5? E XERCISE 4.11. Here’s our last discussion of dartboards [maybe?]: One of the prob- lems with the probability density function approaches from Example 4.3.18 and Exer- cise 4.10 is the assumption that the functions were linear (at least in pieces). It would be much more sensible to assume they were more bell-shaped, maybe like the Normal distribution. Suppose your friend Mohammad Wang is an excellent dart-player. He throws at a board and you measure the x-coordinate of where the dart goes, as in Exercise 4.10 with the center corresponding to x = 0. You notice that his darts are rarely – only 5% of the time in total! – more than 5cm from the center of the board. Fill in the blanks: “MW’s dart hits’ x-coordinates are an RV X which is Normally distributed with mean µX = and standard deviation σX = .” Explain, of course. How often does MW completely miss the dartboard? Its radius is 10cm. How often does he hit the bull’s-eye? Remember its radius is 1.5cm, meaning that it stretches from x coordinate −1.5 to x-coordinate 1.5. CHAPTER 5 Bringing Home the Data In this chapter, we start to get very practical on the matter of tracking down good data in the wild and bringing it home. This is actually a very large and important subject – there are entire courses and books on Experimental Design, Survey Methodology, and Research Methods specialized for a range of particular disciplines (medicine, psychology, sociology, criminology, manufacturing reliability, etc.) – so in this book we will only give a broad introduction to some of the basic issues and approaches. The first component of this introduction will give several of the important definitions for experimental design in the most direct, simplest context: collecting sample data in an attempt to understand a single number about an entire population. As we have mentioned before, usually a population is too large or simply inaccessible and so to determine an im- portant feature of a population of interest, a researcher must use the accessible, affordable data of a sample. If this approach is to work, the sample must be chosen carefully, so as to avoid the dreaded bias. The basic structure of such studies, the meaning of bias, and some of the methods to select bias-minimizing samples, are the subject of the first section of this chapter. It is more complicated to collect data which will give evidence for causality, for a causal relationship between two variables under study. But we are often interested in such relationships – which drug is a more effective treatment for some illness, what advertise- ment will induce more people to buy a particular product, or what public policy leads to the strongest economy. In order to investigate causal relationships, it is necessary not merely to observe, but to do an actual experiment; for causal questions about human subjects, the gold standard is a randomized, placebo-controlled, double-blind experiment, sometimes called simply a randomized, controlled trial [RCT], which we describe in the second section. There is something in the randomized, controlled experiment which makes many peo- ple nervous: those in the control group are not getting what the experimenter likely thinks is the best treatment. So, even though society as a whole may benefit from the knowledge we get through RCTs, it almost seems as if some test subjects are being mistreated. While the scientific research community has come to terms with this apparent injustice, there are definitely experiments which could go too far and cross an important ethical lines. In fact, history has shown that a number of experiments have actually been done which we now consider to be clearly unethical. It is therefore important to state clearly some ethical 91 92 5. BRINGING HOME THE DATA guidelines which future investigations can follow in order to be confident to avoid mistreat- ment of test subjects. One particular set of such guidelines for ethical experimentation on human subjects is the topic of the third and last section of this chapter. 5.1. STUDIES OF A POPULATION PARAMETER 93 5.1. Studies of a Population Parameter Suppose we are studying some population, and in particular a variable defined on that population. We are typically interested in finding out the following kind of characteristic of our population: D EFINITION 5.1.1. A [population] parameter is a number which is computed by knowing the values of a variable for every individual in the population. E XAMPLE 5.1.2. If X is a quantitative variable on some population, the population mean µX of X is a population parameter – to compute this mean, you need to add together the values of X for all of individuals in the population. Likewise, the population standard deviation σX of X is another parameter. For example, we asserted in Example 4.3.28 that the heights of adult American men are N(69, 2.8). Both the 69 and 2.8 are population parameters here. E XAMPLE 5.1.3. If, instead, X were a categorical variable on some population, then the relative frequency (also called the population proportion) of some value A of X – the fraction of the population that has that value – is another population parameter. After all, to compute this fraction, you have to look at every single individual in the population, all N of them, say, and see how many of them, say NA , make the X take the value A, then compute the relative frequency NA /N. Sometimes one doesn’t have to look at the specific individuals and compute that fraction nA /N to find a population proportion. For example, in Example 4.3.28, we found that 14.1988% of adult American men are taller than 6 feet, assuming, as stated above, that adult American men’s heights are distributed like N(69, 2.8) – using, notice, those parameters µX and σX of the height distribution, for which the entire population must have been examined. What this means is that the relative frequency of the value “yes” for the categorical variable “is this person taller than 6 feet?” is .141988. This relative frequency is also a parameter of the same population of adult American males. Parameters must be thought of as fixed numbers, out there in the world, which have a single, specific value. However, they are very hard for researchers to get their hands on, since to compute a parameter, the variable values for the entire population must be measured. So while the parameter is a single, fixed value, usually that value is unknown. What can (and does change) is a value coming from a sample. D EFINITION 5.1.4. A [sample] statistic is a number which is computed by knowing the values of a variable for the individuals from only a sample. E XAMPLE 5.1.5. Clearly, if we have a population and quantitative variable X, then any time we choose a sample out of that population, we get a sample mean and sample standard deviation Sx , both of which are statistics. 94 5. BRINGING HOME THE DATA Similarly, if we instead have a categorical variable Y on some population, we take a sample of size n out of the population and count how many individuals in the sample – say nA – have some value A for their value of Y , then the nA /n is a statistic (which is also called the sample proportion and frequently denoted pb ). Two different researchers will choose different samples and so will almost certainly have different values for the statistics they compute, even if they are using the same formula for their statistic and are looking at the same population. Likewise, one researcher taking repeated samples from the same population will probably get different values each time for the statistics they compute. So we should think of a statistic as an easy, accessible number, changing with each sample we take, that is merely an estimate of the thing we want, the parameter, which is one, fixed number out in the world, but hidden from out knowledge. So while getting sample statistics is practical, we need to be careful that they are good estimates of the corresponding parameters. Here are some ways to get better estimates of this kind: (1) Pick a larger sample. This seems quite obvious, because the larger is the sample, the closer it is to being the whole population and so the better its approximating statistics will estimate the parameters of interest. But in fact, things are not really quite so simple. In many very practical situations, it would be completely infeasi- ble to collect sample data on a sample which was anything more than a miniscule part of the population of interest. For example, a national news organization might want to survey the American population, but it would be entirely prohibitive to get more than a few thousand sample data values, out of a population of hundreds of millions – so, on the order of tenths of a percent. Fortunately, there is a general theorem which tells us that, in the long run, one particular statistic is a good estimator of one particular parameter: FACT 5.1.6. The Law of Large Numbers: Let X be a quantitative variable on some population. Then as the sizes of samples (each made up of individuals chosen randomly and independently from the population) get bigger and bigger, the corresponding sample means x get closer and closer to the population mean µX . (2) Pick a better statistic. It makes sense to use the sample mean as a statistic to estimate the population mean and the sample proportion to estimate the population proportion. But it is less clear where the somewhat odd formula for the sample standard deviation came from – remember, it differs from the population standard deviation by having an n − 1 in the denominator instead of an n. The reason, whose proof is too technical to be included here, is that the formula we gave for SX is a better estimator for σX than would have be the version which simply had the same n in the denominator. 5.1. STUDIES OF A POPULATION PARAMETER 95 In a larger sense, “picking a better statistic” is about getting higher quality estimates from your sample. Certainly using a statistic with a clever formula is one way to do that. Another is to make sure that your data is of the highest quality possible. For example, if you are surveying people for their opinions, the way you ask a question can have enormous consequences in how your subjects answer: “Do you support a woman’s right to control her own body and her reproduction?” and “Do you want to protect the lives of unborn children?” are two heavy-handed approaches to asking a question about abortion. Collectively, the impacts of how a question is asked are called wording effects, and are an important topic social scientists must understand well. (3) Pick a better sample. Sample quality is, in many ways, the most important and hardest issue in this kind of statistical study. What we want, of course, is a sample for which the statistic(s) we can compute give good approximations for the pa- rameters in which we are interested. There is a name for this kind of sample, and one technique which is best able to create these good samples: randomness. D EFINITION 5.1.7. A sample is said to be representative of its population if the values of its sample means and sample proportions for all variables relevant to the subject of the research project are good approximations of the corresponding population means and proportions. It follows almost by definition that a representative sample is a good one to use in the process of, as we have described above, using a sample statistic as an estimate of a population parameter in which you are interested. The question is, of course, how to get a representative sample. The answer is that it is extremely hard to build a procedure for choosing samples which guarantees representative samples, but there is a method – using randomness – which at least can reduce as much as possible one specific kind of problem samples might have. D EFINITION 5.1.8. Any process in a statistical study which tends to produce results which are systematically different from the true values of the population parameters under investigation is called biased. Such a systematic deviation from correct values is called bias. The key word in this definition is systematically: a process which has a lot of variation might be annoying to use, it might require the researcher to collect a huge amount of data to average together, for example, in order for the estimate to settle down on something near the true value – but it might nevertheless not be biased. A biased process might have less variation, might seem to get close to some particular value very quickly, with little data, but would never give the correct answer, because of the systematic deviation it contained. 96 5. BRINGING HOME THE DATA The hard part of finding bias is to figure out what might be causing that systematic deviation in the results. When presented with a sampling method for which we wish to think about sources of possible bias, we have to get creative. E XAMPLE 5.1.9. In a democracy, the opinion of citizens about how good a job their elected officials are doing seems like an interesting measure of the health of that democracy. At the time of this writing, approximately two months after the inauguration of the 45th president of the United States, the widely respected Gallup polling organization reports [Gal17] that 56% of the population approve of the job the president is doing and 40% disapprove. [Presumably, 4% were neutral or had no opinion.] According to the site from which these numbers are taken, “Gallup tracks daily the percentage of Americans who approve or dis- approve of the job Donald Trump is doing as president. Daily results are based on telephone interviews with approximately 1,500 national adults....” Presumably, Gallup used the sample proportion as an estimator computed with the re- sponses from their sample of 1500 adults. So it was a good statistic for the job, and the sample size is quite respectable, even if not a very large fraction of the entire adult Amer- ican population, which is presumably the target population of this study. Gallup has the reputation for being a quite neutral and careful organization, so we can also hope that the way they worded their questions did not introduce any bias. A source of bias that does perhaps cause some concern here is that phrase “telephone interviews.” It is impossible to do telephone interviews with people who don’t have tele- phones, so there is one part of the population they will miss completely. Presumably, also, Gallup knew that if they called during normal working days and hours, they would not get working people at home or even on cell phones. So perhaps they called also, or only, in the evenings and on weekends – but this approach would tend systematically to miss people who had to work very long and/or late hours. So we might worry that a strategy of telephone interviews only would be biased against those who work the longest hours, and those people might tend to have similar political views. In the end, that would result in a systematic error in this sampling method. Another potential source of bias is that even when a person is able to answer their phone, it is their choice to do so: there is little reward in taking the time to answer an opinion survey, and it is easy simply not to answer or to hang up. It is likely, then, that only those who have quite strong feelings, either positive or negative, or some other strong personal or emotional reason to take the time, will have provided complete responses to this telephone survey. This is potentially distorting, even if we cannot be sure that the effects are systematically in one direction or the other. 5.1. STUDIES OF A POPULATION PARAMETER 97 [Of course, Gallup pollsters have an enormous amount of experience and have presum- ably thought the above issues through completely and figure out how to work around it – but we have no particular reason to be completely confident in their results other than our faith in their reputation, without more details about what work-arounds they used. In science, doubt is always appropriate.] One of the issues we just mentioned about the Gallup polling of presidential approval ratings has its own name: D EFINITION 5.1.10. A sample selection method that involves any substantial choice of whether to participate or not suffers from what is called voluntary sample bias. Voluntary sample bias is incredibly common, and yet is such a strong source of bias that it should be taken as a reason to disregard completely the supposed results of any study that it affects. Volunteers tend to have strong feelings that drive them to participate, which can have entirely unpredictable but systematic distorting influence on the data they provide. Web-based opinion surveys, numbers of thumbs-up or -down or of positive or negative comments on a social media post, percentages of people who call in to vote for or against some public statement, etc., etc. – such widely used polling methods produce nonsensical results which will be instantly rejected by anyone with even a modest statistical knowledge. Don’t fall for them! We did promise above one technique which can robustly combat bias: randomness. Since bias is based on a systematic distortion of data, any method which completely breaks all systematic processes in, for example, sample selection, will avoid bias. The strongest such sampling method is as follows. D EFINITION 5.1.11. A simple random sample [SRS] is a sample of size n, say, chosen from a population by a method which produces all samples of size n from that population with equal probability. It is oddly difficult to tell if a particular sample is an SRS. Given just a sample, in fact, there is no way to tell – one must ask to see the procedure that had been followed to make that sample and then check to see if that procedure would produce any subset of the population, of the same size as the sample, with equal probability. Often, it is easier to see that a sampling method does not make SRSs, by finding some subsets of the population which have the correct size but which the sampling method would never choose, meaning that they have probability zero of being chosen. That would mean some subsets of the correct size would have zero probability and others would have a positive probability, meaning that not all subsets of that size would have the same probability of being chosen. 98 5. BRINGING HOME THE DATA Note also that in an SRS it is not that every individual has the same probability of being chosen, it must be that every group of individuals of the size of the desired sample has the same probability of being chosen. These are not the same thing! E XAMPLE 5.1.12. Suppose that on Noah’s Ark, the animals decide they will form an advisory council consisting of an SRS of 100 animals, to help Noah and his family run a tight ship. So a chimpanzee (because it has good hands) puts many small pieces of paper in a basket, one for each type of animal on the Ark, with the animal’s name written on the paper. Then the chimpanzee shakes the basket well and picks fifty names from the basket. Both members of the breeding pair of that named type of animal are then put on the advisory council. Is this an SRS from the entire population of animals on the Ark? First of all, each animal name has a chance of 50/N, where N is the total number of types of animals on the Ark, of being chosen. Then both the male and female of that type of animal are put on the council. In other words, every individual animal has the same probability – 50/N – of being on the council. And yet there are certainly collections of 100 animals from the Ark which do not consist of 50 breeding pairs: for example, take 50 female birds and 50 female mammals; that collection of 100 animals has no breeding pairs at all. Therefore this is a selection method which picks each individual for the sample with equal probability, but not each collection of 100 animals with the same probability. So it is not an SRS. With a computer, it is fairly quick and easy to generate an SRS: FACT 5.1.13. Suppose we have a population of size N out of which we want to pick an SRS of size n, where n < N. Here is one way to do so: assign every individual in the popu- lation a unique ID number, with say d digits (maybe student IDs, Social Security numbers, new numbers from 1 to N chosen in any way you like – randomness not needed here, there is plenty of randomness in the next step). Have a computer generate completely random d-digit number, one after the other. Each time, pick the individual from the population with that ID number as a new member of the sample. If the next random number generated by the computer is a repeat of one seen before, or if it is a d-digit number that doesn’t happen to be any individual’s ID number, then simply skip to the next random number from the computer. Keep going until you have n individuals in your sample. The sample created in this way will be an SRS. 5.2. STUDIES OF CAUSALITY 99 5.2. Studies of Causality If we want to draw conclusions about causality, observations are insufficient. This is because simply seeing B always follow A out in the world does not tell us that A causes B. For example, maybe they are both caused by Z, which we didn’t notice had always happened before those A and B, and A is simply a bit faster than B, so it seems always to proceed, even to cause, B. If, on the other hand, we go out in the world and do A and then always see B, we would have more convincing evidence that A causes B. Therefore, we distinguish two types of statistical studies D EFINITION 5.2.1. An observational study is any statistical study in which the re- searchers merely look at (measure, talk to, etc.) the individuals in which they are inter- ested. If, instead, the researchers also change something in the environment of their test subjects before (and possibly after and during) taking their measurements, then the study is an experiment. E XAMPLE 5.2.2. A simple survey of, for example, opinions of voters about political candidates, is an observational study. If, as is sometimes done, the subject is told something like “let me read you a statement about these candidates and then ask you your opinion again” [this is an example of something called push-polling], then the study has become an experiment. Note that to be considered an experiment, it is not necessary that the study use princi- ples of good experimental design, such as those described in this chapter, merely that the researchers do something to their subjects. E XAMPLE 5.2.3. If I slap my brother, notice him yelp with pain, and triumphantly turn to you and say “See, slapping hurts!” then I’ve done an experiment, simply because I did something, even if it is a stupid experiment [tiny non-random sample, no comparison, etc., etc.]. If I watch you slap someone, who cries out with pain, and then I make the same tri- umphant announcement, then I’ve only done an observational study, since the action taken was not by me, the “researcher.” When we do an experiment, we typically impose our intentional change on a number of test subjects. In this case, no matter the subject of inquiry, we steal a word from the medical community: D EFINITION 5.2.4. The thing we do to the test subjects in an experiment is called the treatment. 5.2.1. Control Groups. If we are doing an experiment to try to understand something in the world, we should not simply do the interesting new treatment to all of our subjects 100 5. BRINGING HOME THE DATA and see what happens. In a certain sense, if we did that, we would simply be changing the whole world (at least the world of all of our test subjects) and then doing an observational study, which, as we have said, can provide only weak evidence of causality. To really do an experiment, we must compare two treatments. Therefore any real experiment involves at least two groups. D EFINITION 5.2.5. In an experiment, the collection of test subjects which gets the new, interesting treatment is called the experimental group, while the remaining subjects, who get some other treatment such as simply the past common practice, are collectively called the control group. When we have to put test subjects into one of these two groups, it is very important to use a selection method which has no bias. The only way to be sure of this is [as discussed before] to use a random assignment of subjects to the experimental or control group. 5.2.2. Human-Subject Experiments: The Placebo Effect. Humans are particularly hard to study, because their awareness of their environments can have surprising effects on what they do and even what happens, physically, to their bodies. This is not because people fake the results: there can be real changes in patients’ bodies even when you give them a medicine which is not physiologically effective, and real changes in their performance on tests or in athletic events when you merely convince them that they will do better, etc. D EFINITION 5.2.6. A beneficial consequence of some treatment which should not di- rectly [e.g., physiologically] cause an improvement is called the Placebo Effect. Such a “fake” treatment, which looks real but has no actual physiological effect, is called a placebo. Note that even though the Placebo Effect is based on giving subjects a “fake” treatment, the effect itself is not fake. It is due to a complex mind-body connection which really does change the concrete, objectively measurable situation of the test subjects. In the early days of research into the Placebo Effect, the pill that doctors would give as a placebo would look like other pills, but would be made just of sugar (glucose), which (in those quite small quantities) has essentially no physiological consequences and so is a sort of neutral dummy pill. We still often call medical placebos sugar pills even though now they are often made of some even more neutral material, like the starch binder which is used as a matrix containing the active ingredient in regular pills – but without any active ingredient. Since the Placebo Effect is a real phenomenon with actual, measurable consequences, when making an experimental design and choosing the new treatment and the treatment for the control group, it is important to give the control group something. If they get nothing, they do not have the beneficial consequences of the Placebo Effect, so they will not have as good measurements as the experimental group, even if the experimental treatment had 5.2. STUDIES OF CAUSALITY 101 no actual useful effect. So we have to equalize for both groups the benefit provided by the Placebo Effect, and give them both an treatment which looks about the same (compare pills to pills, injections to injections, operations to operations, three-hour study sessions in one format to three-hour sessions in another format, etc.) to the subjects. D EFINITION 5.2.7. An experiment in which there is a treatment group and a control group, which control group is given a convincing placebo, is said to be placebo-controlled. 5.2.3. Blinding. We need one last fundamental tool in experimental design, that of keeping subjects and experimenters ignorant of which subject is getting which treatment, experimental or control. If the test subjects are aware of into which group they have been put, that mind-body connection which causes the Placebo Effect may cause a systematic difference in their outcomes: this would be the very definition of bias. So we don’t tell the patients, and make sure that their control treatment looks just like the real experimental one. It also could be a problem if the experimenter knew who was getting which treatment. Perhaps if the experimenter knew a subject was only getting the placebo, they would be more compassionate or, alternatively, more dismissive. In either case, the systematically different atmosphere for that group of subjects would again be a possible cause of bias. Of course, when we say that the experimenter doesn’t know which treatment a partic- ular patient is getting, we mean that they do not know that at the time of the treatment. Records must be kept somewhere, and at the end of the experiment, the data is divided between control and experimental groups to see which was effective. D EFINITION 5.2.8. When one party is kept ignorant of the treatment being adminis- tered in an experiment, we say that the information has been blinded. If neither subjects nor experimenters know who gets which treatment until the end of the experiment (when both must be told, one out of fairness, and one to learn something from the data that was collected), we say that the experiment was double-blind. 5.2.4. Combining it all: RCTs. This, then is the gold standard for experimental de- sign: to get reliable, unbiased experimental data which can provide evidence of causality, the design must be as follows: D EFINITION 5.2.9. An experiment which is • randomized • placebo-controlled. • double-blind is called, for short, a randomized, controlled trial [RCT] (where the “placebo-” and “double-blind” are assumed even if not stated). 102 5. BRINGING HOME THE DATA 5.2.5. Confounded Lurking Variables. A couple of last terms in this subject are quite poetic but also very important. D EFINITION 5.2.10. A lurking variable is a variable which the experimenter did not put into their investigation. So a lurking variable is exactly the thing experimenters most fear: something they didn’t think of, which might or might not affect the study they are doing. Next is a situation which also could cause problems for learning from experiments. D EFINITION 5.2.11. Two variables are confounded when we cannot statistically dis- tinguish their effects on the results of our experiments. When we are studying something by collecting data and doing statistics, confounded variables are a big problem, because we do not know which of them is the real cause of the phenomenon we are investigating: they are statistically indistinguishable. The combination of the two above terms is the worst thing for a research project: what if there is a lurking variable (one you didn’t think to investigate) which is confounded with the variable you did study? This would be bad, because then your conclusions would apply equally well (since the variables are statistically identical in their consequences) to that thing you didn’t think of ... so your results could well be completely misunderstanding cause and effect. The problem of confounding with lurking variables is particularly bad with observa- tional studies. In an experiment, you can intentionally choose your subjects very randomly, which means that any lurking variables should be randomly distributed with respect to any lurking variables – but controlled with respect to the variables you are studying – so if the study finds a causal relationship in your study variables, in cannot be confounded with a lurking variable. E XAMPLE 5.2.12. Suppose you want to investigate whether fancy new athletic shoes make runners faster. If you just do an observational study, you might find that those ath- letes with the new shoes do run faster. But a lurking variable here could be how rich the athletes are, and perhaps if you looked at rich and poor athletes they would have the same relationship to slow and fast times as the new- vs old-shoe wearing athletes. Essentially, the variable what kind of shoe is the athlete wearing (categorical with the two values new and old) is being confounded with the lurking variable how wealthy is the athlete. So the conclusion about causality fancy new shoes make them run faster might be false, and in- stead the real truth might be wealthy athletes, who have lots of support, good coaches, good nutrition, and time to devote to their sport, run faster. If, instead, we did an experiment, we would not have this problem. We would select athletes at random – so some would be wealthy and some not – and give half of them (the experimental group) the fancy new shoes and the other half (the control group) the old type. 5.2. STUDIES OF CAUSALITY 103 If the type of shoe was the real cause of fast running, we would see that in our experimental outcome. If really it is the lurking variable of the athlete’s wealth which matters, then we would see neither group would do better than the other, since they both have a mixture of wealthy and poor athletes. If the type of shoe really is the cause of fast running, then we would see a difference between the two groups, even though there were rich and poor athletes in both groups, since only one group had the fancy new shoes. In short, experiments are better at giving evidence for causality than observational stud- ies in large part because an experiment which finds a causal relationship between two vari- ables cannot be confounding the causal variable under study with a lurking variable. 104 5. BRINGING HOME THE DATA 5.3. Experimental Ethics Experiments with human subjects are technically hard to do, as we have just seen, because of things like the Placebo Effect. Even beyond these difficulties, they are hard because human subjects just don’t do what we tell them, and seem to want to express their free will and autonomy. In fact, history has many (far too many) examples of experiments done on human sub- jects which did not respect their humanity and autonomy – see, for example, the Wikipedia page on unethical human experimentation [Wik17b]. The ethical principles for human subject research which we give below are largely based on the idea of respecting the humanity and autonomy of the test subjects, since the lack of that respect seems to be the crucial failure of many of the generally acknowledged unethical experiments in history. Therefore the below principles should always be taken as from the point of view of the test subjects, or as if they were designed to create systems which protect those subjects. In particular, a utilitarian calculus of the greatest good for the greatest number might be appealing to some, but modern philosophers of experimental ethics generally do not allow the researchers to make that decision themselves. If, for example, some subjects were willing and chose to experience some negative consequences from being in a study, that might be alright, but it is never to be left up to the researcher. 5.3.1. “Do No Harm”. The Hippocratic Oath, a version of which is thought in popular culture to be sworn by all modern doctors, is actually not used much at all today in its original form. This is actually not that strange, since it sounds quite odd and archaic1 to modern ears – it begins I swear by Apollo the physician, and Asclepius, and Hygieia and Panacea and all the gods and goddesses as my witnesses that... It also has the odd requirements that physicians not use a knife, and will remain celibate, etc. One feature, often thought to be part of the Oath, does not exactly appear in the tra- ditional text but is probably considered the most important promise: First, do no harm [sometimes seen in the Latin version, primum nil nocere]. This principle is often thought of as constraining doctors and other care-givers, which is why, for example, the American Medical Association forbids doctors from participation in executions, even when they are legal in certain jurisdictions in the United States. It does seem like good general idea, in any case, that those who have power and au- thority over others should, at the very least, not harm them. In the case of human subject experimentation, this is thought of as meaning that researchers must never knowingly harm their patients, and must in fact let the patients decide what they consider harm to be. 1It dates from the 5th century BCE, and is attributed to Hippocrates of Kos [US 12]. 5.3. EXPERIMENTAL ETHICS 105 5.3.2. Informed Consent. Continuing with the idea of letting subjects decide what harms they are willing to experience or risk, one of the most important ethical principles for human subject research is that test subjects must be asked for informed consent. What this means is that they must be informed of all of the possible consequences, positive and (most importantly) negative, of participation in the study, and then given the right to decide if they want to participate. The information part does not have to tell every detail of the experimental design, but it must give every possible consequence that the researchers can imagine. It is important when thinking about informed consent to make sure that the subjects really have the ability to exercise fully free will in their decision to give consent. If, for example, participation in the experiment is the only way to get some good (health care, monetary compensation in a poor neighborhood, a good grade in a class, advancement in their job, etc.) which they really need or want, the situation itself may deprive them of their ability freely to say no – and therefore yes, freely. 5.3.3. Confidentiality. The Hippocratic Oath does also require healers to protect the privacy of their patients. Continuing with the theme of protecting the autonomy of test subjects, then, it is considered to be entirely the choice of subject when and how much information about their participation in the experiment will be made public. The kinds of information protected here run from, of course, the subjects’ performance in the experimental activities, all the way to the simple fact of participation itself. There- fore, ethical experimenters must make it possible for subject to sign up for and then do all parts of the experiment without anyone outside the research team knowing this fact, should the subject want this kind of privacy. As a practical matter, something must be revealed about the experimental outcomes in order for the scientific community to be able to learn something from that experiment. Typically this public information will consist of measures like sample means and other data which are aggregated from many test subjects’ results. Therefore, even if it were know what the mean was and that a person participated in the study, the public would not be able to figure out what that person’s particular result was. If the researchers want to give more precise information about one particular test sub- ject’s experiences, or about the experiences of a small enough number of subjects that indi- vidual results could be disaggregated from what was published, then the subjects’ identities must be hidden, or anonymized. This is done by removing from scientific reports all per- sonally identifiable information [PII] such as name, social security or other ID number, address, phone number, email address, etc. 5.3.4. External Oversight [IRB]. One last way to protect test subjects and their au- tonomy which is required in ethical human subject experimentation is to give some other, disinterested, external group as much power and information as the researchers themselves. 106 5. BRINGING HOME THE DATA In the US, this is done by requiring all human subject experimentation to get approval from a group of trained and independent observers, called the Institutional Review Board [IRB] before the start of the experiment. The IRB is given a complete description of all details of the experimental design and then chooses whether or not to give its approval. In cases when the experiment continues for a long period of time (such as more than one year), progress reports must be given to the IRB and its re-approval sought. Note that the way this IRB requirement is enforced in the US is by requiring approval by a recognized IRB for experimentation by any organization which wants ever to receive US Federal Government monies, in the form of research grants, government contracts, or even student support in schools. IRBs tend to be very strict about following rules, and if the ever see a violation at some such organization, that organization will quickly get excluded from federal funds for a very long time. As a consequence, all universities, NGOs, and research institutes in the US, and even many private organizations or companies, are very careful about proper use of IRBs. EXERCISES 107 Exercises E XERCISE 5.1. In practice, wording effects are often an extremely strong influence on the answers people give when surveyed. So... Suppose you were doing a survey of American voters opinions of the president. Think of a way of asking a question which would tend to maximize the number of people who said they approved of the job he is doing. Then think of another way of asking a question which would tend to minimize that number [who say they approve of his job performance]. E XERCISE 5.2. Think of a survey question you could ask in a survey of the general population of Americans in response to which many [most?] people would lie. State what would be the issue you would be investigating with this survey question, as a clearly de- fined, formal variable and parameter on the population of all Americans. Also tell exactly what would be the wording of the question you think would get lying responses. Now think of a way to do an observational study which would get more accurate values for this variable and for the parameter of interest. Explain in detail. E XERCISE 5.3. Many parents believe that their small children get a bit hyperactive when they eat or drink sweets (candies, sugary sodas, etc.), and so do not let their kids have such things before nap time, for example. A pediatrician at Euphoria State University Teaching Hospital [ESUTH] thinks instead that it is the parents’ expectations about the effects of sugar which cause their children to become hyperactive, and not the sugar at all. Describe a randomized, placebo-controlled, double-blind experiment which would col- lect data about this ESUTH pediatrician’s hypothesis. Make sure you are clear about both which part of your experimental procedure addresses each of those important components of good experimental design. E XERCISE 5.4. Is the experiment you described in the previous exercise an ethical one? What must the ESUTH pediatrician do before, during, and after the experiment to make sure it is ethical? Make sure you discuss (at least) the checklist of ethical guidelines from this chapter and how each point applies to this particular experiment. Part 3 Inferential Statistics We are now ready to make (some) inferences about the real world based on data – this subject is called inferential statistics. We have seen how to display and interpret 1- and 2-variable data. We have seen how to design experiments, particularly experiments whose results might tell us something about cause and effect in the real world. We even have some principles to help us do such experimentation ethically, should our subjects be human beings. Our experimental design principles use randomness (to avoid bias), and we have even studied the basics of probability theory, which will allow us to draw the best possible conclusions in the presence of randomness. What remains to do in this part is to start putting the pieces together. In particular, we shall be interested in drawing the best possible conclusions about some population parameter of interest, based on data from a sample. Since we know always to seek simple random samples (again, to avoid bias), our inferences will be never be completely sure, instead they will be built on (a little bit of) probability theory. The basic tools we describe for this inferential statistics are the confidence interval and the hypothesis test (also called test of significance). In the first chapter of this Part, we start with the easiest cases of these tools, when they are applied to inferences about the population mean of a quantitative RV. Before we do that, we have to discuss the Central Limit Theorem [CLT], which is both crucial to those tools and one of the most powerful and subtle theorems of statistics. CHAPTER 6 Basic Inferences The purpose of this chapter is to introduce two basic but powerful tools of inferential statistics, the confidence interval and the hypothesis test (also called test of significance), in the simplest case of looking for the population mean of a quantitative RV. This simple case of these tool is based, for both of them, on a beautiful and amazing theorem called the Central Limit Theorem, which is therefore the subject of the first section of the chapter. The following sections then build the ideas and formulæ first for confidence intervals and then for hypothesis tests. Throughout this chapter, we assume that we are working with some (large) population on which there is defined a quantitative RV X. The population mean σX is, of course, a fixed number, out in the world, unchanging but also probably unknown, simply because to compute it we would have to have access to the values of X for the entire population. Strangely, we assume in this chapter that while we do not know µX , we do know the population standard deviation σX , of X. This is actually quite a silly assumption – how could we know σX if we didn’t already know µX ? But we make this assumption because if makes this first version of confidence intervals and hypothesis tests particularly simple. (Later chapters in this Part will remove this silly assumption.) Finally, we always assume in this chapter that the samples we use are simple random samples, since by now we know that those are the best kind. 111 112 6. BASIC INFERENCES 6.1. The Central Limit Theorem Taking the average [mean] of a sample of quantitative data is actually a very nice pro- cess: the arithmetic is simple, and the average often has the nice property of being closer to the center of the data than the values themselves being combined or averaged. This is because while a random sample may have randomly picked a few particularly large (or particularly small) values from the data, it probably also picked some other small (or large) values, so that the mean will be in the middle. It turns out that these general observations of how nice a sample mean can be explained and formalized in a very important Theorem: FACT 6.1.1. The Central Limit Theorem [CLT] Suppose we have a large population on which is defined a quantitative random variable X whose population mean is µX and whose population standard deviation is σX . Fix a whole number n ≥ 30. As we take repeated, independent SRSs of size n, the distribution of the sample means x of these SRSs √ is approximately N(µX , σX / n). That is, the distribution of x is approximately Normal √ with mean µX and standard deviation σX / n. Furthermore, as n gets bigger, the Normal approximation gets better. Note that the CLT has several nice pieces. First, it tells us that the middle of the his- togram of sample means, as we get repeated independent samples, is the same as the mean of the original population – the mean of the sample means is the population mean. We might write this as µx = µX . Second, the CLT tells us precisely how much less variation there is in the sample means because of the process noted above whereby averages are closer to the middle of some data √ than are the data values themselves. The formula is σx = σX / n. Finally and most amazingly, the CLT actually tells us exactly what is the shape of the distribution for x – and it turns out to be that complicated formula we gave Defini- tion 4.3.19. This is completely unexpected, but somehow the universe knows that formula for the Normal distribution density function and makes it appear when we construct the histogram of sample means. Here is an example of how we use the CLT: E XAMPLE 6.1.2. We have said elsewhere that adult American males’ heights in inches are distributed like N(69, 2.8). Supposing this is true, let us figure out what is the proba- bility that 52 randomly chosen adult American men, lying down in a row with each one’s feet touching the next one’s head, stretch the length of a football field. [Why 52? Well, an American football team may have up to 53 people on its active roster, and one of them has to remain standing to supervise everyone else’s formation lying on the field....] First of all, notice that a football field is 100 yards long, which is 300 feet or 3600 inches. If every single one of our randomly chosen men was exactly the average height for 6.1. THE CENTRAL LIMIT THEOREM 113 adult men, that would a total of 52 ∗ 69 = 3588 inches, so they would not stretch the whole length. But there is variation of the heights, so maybe it will happen sometimes.... So imagine we have chosen 52 random adult American men. Measure each of their heights, and call those numbers x1 , x2 , . . . , x52 . What we are trying to figure out is whether P xi ≥ 3600. More precisely, we want to know X P xi ≥ 3600 . Nothing in that looks familiar, but remember that the 52 adult men were chosen randomly. The best way to choose some number, call it n = 52, of individuals from a population is to choose an SRS of size n. Let’s also assume that we did that here. Now, having an SRS, we know from the CLT √ that the sample mean x is N(69, 2.8/ 52) or, doing the arithmetic, N(69, .38829). But the question we are considering here doesn’t mention x, you cry! Well, it almost does: x is the sample mean given by P P xi xi x= = . n 52 What that means is that the inequality X xi ≥ 3600 amounts to exactly the same thing, by dividing both sides by 52, as the inequality P xi 3600 ≥ 52 52 or, in other words, x ≥ 69.23077 . Since these inequalities all amount to the same thing, they have the same probabilities, so X P xi ≥ 3600 = P (x ≥ 69.23077) . But remember x was N(69, .38829), so we can calculate this probability with LibreOffice Calc or Microsoft Excel as P (x ≥ 69.23077) = 1 − P (x < 69.23077) = NORM.DIST(69.23077, 69, .38829, 1) = .72385 where here we first use the probability rule for complements to turn around the inequality into the direction that NORM.DIST calculates. Thus the chance that 52 randomly chosen adult men, lying in one long column, are as long as a football field, is 72.385%. 114 6. BASIC INFERENCES 6.2. Basic Confidence Intervals As elsewhere in this chapter, we assume that we are working with some (large) popula- tion on which there is defined a quantitative RV X. The population mean µX is unknown, and we want to estimate it. world, unchanging but also probably unknown, simply because to compute it we would have to have access to the values of X for the entire population. We continue also with our strange assumption that while we do not know µX , we do know the population standard deviation σX , of X. Our strategy to estimate µX is to take an SRS of size n, compute the sample mean x of X, and then to guess that µX ≈ x. But this leaves us wondering how good an approximation x is of µX . The strategy we take for this is to figure how close µX must be to x – or x to µX , it’s the same thing, and in fact to be precise enough to say what is the probability that µX is a certain distance from x. That is, if we choose a target probability, call it L, we want to make an interval of real numbers centered on x with the probability of µX being in that interval being L. Actually, that is not really a sensible thing to ask for: probability, remember, is the fraction of times something happens in repeated experiments. But we are not repeatedly choosing µX and seeing if it is in that interval. Just the opposite, in fact: µX is fixed (al- though unknown to us), and every time we pick a new SRS – that’s the repeated experiment, choosing new SRSs! – we can compute a new interval and hope that that new interval might contain µX . The probability L will correspond to what fraction of those newly computed intervals which contain the (fixed, but unknown) µX . How to do even this? Well, the Central Limit Theorem tells us that the distribution of x as we take repeated SRSs – exactly the repeatable experiment we are imagining doing – is approximately Nor- √ mal with mean µX and standard deviation σX / n. By the 68-95-99.7 Rule: √ (1) 68% of the time we take samples, the resulting x will be within σX / n units on the number line of µX . Equivalently (since the distance from A to B is the same as the distance from B to A!), 68% of the time we take samples, µX will be within √ σX / n of x. In other words, 68% of the time we take samples, µX will happen to √ √ lie in the interval from x − σX / n to x + σX / n. √ (2) Likewise, 95% of the time we take samples, the resulting x will be within 2σX / n units on the number line of µX . Equivalently (since the distance from A to B is still the same as the distance from B to A!), 95% of the time we take samples, µX √ will be within 2σX / n of x. In other words, 95% of the time we take samples, √ √ µX will happen to lie in the interval from x − 2σX / n to x + 2σX / n. (3) Likewise, 99.7% of the time we take samples, the resulting x will be within √ 3σX / n units on the number line of µX . Equivalently (since the distance from A 6.2. BASIC CONFIDENCE INTERVALS 115 to B is still the same as the distance from B to A!), 99.7% of the time we take sam- √ ples, µX will be within 3σX / n of x. In other words, 99.7% of the time we take √ √ samples, µX will happen to lie in the interval from x − 3σX / n to x + 3σX / n. √ Notice the general shape here is that the interval goes from x − zL∗ σX / n to x + √ zL∗ σX / n, where this number zL∗ has a name: D EFINITION 6.2.1. The critical value zL∗ with probability L for the Normal distribu- tion is the number such that the Normal distribution N(µX , σX ) has probability L between µX − zL∗ σX and µX + zL∗ σX . Note the probability L in this definition is usually called the confidence level. If you think about it, the 68-95-99.7 Rule is exactly telling us that zL∗ = 1 if L = .68, zL∗ = 2 if L = .95, and zL∗ = 3 if L = .997. It’s actually convenient to make a table of similar values, which can be calculated on a computer from the formula for the Normal distribution. FACT 6.2.2. Here is a useful table of critical values for a range of possible confidence levels: L .5 .8 .9 .95 .99 .999 ∗ zL .674 1.282 1.645 1.960 2.576 3.291 Note that, oddly, the zL∗ here for L = .95 is not 2, but rather 1.96! This is actually more accurate value to use, which you may choose to use, or you may continue to use zL∗ = 2 when L = .95, if you like, out of fidelity to the 68-95-99.7 Rule. Now, using these accurate critical values we can define an interval which tells us where we expect the value of µX to lie. D EFINITION 6.2.3. For a probability value L, called the confidence level, the interval √ √ of real numbers from x − zL∗ σX / n to x + zL∗ σX / n is called the confidence interval for µX with confidence level L. The meaning of confidence here is quite precise (and a little strange): FACT 6.2.4. Any particular confidence interval with confidence level L might or might not actually contain the sought-after parameter µX . Rather, what it means to have confi- dence level L is that if we take repeated, independent SRSs and compute the confidence interval again for each new x from the new SRSs, then a fraction of size L of these new intervals will contain µX . Note that any particular confidence interval might or might not contain µX not because µX is moving around, but rather the SRSs are different each time, so the x is (potentially) different, and hence the interval is moving around. The µX is fixed (but unknown), while the confidence intervals move. 116 6. BASIC INFERENCES Sometimes the piece we add and subtract from the x to make a confidence interval is given a name of its own: D EFINITION 6.2.5. When we write a confidence interval for the population mean µX √ of some quantitative variable X in the form x − E to x + E, where E = zL∗ σX / n, we call E the margin of error [or, sometimes, the sampling error] of the confidence interval. Note that if a confidence interval is given without a stated confidence level, particularly in the popular press, we should assume that the implied level was .95 . 6.2.1. Cautions. The thing that most often goes wrong when using confidence inter- vals is that the sample data used to compute the sample mean x and then the endpoints x ± E of the interval is not from a good SRS. It is hard to get SRSs, so this is not unex- pected. But we nevertheless frequently assume that some sample is an SRS, so that we can use it to make a confidence interval, even of that assumption is not really justified. Another thing that can happen to make confidence intervals less accurate is to choose too small a sample size n. We have seen that our approach to confidence intervals relies upon the CLT, therefore it typically needs samples of size at least 30. E XAMPLE 6.2.6. A survey of 463 first-year students at Euphoria State University [ESU] found that the [sample] average of how long they reported studying per week was 15.3 hours. Suppose somehow we know that the population standard deviation of hours of study per week at ESU is 8.5 . Then we can find a confidence interval at the 99% confidence level for the mean study per week of all first-year students by calculating the margin of error to be √ √ E == zL∗ σX / n = 2.576 · 8.5/ 463 = 1.01759 and then noting that the confidence interval goes from x − E = 15.3 − 1.01759 = 14.28241 to x + E = 15.3 + 1.01759 = 16.31759 . Note that for this calculation to be doing what we want it to do, we must assume that the 463 first-year students were an SRS out of the entire population of first-year students at ESU. Note also that what it means that we have 99% confidence in this interval from 14.28241 to 16.31759 (or [14.28241, 16.31759] in interval notation) is not, in fact, that we any con- fidence at all in those particular numbers. Rather, we have confidence in the method, in the sense that if we imagine independently taking many future SRSs of size 463 and recal- culating new confidence intervals, then 99% of these future intervals will contain the one, fixed, unknown µX . 6.3. BASIC HYPOTHESIS TESTING 117 6.3. Basic Hypothesis Testing Let’s start with a motivating example, described somewhat more casually than the rest of the work we usually do, but whose logic is exactly that of the scientific standard for hypothesis testing. E XAMPLE 6.3.1. Suppose someone has a coin which they claim is a fair coin (includ- ing, in the informal notion of a fair coin, that successive flips are independent of each other). You care about this fairness perhaps because you will use the coin in a betting game. How can you know if the coin really is fair? Obviously, your best approach is to start flipping the coin and see what comes up. If the first flip shows heads [H], you wouldn’t draw any particular conclusion. If the second was also an H, again, so what? If the third was still H, you’re starting to think there’s a run going. If you got all the way to ten Hs in a row, you would be very suspicious, and if the run went to 100 Hs, you would demand that some other coin (or person doing the flipping) be used. Somewhere between two and 100 Hs in a row, you would go from bland acceptance of fairness to nearly complete conviction that this coin is not fair – why? After all, the person flipping the coin and asserting its fairness could say, correctly, that it is possible for a fair coin to come up H any number of times in a row. Sure, you would reply, but it is very unlikely: that is, given that the coin is fair, the conditional probability that those long runs without Ts would occur is very small. Which in turn also explains how you would draw the line, between two and 100 Hs in a row, for when you thought the the improbability of that particular run of straight Hs was past the level you would be willing to accept. Other observers might draw the line elsewhere, in fact, so there would not be an absolutely sure conclusion to the question of whether the coin was fair or not. It might seem that in the above example we only get a probabilistic answer to a yes/no question (is the coin fair or not?) simply because the thing we are asking about is, by nature, a random process: we cannot predict how any particular flip of the coin will come out, but the long-term behavior is what we are asking about; no surprise, then, that the answer will involve likelihood. But perhaps other scientific hypotheses will have more decisive answers, which do not invoke probability. Unfortunately, this will not be the case, because we have seen above that it is wise to introduce probability into an experimental situation, even if it was not there originally, in order to avoid bias. Modern theories of science (such as quantum mechanics, and also, although in a different way, epidemiology, thermodynamics, genetics, and many other sci- ences) also have some amount of randomness built into their very foundations, so we should expect probability to arise in just about every kind of data. 118 6. BASIC INFERENCES Let’s get a little more formal and careful about what we need to do with hypothesis testing. 6.3.1. The Formal Steps of Hypothesis Testing. (1) State what is the population under study. (2) State what is the variable of interest for this population. For us in this section, that will always be a quantitative variable X. (3) State which is the resulting population parameter of interest. For us in this section, that will always be the population mean µX of X. (4) State two hypotheses about the value of this parameter. One, called the null hy- pothesis and written H0 , will be a statement that the parameter of interest has a particular value, so H0 : µX = µ0 where µ0 is some specific number. The other is the interesting alternative we are considering for the value of that parameter, and is thus called the alternative hypothesis, written Ha . The alternative hypothesis can have one of three forms: Ha : µX < µ0 , Ha : µX > µ0 , or Ha : µX 6= µ0 , where µ0 is the same specific number as in H0 . (5) Gather data from an SRS and compute the sample statistic which is best related to the parameter of interest. For us in this section, that will always be the sample mean X (6) Compute the following conditional probability ! getting values of the statistic which are as extreme, p=P H0 . or more extreme, as the ones you did get This is called the p-value of the test. (7) If the p-value is sufficiently small – typically, p < .05 or even p < .01 – announce “We reject H0 , with p = hnumber herei.” Otherwise, announce “We fail to reject H0 , with p = hnumber herei.” (8) Translate the result just announced into the language of the original question. As you do this, you can say “There is strong statistical evidence that ...” if the p-value is very small, while you should merely say something like “There is evidence that...” if the p-value is small but not particularly so. 6.3. BASIC HYPOTHESIS TESTING 119 Note that the hypotheses H0 and Ha are statements, not numbers. So don’t write some- thing like H0 = µX = 17; you might use H0 = “µX = 17” or Ho : µX = 17 (we always use the latter in this book). 6.3.2. How Small is Small Enough, for p-values? Remember how the p-value is de- fined: ! getting values of the statistic which are as extreme, p=P H0 . or more extreme, as the ones you did get In other words, if the null hypothesis is true, maybe the behavior we saw with the sample data would sometimes happen, but if the probability is very small, it starts to seem that, under the assumption H0 is true, the sample behavior was a crazy fluke. If the fluke is crazy enough, we might want simply to say that since the sample behavior actually happened, it makes us doubt that H0 is true at all. For example, if p = .5, that means that under the assumption H0 is true, we would see behavior like that of the sample about every other time we take an SRS and compute the sample statistic. Not much of a surprise. If the p = .25, that would still be behavior we would expect to see in about one out of every four SRSs, when the H0 is true. When p gets down to .1, that is still behavior we expect to see about one time in ten, when H0 is true. That’s rare, but we wouldn’t want to bet anything important on it. Across science, in legal matters, and definitely for medical studies, we start to reject H0 when p < .05. After all, if p < .05 and H0 is true, then we would expect to see results as extreme as the ones we saw in fewer than one SRS out of 20. There is some terminology for these various cut-offs. D EFINITION 6.3.2. When we are doing a hypothesis test and get a p-value which sat- isfies p < α, for some real number α, we say the data are statistically significant at level α. Here the value α is called the significance level of the test, as in the phrase “We reject H0 at significance level α,” which we would say if p < α. E XAMPLE 6.3.3. If we did a hypothesis test and got a p-value of p = .06, we would say about it that the result was statistically significant at the α = .1 level, but not statistically significant at the α = .05 level. In other words, we would say “We reject the null hypothesis at the α = .1 level,” but also “We fail to reject the null hypothesis at the α = .05 level,”. 120 6. BASIC INFERENCES FACT 6.3.4. The courts in the United States, as well as the majority of standard sci- entific and medical tests which do a formal hypothesis test, use the significance level of α = .05. In this chapter, when not otherwise specified, we will use that value of α = .05 as a default significance level. E XAMPLE 6.3.5. We have said repeatedly in this book that the heights of American males are distributed like N(69, 2.8). Last semester, a statistics student named Mohammad Wong said he thought that had to be wrong, and decide to do a study of the question. MW is a bit shorter than 69 inches, so his conjecture was that the mean height must be less, also. He measured the heights of all of the men in his statistics class, and was surprised to find that the average of those 16 men’s heights was 68 inches (he’s only 67 inches tall, and he thought he was typical, at least for his class1). Does this support his conjecture or not? Let’s do the formal hypothesis test. The population that makes sense for this study would be all adult American men today – MW isn’t sure if the claim of American men’s heights having a population mean of 69 inches was always wrong, he is just convinced that it is wrong today. The quantitative variable of interest on that population is their height, which we’ll call X. The parameter of interest is the population mean µX . The two hypotheses then are H0 : µX = 69 and Ha : µX < 69 , where the basic idea in the null hypothesis is that the claim in this book of men’s heights having mean 69 is true, while the new idea which MW hopes to find evidence for, encoded in alternative hypothesis, is that the true mean of today’s men’s heights is less than 69 inches (like him). MW now has to make two bad assumptions: the first is that the 16 students in his class are an SRS drawn from the population of interest; the second, that the population standard deviation of the heights of individuals in his population of interest is the same as the population standard deviation of the group of all adult American males asserted elsewhere in this book, 2.8 . These are definitely bad assumptions – particularly that MW’s male classmates are an SRS of the population of today’s adult American males – but he has to make them nevertheless in order to get somewhere. The sample mean height X for MW’s SRS of size n = 16 is X = 68. 1When an experimenter tends to look for information which supports their prior ideas, it’s called confir- mation bias – MW may have been experiencing a bit of this bias when he mistakenly thought he was average in height for his class. 6.3. BASIC HYPOTHESIS TESTING 121 MW can now calculate the p-value of this test, using the Central Limit Theorem. Ac- √ cording to the CLT, the distribution of X is N(69, 2.8/ 16). Therefore the p-value is ! MW would get values of X which are as p=P H0 = P (X < 69) . extreme, or more extreme, as the ones he did get Which, by what we just observed the CLT tells us, is computable by √ normalcdf(−9999, 68, 69, 2.8/ 16) on a calculator, or NORM.DIST(68, 69, 2.8/SQRT(16), 1) in a spreadsheet, either of which gives a value around .07656 . This means that if MW uses the 5% significance level, as we often do, the result is not statistically significant. Only at the much cruder 10% significance level would MW say that he rejects the null hypothesis. In other words, he might conclude his project by saying “My research collected data about my conjecture which was statistically insignificant at the 5% significance level but the data, significant at the weaker 10% level, did indicate that the average height of American men is less than the 69 inches we were told it is (p = .07656).” People who talk to MW about his study should have additional concerns about his assump- tions of having an SRS and of the value of the population standard deviation 6.3.3. Calculations for Hypothesis Testing of Population Means. We put together the ideas in §6.3.1 above and the conclusions of the Central Limit Theorem to summarize what computations are necessary to perform: FACT 6.3.6. Suppose we are doing a formal hypothesis test with variable X and param- eter of interest the population mean µX . Suppose that somehow we know the population standard deviation σX of X. Suppose the null hypothesis is H0 : µX = µ0 where µ0 is a specific number. Suppose also that we have an SRS of size n which yielded the sample mean X. Then exactly one of the following three situations will apply: (1) If the alternative hypothesis is Ha : µX < µ0 then the p-value of the test can be calculated in any of the following ways √ (a) the area to the left of X under the graph of a N(µ0 , σX / n) distribution, √ (b) normalcdf(−9999, X, µ0 , σX / n) on a calculator, or (c) NORM.DIST(X, µ0 , σX /SQRT(n), 1) on a spreadsheet. (2) If the alternative hypothesis is Ha : µX > µ0 then the p-value of the test can be calculated in any of the following ways 122 6. BASIC INFERENCES √ (a) the area to the right of X under the graph of a N(µ0 , σX / n) distribution, √ (b) normalcdf(X, 9999, µ0, σX / n) on a calculator, or (c) 1-NORM.DIST(X, µ0 , σX /SQRT(n), 1) on a spreadsheet. (3) If the alternative hypothesis is Ha : µX 6= µ0 then the p-value of the test can be found by using the approach in exactly one of the following three situations: (a) If X < µ0 then p is calculated by any of the following three ways: √ (i) two times the area to the left of X under the graph of a N(µ0 , σX / n) distribution, √ (ii) 2 * normalcdf(−9999, X, µ0, σX / n) on a calculator, or (iii) 2 * NORM.DIST(X, µ0 , σX /SQRT(n), 1) on a spreadsheet. (b) If X > µ0 then p is calculated by any of the following three ways: √ (i) two times the area to the right of X under the graph of a N(µ0 , σX / n) distribution, √ (ii) 2 * normalcdf(X, 9999, µ0, σX / n) on a calculator, or (iii) 2 * (1-NORM.DIST(X, µ0 , σX /SQRT(n), 1)) on a spread- sheet. (c) If X = µ0 then p = 1. Note the reason that there is that multiplication by two if the alternative hypothesis is Ha : µX 6= µ0 is that there are two directions – the distribution has two tails – in which the values can be more extreme than X. For this reason we have the following terminology: D EFINITION 6.3.7. If we are doing a hypothesis test and the alternative hypothesis is Ha : µX > µ0 or Ha : µX < µ0 then this is called a one-tailed test. If, instead, the alternative hypothesis is Ha : µX 6= µ0 then this is called a two-tailed test. E XAMPLE 6.3.8. Let’s do one very straightforward example of a hypothesis test: A cosmetics company fills its best-selling 8-ounce jars of facial cream by an automatic dispensing machine. The machine is set to dispense a mean of 8.1 ounces per jar. Uncon- trollable factors in the process can shift the mean away from 8.1 and cause either underfill or overfill, both of which are undesirable. In such a case the dispensing machine is stopped and recalibrated. Regardless of the mean amount dispensed, the standard deviation of the amount dispensed always has value .22 ounce. A quality control engineer randomly selects 30 jars from the assembly line each day to check the amounts filled. One day, the sample mean is X = 8.2 ounces. Let us see if there is sufficient evidence in this sample to indicate, at the 1% level of significance, that the machine should be recalibrated. The population under study is all of the jars of facial cream on the day of the 8.2 ounce sample. The variable of interest is the weight X of the jar in ounces. The population parameter of interest is the population mean µX of X. 6.3. BASIC HYPOTHESIS TESTING 123 The two hypotheses then are H0 : µX = 8.1 and Ha : µX 6= 8.1 . The sample mean is X = 8.2 , and the sample – which we must assume to be an SRS – is of size n = 30. Using the case in Fact 6.3.6 where the alternative hypothesis is Ha : µX 6= µ0 and the sub-case where X > µ0 , we compute the p-value by 2 * (1-NORM.DIST(8.2, 8.1, .22/SQRT(30), 1)) on a spreadsheet, which yields p = .01278 . Since p is not less than .01, we fail to reject H0 at the α = .01 level of significance. The quality control engineer should therefore say to company management “Today’s sample, though off weight, was not statistically significant at the stringent level of significance of α = .01 that we have chosen to use in these tests, that the jar-filling machine is in need of recalibration today (p = .01278).” 6.3.4. Cautions. As we have seen before, the requirement that the sample we are using in our hypothesis test is a valid SRS is quite important. But it is also quite hard to get such a good sample, so this is often something that can be a real problem in practice, and something which we must assume is true with often very little real reason. It should be apparent from the above Facts and Examples that most of the work in doing a hypothesis test, after careful initial set-up, comes in computing the p-value. Be careful of the phrase statistically significant. It does not mean that the effect is large! There can be a very small effect, the X might be very close to µ0 and yet we might reject the null hypothesis if the population standard deviation σX were sufficiently small, or √ even if the sample size n were large enough that σX / n became very small. Thus, oddly enough, a statistically significant result, one where the conclusion of the hypothesis test was statistically quite certain, might not be significant in the sense of mattering very much. With enough precision, we can be very sure of small effects. Note that the meaning of the p-value is explained above in its definition as a conditional probability. So p does not compute the probability that the null hypothesis H0 is true, or any such simple thing. In contrast, the Bayesian approach to probability, which we chose not to use in the book, in favor of the frequentist approach, does have a kind of hypothesis test which includes something like the direct probability that H0 is true. But we did not follow the Bayesian approach here because in many other ways it is more confusing. In particular, one consequence of the real meaning of the p-value as we use it in this book is that sometimes we will reject a true null hypothesis H0 just out of bad luck. In 124 6. BASIC INFERENCES fact, if p is just slightly less than .05, we would reject H0 at the α = .05 significance level even though, in slightly less than one case in 20 (meaning 1 SRS out of 20 chosen independently), we would do this rejection even though H0 was true. We have a name for this situation. D EFINITION 6.3.9. When we reject a true null hypothesis H0 this is called a type I error. Such an error is usually (but not always: it depends upon how the population, variable, parameter, and hypotheses were set up) a false positive, meaning that something exciting and new (or scary and dangerous) was found even though it is not really present in the population. E XAMPLE 6.3.10. Let us look back at the cosmetic company with a jar-filling machine from Example 6.3.8. We don’t know what the median of the SRS data was, but it wouldn’t be surprising if the data were symmetric and therefore the median would be the same as the sample mean X = 8.2 . That means that there were at least 15 jars with 8.2 ounces of cream in them, even though the jars are all labelled “8oz.” The company is giving way at least .2 × 15 = 3 ounces of the very valuable cream – in fact, probably much more, since that was simply the overfilling in that one sample. So our intrepid quality assurance engineer might well propose to management to in- crease the significance level α of the testing regime in the factory. It is true that with a larger α, it will be easier for simple randomness to result in type I errors, but unless the recalibration process takes a very long time (and so results in fewer jars being filled that day), the cost-benefit analysis probably leans towards fixing the machine slightly too often, rather than waiting until the evidence is extremely strong it must be done. EXERCISES 125 Exercises E XERCISE 6.1. You buy seeds of one particular species to plant in your garden, and the information on the seed packet tells you that, based on years of experience with that species, the mean number of days to germination is 22, with standard deviation 2.3 days. What is the population and variable in that information? What parameter(s) and/or statistic(s) are they asserting have particular values? Do you think they can really know the actual parameter(s) and/or statistic’s(s’) value(s)? Explain. You plant those seeds on a particular day. What is the probability that the first plant closest to your house will germinate within half a day of the reported mean number of days to germination – that is, it will germinate between 21.5 and 22.5 after planting? You are interested in the whole garden, where you planted 160 seeds, as well. What is the probability that the average days to germination of all the plants in your garden is between 21.5 and 22.5 days? How do you know you can use the Central Limit Theorem to answer this problem – what must you assume about those 160 seeds from the seed packet in order for the CLT to apply? E XERCISE 6.2. You decide to expand your garden and buy a packet of different seeds. But the printing on the seed packet is smudged so you can see that the standard deviation for the germination time of that species of plant is 3.4 days, but you cannot see what the mean germination time is. So you plant 100 of these new seeds and note how long each of them takes to germinate: the average for those 100 plants is 17 days. What is a 90% confidence interval for the population mean of the germination times of plants of this species? Show and explain all of your work. What assumption must you make about those 100 seeds from the packet in order for your work to be valid? What does it mean that the interval you gave had 90% confidence? Answer by talking about what would happen if you bought many packets of those kinds of seeds and planted 100 seeds in each of a bunch of gardens around your community. E XERCISE 6.3. An SRS of size 120 is taken from the student population at the very large Euphoria State University [ESU], and their GPAs are computed. The sample mean GPA is 2.71 . Somehow, we also know that the population standard deviation of GPAs at ESU is .51 . Give a confidence interval at the 90% confidence level for the mean GPA of all students at ESU. You show the confidence interval you just computed to a fellow student who is not taking statistics. They ask, “Does that mean that 90% of students at ESU have a GPA which is between a and b?” where a and b are the lower and upper ends of the interval you computed. Answer this question, explaining why if the answer is yes and both why not and what is a better way of explaining this 90% confidence interval, if the answer is no. 126 6. BASIC INFERENCES E XERCISE 6.4. The recommended daily calorie intake for teenage girls is 2200 calories per day. A nutritionist at Euphoria State University believes the average daily caloric intake of girls in her state to be lower because of the advertising which uses underweight models targeted at teenagers. Our nutritionist finds that the average of daily calorie intake for a random sample of size n = 36 of teenage girls is 2150. Carefully set up and perform the hypothesis test for this situation and these data. You may need to know that our nutritionist has been doing studies for years and has found that the standard deviation of calorie intake per day in teenage girls is about 200 calories. Do you have confidence the nutritionist’s conclusions? What does she need to be care- ful of, or to assume, in order to get the best possible results? E XERCISE 6.5. The medication most commonly used today for post-operative pain relieve after minor surgery takes an average of 3.5 minutes to ease patients’ pain, with a standard deviation of 2.1 minutes. A new drug is being tested which will hopefully bring relief to patients more quickly. For the test, 50 patients were randomly chosen in one hospital after minor surgeries. They were given the new medication and how long until their pain was relieved was timed: the average in this group was 3.1 minutes. Does this data provide statistically significant evidence, at the 5% significance level, that the new drug acts more quickly than the old? Clearly show and explain all your set-up and work, of course! E XERCISE 6.6. The average household size in a certain region several years ago was 3.14 persons, while the standard deviation was .82 persons. A sociologist wishes to test, at the 5% level of significance, whether the mean household size is different now. Perform the test using new information collected by the sociologist: in a random sample of 75 households this past year, the average size was 2.98 persons. E XERCISE 6.7. A medical laboratory claims that the mean turn-around time for per- formance of a battery of tests on blood samples is 1.88 business days. The manager of a large medical practice believes that the actual mean is larger. A random sample of 45 blood samples had a mean of 2.09 days. Somehow, we know that the population standard deviation of turn-around times is 0.13 day. Carefully set up and perform the relevant test at the 10% level of significance. Explain everything, of course. Bibliography [Gal17] Gallup, Presidential Job Approval Center, 2017, https://www.gallup.com/ interactives/185273/presidential-job-approval-center.aspx, Accessed: 2 April 2017. [Huf93] Darrell Huff, How to Lie with Statistics, W.W. Norton & Company, 1993. [PB] Emily Pedersen and Alexander Barreira, Former UC Berkeley political science professor Ray- mond Wolfinger dies at 83, The Daily Californian, 11 February 2015, https://dailycal. org/2015/02/11/former-uc-berkeley-political-science-professor- raymond-wolfinger-dies-83/, Accessed: 21 February 2017. [Sta02] Richard Stallman, Free Software, Free Society: Selected Essays of Richard M. Stallman, Free Soft- ware Foundation, 2002. [TNA10] Mark Twain, Charles Neider, and Michael Anthony, The Autobiography of Mark Twain, Wiley Online Library, 2010. [Tuf06] Edward Tufte, The Cognitive Style of PowerPoint: Pitching Out Corrupts Within, (2nd ed.), Graphics Press, Cheshire, CT, USA, 2006. [US 12] US National Library of Medicine, Greek medicine, 2012, https://www.nlm.nih.gov/hmd/ greek/, Accessed 11-April-2017. [Wik17a] Wikipedia, Cantor’s diagonal argument, 2017, https://en.wikipedia.org/wiki/ Cantor%27s_diagonal_argument, Accessed 5-March-2017. [Wik17b] , Unethical human experimentation, 2017, https://en.wikipedia.org/wiki/ Unethical_human_experimentation, Accessed 7-April-2017. 127 Index 68-95-99.7 Rule, 83–86, 114, 115 addition rule for disjoint events, 57 Addition Rule for General Events, 63 ∅, empty set, 56 aggregated experimental results, for ∩, intersection, 56 confidentiality, 105 ∪, union, 56 alternative hypothesis, Ha , 118–123 ⊂, subset, 55 American Medical Association, 104 E c , complement, 56 amorphous, for scatterplots or associations, 35 Ha , alternative hypothesis, 118–123 and, for events, 56 H0 , null hypothesis, 118–121, 123, 124 anecdote, not the singular of data, 52 1.5 IQR Rule for Outliers, 26 anonymization of experimental results, 105 IQR, inter-quartile range, 23, 25, 26, 31, 32 Apollo the physician, 104 µX , population mean, 18, 93–95, 110–112, Asclepius, 104 114–116, 118, 120–122 “at random”, 65, 74 N , population size, 6 autonomy, value for human test subjects, 104, N (0, 1), the standard Normal distribution, 81 105 N (µX , σX ), Normally distributed with mean µX AVERAGE, sample mean in spreadsheets, 41 and standard deviation σX , 78, 112, 114, 115 average n, sample size, 6, 123 see: mean, 18, 112 P (A | B), conditional probability, 67 Q1 , first quartile, 22, 26–28 bar chart, 7 Q3 , third quartile, 22, 26–28 relative frequency, 7, 8 r, correlation coefficient, 36 Bayesian, 53, 123 Sx , sample standard deviation, 23, 25, 93, 94 bias, 52, 91, 95, 96, 110, 117 Sx2 , sample variance, 23, 25 biased coin, 64 Σ, summation notation, 17 bins, in a histogram, 12 σX , population standard deviation, 24, 25, 93, 94, bivariate data, 33 111, 112, 114, 116, 120, 121, 123 bivariate statistics, 2 2 σX , population variance, 24, 25 blinded experiment, 101 x, sample mean, 18, 19, 23, 24, 40, 93–95, boxplot, box-and-whisker plot, 27, 32 112–116, 118, 120–124 showing outliers, 28 xmax , maximum value in dataset, 22, 26–28 butterfly in the Amazon rainforest, 54 xmin , minimum value in dataset, 22, 26–28 yb, y values on an approximating line, 40 Calc [LibreOffice], 41, 42, 47, 83, 113, 121–123 zL∗ , critical value, 115 calculator, 24, 40, 78, 82, 89, 121, 122 categorical variable, 6 abortion, 95 causality, 91, 102, 103, 110 129 130 INDEX causation, 46 empty set, ∅, 56 center of a histogram, dataset, or distribution, 15 epidemiology, 117 Central Limit Theorem, CLT, 110–114, 121 equiprobable, 65 classes, in a histogram, 12 ethics, experimental, 91, 104 Clemens, Samuel [Mark Twain], ix even number, definition, 2 CLT, Central Limit Theorem, 110–114, 121 event, 55, 57–63 coin Excel [Microsoft], 41, 83, 113, 121–123 biased, 64 expectation, 72 fair, 64, 69, 70, 117 expected value, 72 complement, E c , 56–58, 60 experiment, 99, 102, 103 conditional probability, P (A | B), 67, 123 experimental design, 52, 91 confidence interval, 110, 111 experimental ethics, 52, 91 confidence interval for µX with confidence level experimental group, 100, 102 L, 115, 116 experimental treatment, 99 confidence level, 115, 116 explanatory variable, 33 confirmation bias, 120 extrapolation, 47 confounded, 102, 103 continuous random variable, 69, 90 failure to reject H0 , 118, 119, 123 control group, 100, 102 fair coin, 64, 69, 70, 117 CORREL, correlation coefficient in spreadsheets, fair, in general, 65 41 fake news, ix correlation coefficient, r, 36 false positive, 124 correlation is not causation finite probability models, 63 but it sure is a hint, 46 first quartile, 22 countably infinite, 69 first, do no harm, 104 critical value, zL∗ , 115 five-number summary, 27 free will, 104 data, not the plural of anecdote, 52 frequency, 7 dataset, 7 relative, 7 default significance level, 120 frequentist approach to probability, 53, 123 definition, in mathematics, 2 democracy, 96 Gallup polling organization, 96 density function, for a continuous random Gauss, Carl Friedrich, 78 variable, 74, 77, 112 Gaussian distribution, 78 dependent variable, 33 genetics, 117 deterministic, 34 “given,” the known event in conditional direction of a linear association, 35 probability, 67 disaggregation of experimental results, 105 Great Recession, 20 discrete random variable, 69 disjoint events, 57, 59, 62, 63 Hippocrates of Kos, 104 Disraeli, Benjamin, ix Hippocratic Oath, 104, 105 distribution, 15, 70, 73, 112 histogram, 12, 13, 32 do no harm, 104 relative frequency, 14 double-blind experiment, 101 How to Lie with Statistics, ix Huff, Darrell, ix Empirical Rule, 83 Hygieia, 104 INDEX 131 hypothesis, 124 multi-variable statistics, 2 alternative, Ha , 118–123 multimodal histogram, dataset, or distribution, 15 null, H0 , 118–121, 123, 124 mutually exclusive events, 57 hypothesis test, 110, 111, 117, 121–123 negative linear association, 35 imperfect knowledge, 66 news, fake, ix income distribution, 20 non-deterministic, 34 independent events, 65, 67, 112, 117, 124 NORM.DIST, the cumulative Normal distribution independent variable, 33 in spreadsheets, 83, 113, 121–123 individual in a statistical study, 5 Normal distribution with mean µX and standard inferential statistics, 110 deviation σX , 77, 112 informed consent, 105 normalcdf, the cumulative Normal distribution insensitive to outliers, 20, 23, 25, 26 on a TI-8x calculator, 82, 121, 122 Insert Trend Line, display LSRL in Normally distributed with mean µX and standard spreadsheet scatterplots, 42 deviation σX , N (µX , σX ), 78, 112, 114, 115 Institutional Review Board, IRB, 106 not, for an event, 56 inter-quartile range, IQR, 23, 25 null hypothesis, H0 , 118–121, 123, 124 interpolation, 43 intersection, ∩, 56, 57, 60, 61 objectivity, 52 IRB, Institutional Review Board, 106 observational studies, 103 observational study, 99, 102 Kernler, Dan, 83 one-tailed test, 122 one-variable statistics, 2 Law of Large Numbers, 94 or, for events, 56 leaf, in stemplot, 11 outcome of an experiment, 55 least squares regression line, LSRL, 40 outlier, 20, 25, 26, 28 left-skewed histogram, dataset, or distribution, 21 bivariate, 45 LibreOffice Calc, 41, 42, 47, 83, 113, 121–123 lies, ix p-value of a hypothesis test, 118, 119, 123 lies, damned, ix Panacea, 104 linear association, 35 parameter, population, 93–95, 110, 122, 124 lower half data, 22 personally identifiable information, PII, 105 LSRL, least squares regression line, 40 photon, 54 lurking variable, 102, 103 pie chart, 9 pig, yellow, 17 margin of error, 116 PII, personally identifiable information, 105 mean, 18–21, 25, 31, 112, 122 placebo, 100 population, 18, 93–95, 110–112, 114–116, Placebo Effect, 100, 104 118, 120–122 placebo-controlled experiment, 101 sample, 18, 19, 23, 40, 93–95, 112–116, 118, population mean, µX , 18, 93–95, 110–112, 114, 120–124 118, 120–122 media, 28 population of a statistical study, 5, 93, 112, 122, median, 18, 20, 21, 23, 25, 27, 31, 124 124 Microsoft Excel, 41, 83, 113, 121–123 population parameter, 93–95, 110, 122, 124 mode, 17, 19, 23, 31 population proportion, 93–95 MS Excel, 41, 83, 113, 121–123 population size, N , 6 132 INDEX population standard deviation, σX , 24, 25, 93, 94, sample mean, x, 18, 19, 23, 40, 93–95, 112–116, 111, 112, 114, 116, 120, 121, 123 118, 120–124 population variance, σX 2 , 24, 25 sample proportion, 94–96 positive linear association, 35 sample size, n, 6, 123 presidential approval ratings, 96 sample space, 55, 57, 58, 60–63 primum nil nocere sample standard deviation, Sx , 23, 25, 93, 94 see: first, do no harm, 104 sample statistic, 93, 95 probability density function, for a continuous sample variance, Sx2 , 23, 25 random variable, 74, 77, 112 sampling error, 116 probability model, 57 scatterplot, 35 probability theory, 52, 110 sensitive to outliers, 20–22, 25, 26, 28, 45 proof, 2 shape proportion histogram, 15 population, 93–95 scatterplot, 35 sample, 94–96 Show Equation, display LSRL equation in push-polling, 99 spreadsheets, 42 significance level, 119–124 quantitative variable, 6, 11, 17, 93, 94, 110–112, default, 120, 121 114, 116, 118, 120 simple random sample, SRS, 97, 98, 110–116, quantum mechanics, 54, 117 118–121, 123–125 quartile, 22, 26, 27, 31 Simpson’s Paradox, 48 QUARTILE.EXC, quartile computation in skewed histogram, dataset, or distribution, 15, 21 spreadsheets, 25 left, 21 QUARTILE.INC, quartile computation in right, 21 spreadsheets, 25 slope of a line, 35, 39 spread of a histogram, dataset, or distribution, 15, random variable, RV, 69, 112 22–26 randomized experiment, 101 spreadsheet, see LibreOffice Calc and MS Excel randomized, controlled trial, RCT, 91, 101 SRS, simple random sample, 97, 98, 110–116, randomized, placebo-controlled, double-blind 118–121, 123–125 experiment, 52, 91 standard deviation, 23–25, 31, 32, 93, 94, 111, randomness, 52, 95, 110, 117, 124 112, 114, 116, 120–123 range, 22, 25, 31, 32 standard Normal distribution, N (0, 1), 81 RCT, randomized, controlled trial, 52, 101 standard Normal RV, 81 rejection of H0 , 118, 119, 121, 123, 124 standardizing a Normal RV, 82, 83 relative frequency, 7 statistic, sample, 93, 95 representative sample, 95 statistically indistinguishable, 102 residual, for data values and LSRLs, 39 statistically significant, for data in a hypothesis response variable, 33 test, 119, 121, 123 right-skewed histogram, dataset, or distribution, STDEV.P, population standard deviation in 21 spreadsheets, 25 rise over run, see slope of a line STDEV.S, sample standard deviation in RV, random variable, 69, 112 spreadsheets, 25, 41 stem, in stemplot, 11 sample, 6, 110, 112, 119, 122–124 stem-and-leaf plot, stemplot, 11 INDEX 133 strength of an association, 35 ”We reject the null hypothesis H0 .”, 118, 119, strong association, 35 121, 123, 124 strong statistical evidence, 118 weak association, 35 subset, ⊂, 55, 57 wording effects, 95, 107 sugar pill, 100 y-intercept of a line, 39 summation notation, Σ, 17 survey methodology, 91 symmetric histogram, dataset, or distribution, 15, 124 test of significance, 110, 111 thermodynamics, 117 third quartile, 22 treatment, experimental, 99 Tufte, Edward, 46 Twain, Mark [Samuel Clemens], ix two-tailed test, 122 type I error, 124 unethical human experimentation, 104 uniform distribution on [xmin , xmax ], 75 unimodal histogram, dataset, or distribution, 15 union, ∪, 56, 57, 60 upper half data, 22 utilitarianism, 104 VAR.P, population variance in spreadsheets, 25 VAR.S, sample variance in spreadsheets, 25 variability, see spread of a histogram, dataset, or distribution variable, 6, 93, 122, 124 categorical, 6 dependent, 33 explanatory, 33 independent, 33 quantitative, 6, 11, 17, 93, 94, 110–112, 114, 116, 118, 120 response, 33 variance, 23–25 Venn diagram, 57–62 voluntary sample bias, 97 voters, 5 ”We fail to reject the null hypothesis H0 .”, 118, 119, 123