Table of Contents### Analyzing Bottom-Line Data

Entire Site

Be careful when analyzing bottom-line data. For example, suppose the target goal is to ensure that a person new to a Web site can find and purchase an item in 20 minutes or less. When running our test, we get times of 20, 15, 45, 10, 5, and 25 for our six participants. The mean or average time for this is 20 minutes. Looks pretty good! The median for this set of numbers is 17.5—even better!

However, the problem is that there is very little certainty here because there are only six participants and the results are highly variable. If you calculate the standard deviation, a measure of how variable the numbers in this set are, you will find that the value is approximately 14. If we divide the standard deviation by the square root of the number of samples we have (6), we get 5.8. This is the standard error of the mean, and it tells us how much variation we can expect in the typical value. It is plausible that the typical value is as small as the mean minus twice the standard error of the mean, resulting in a lower bound of 8.5, or as large as the mean plus twice the standard error of the mean, or 32. This latter value would clearly be far from our stated goal of 20 minutes!