Statistical Concepts

Session 2: Hypothesis Testing

The Binomial Distribution
Each trial can have two outcomes, e.g. success and failure.
The probability of success is p and failure is q, therefore p+q=1
If the trial is repeated n times then the total number of successes is x.
The probability of a success on each trial must be independent of the results of previous trials.

Pascal's Triangle

 1
 1 1
 1 2 1
 1 3 3 1
 1 4 6 4 1
 1 5 10 10 5 1
 1 6 15 20 15 6 1
 1 7 21 35 35 21 7 1
 1 8 28 56 70 56 28 8 1
 1 9 36 84 126 126 84 36 9 1
 1 10 45 120 210 252 210 120 45 10 1

The contents of each cell can be calculated using the equation:

 nCx = ___n!___ x!(n-x)!

n = number of trials (first row is 0 trials)
x = number of successes (first cell in each row is 0 successes)

The probability of x successes in n trials can be calculated using the equation: P(x) = nCxpxq(n-x)
If p = q = ½, the equation simplifies to:
 P(x) = nCx 2n

Hypothesis Testing

Usually the aim of an exeriment is to demonstrate a significant difference or relationship between groups or sets of results.

Significant means "unlikely to have occurred by chance".

If I toss a coin 10 times and get 8 heads and 2 tails, I have found a difference. What I really want to know is whether this difference is unlikely to have occurred by chance. If it is unlikely then it suggests that the coin may be biased.

The Null Hypothesis in this case is that the coin is fair.
The Alternative Hypothesis is that the coin is biased.

In order to assess the evidence, I make an assumption that the Null Hypothesis is true. I can then calculate the probability that this hypothesis is true, which is the probability of obtaining a result as unlikely, or more unlikely, as the actual result.

 Number of heads 0 1 2 3 4 5 6 7 8 9 10 nCx 1 10 45 120 210 252 210 120 45 10 1 Probability 0.00098 0.00977 0.04395 0.11719 0.20508 0.24609 0.20508 0.11719 0.04395 0.00977 0.00098

You can check that the total probability adds up to 1 (I have).

The probability of 8 heads is .04395.
The probability of 8 or more heads is .04395 + .00977 + .00098 = .05470

However, 2 heads or fewer would also be as unlikely, so the probability of a result at least as unlikely as 8 heads is:
.00098 + .00977 + .04395 + .04395 + .00977 + .00098 = .10940

So, does obtaining 8 heads out of 10 suggest that the coin is biased?
NO, because the probability of a fair coin giving a result at least unlikely as this is 11%.

You can also this of this probability (aka significance value) being the chance that the Null Hypothesis is true.

In Psychology, the normal Level of Significance to use is 5%.

If the probability of the Null Hypothesis being true is less than 5%, then the result is Significant because there is significant evidence that the Null Hypothesis is false and the Alternative Hypothesis is true.

Statistical Tests

Usually, it is not possible to calculate the probability directly, so a Statistical Test is used. The observed result is used to calculate a statistic (e.g. F, t, Chi-squared, U). This statistic is then compared with critical values in tables or is used by a computer to calculate the probability. For many tests, if the statistic has a large value, the corresponding probability is small.