Significance Levels
If a test of significance gives a
Learning Objective

Outline the process for calculating a
$p$ value and recognize its role in measuring the significance of a hypothesis test.
Key Points
 Significance levels may be used either as a cutoff mark for a
$p$ value or as a desired parameter in the test design.  To compute a
$p$ value from the test statistic, one must simply sum (or integrate over) the probabilities of more extreme events occurring.  In some situations, it is convenient to express the complementary statistical significance (so 0.95 instead of 0.05), which corresponds to a quantile of the test statistic.
 Popular levels of significance are 10% (0.1), 5% (0.05), 1% (0.01), 0.5% (0.005), and 0.1% (0.001).
 The lower the significance level chosen, the stronger the evidence required.
Terms

Student's ttest
Any statistical hypothesis test in which the test statistic follows a Student's
$t$ distribution if the null hypothesis is supported. 
pvalue
The probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.
Full Text
A fixed number, most often 0.05, is referred to as a significance level or level of significance. Such a number may be used either as a cutoff mark for a
$p$ Value
In brief, the (lefttailed)
$p$ Value Graph
Example of a
Hypothesis tests, such as Student's
Using Significance Levels
Popular levels of significance are 10% (0.1), 5% (0.05), 1% (0.01), 0.5% (0.005), and 0.1% (0.001). If a test of significance gives a
In some situations, it is convenient to express the complementary statistical significance (so 0.95 instead of 0.05), which corresponds to a quantile of the test statistic. In general, when interpreting a stated significance, one must be careful to make precise note of what is being tested statistically.
Different levels of cutoff trade off countervailing effects. Lower levels – such as 0.01 instead of 0.05 – are stricter and increase confidence in the determination of significance, but they run an increased risk of failing to reject a false null hypothesis. Evaluation of a given
Key Term Reference
 binomial distribution
 Appears in these related concepts: Goodness of Fit, Comparing Two Populations: Paired Difference Experiment, and Experimental Probabilities
 cumulative distribution function
 Appears in these related concepts: Continuous Probability Distributions, The Uniform Distribution, and Optional Collaborative Classrom Activity
 datum
 Appears in these related concepts: Change of Scale, Controlling for a Variable, and Type I and II Errors
 distribution
 Appears in these related concepts: Application of Knowledge, Monte Carlo Simulation, and Selling to Consumers
 graph
 Appears in these related concepts: Graphical Representations of Functions, Graphing Equations, and Graphs of Equations as Graphs of Solutions
 hypothesis test
 Appears in these related concepts: Hypothesis Tests or Confidence Intervals?, Wilcoxon tTest, and Summary for inference of the difference of two means
 level
 Appears in these related concepts: Factorial Experiments: Two Factors, Statistical Controls, and Randomized Design: SingleFactor
 null hypothesis
 Appears in these related concepts: Statistical Power, Example: Test for Goodness of Fit, and When Does the ZTest Apply?
 probability
 Appears in these related concepts: Theoretical Probability, Rules of Probability for Mendelian Inheritance, and The Addition Rule
 sample
 Appears in these related concepts: Identifying Product Benefits, Surveys, and Basic Inferential Statistics
 sampling
 Appears in these related concepts: Collecting and Measuring Data, Continuous Sampling Distributions, and Confidence Interval, Single Population Mean, Standard Deviation Unknown, Student'st
 sampling distribution
 Appears in these related concepts: Sampling Distributions and the Central Limit Theorem, Creating a Sampling Distribution, and Properties of Sampling Distributions
 significance level
 Appears in these related concepts: Using the Model for Estimation and Prediction, Elements of a Hypothesis Test, and Interpreting NonSignificant Results
 statistical significance
 Appears in these related concepts: Tests of Significance, Was the Result Significant?, and Was the Result Important?
 statistics
 Appears in these related concepts: Communicating Statistics, Population Demography, and Understanding Statistics
Sources
Boundless vets and curates highquality, openly licensed content from around the Internet. This particular resource used the following sources: