What are significance/alpha levels and how does using a one versus a two tailed test affect significance levels?
Significance levels, also known as alpha levels, are a way to determine the threshold at which we reject or fail to reject a null hypothesis in statistical hypothesis testing. It represents the probability of making a Type I error, which is rejecting the null hypothesis when it is actually true.
The significance level is typically set before conducting the hypothesis test and is denoted by the Greek letter alpha (α). The most commonly used significance levels are 0.05 (5%) and 0.01 (1%), although other values can be chosen based on the specific context and requirements of the study.
When conducting a hypothesis test, we compare the calculated test statistic with a critical value determined by the chosen significance level. If the test statistic falls in the critical region (i.e., beyond the critical value), we reject the null hypothesis. Otherwise, if the test statistic falls in the non-critical region, we fail to reject the null hypothesis.
Now, the choice between using a one-tailed or a two-tailed test depends on the research question and the alternative hypothesis we are investigating.
In a one-tailed test, we are only interested in determining if the parameter is significantly different in one specific direction. For example, we might want to know if a new drug has a positive effect, without considering the possibility of a negative effect. In this case, all the significance level is allocated to one tail of the distribution, making the critical region smaller and the test more powerful.
In a two-tailed test, we are interested in determining if the parameter is significantly different in either direction. For example, we might want to investigate if a new fertilizer increases crop yield, considering both the possibility of a positive or negative effect. In this case, the significance level is divided equally between the two tails of the distribution, making each critical region larger and the test less powerful compared to a one-tailed test.
In summary, significance levels or alpha levels are thresholds used in hypothesis testing to determine if we reject or fail to reject the null hypothesis. The choice between a one-tailed or two-tailed test depends on the research question and affects the allocation of the significance level to the respective tails of the distribution.