What is the relationship between levels of confidence and statistical significance?
The relationship between levels of confidence and statistical significance is that they are both measures used in hypothesis testing and inferential statistics.
Levels of confidence, often denoted by a value such as 90%, 95%, or 99%, represent the degree of certainty or belief we have in the reliability of an estimate or hypothesis. In other words, it indicates the range within which we are confident the true population parameter lies. For example, if we have a 95% confidence level, it implies that if we were to repeatedly sample from the population and compute a confidence interval, about 95% of those intervals would contain the true population parameter.
Statistical significance, on the other hand, is a measure of how likely it is that the observed effect or relationship in the data is real and not just due to chance. It is typically determined by conducting a hypothesis test, where a specific null hypothesis (usually stating that there is no effect or relationship) is tested against an alternative hypothesis. If the p-value (the probability of obtaining the observed result or more extreme under the null hypothesis) is below a pre-determined threshold, often 0.05, we say the result is statistically significant.
So, the relationship between levels of confidence and statistical significance is that they both involve quantifying the probability of making a correct inference based on the data. However, while levels of confidence focus on the precision and range of estimation, statistical significance focuses on the strength of evidence against the null hypothesis. It's important to note that they are not directly related to each other, and we can have high levels of confidence without statistical significance, and vice versa.