Suppose we have a test statistic T_ n such that T_ n \sim |Z| where Z \sim N(0,1). In particular, for this problem we know the distribution of T_ n for any fixed n and not just asymptotically. You design the test
\psi _ n = \mathbf{1}(T_ n \geq q_{\eta /2})
where q_\eta is the 1-\eta quantile of a standard Gaussian (i.e., if Z \sim N(0,1), then P(Z > q_{\eta }) = \eta). If \psi = 1, we will reject H_0, and if \psi = 0, we will fail to reject H_0.
With this set-up, you observe a data set and compute T_ n. Consider the following figure:
On which side, to the left or to the right, of T_ n should the value q_{\eta /2} be such that \psi _ n rejects on our data set?
The value q_{\eta /2} should be on the right side of T_n.
In the design of the test, we reject the null hypothesis H_0 if T_n is greater than or equal to q_{\eta /2}. This means that T_n needs to be sufficiently large in order for us to reject the null hypothesis. Therefore, the critical value q_{\eta /2} should be on the right side of T_n, indicating that the rejection region is in the right tail of the distribution.
What is the largest value of q_{\eta /2} such that \psi _ n rejects on our data set?
The largest value of q_{\eta /2} such that \psi _ n rejects on our data set would be the value that corresponds to the significance level (also known as Type I error rate) of the test, which is often denoted as α.
Since α = 1 - \eta, where \eta is the desired confidence level, we can find the largest value of q_{\eta /2} by finding the 1 - α quantile of the standard Gaussian distribution.
Therefore, the largest value of q_{\eta /2} such that \psi _ n rejects on our data set is the value that corresponds to the desired significance level or Type I error rate of the test.
What is the smallest value of \eta so that \psi _ n rejects on our data set? (Note that this is the p-value for our data set.)
\eta =2\times (\text {the area under the curve to the right of A})
\eta =2\times (\text {the area under the curve to the right of B})
\eta =2\times (\text {the area under the curve to the right of C})
unanswered
To find the smallest value of η such that Ψ_n rejects on our data set (i.e., the p-value), we need to calculate the area under the curve to the right of T_n.
Let's consider the three options given:
η = 2 × (the area under the curve to the right of A)
η = 2 × (the area under the curve to the right of B)
η = 2 × (the area under the curve to the right of C)
We can rule out options A and C because we are looking for the smallest value of η, and those options would give us larger areas under the curve.
Therefore, the correct option is:
η = 2 × (the area under the curve to the right of B)
The smallest value of η that satisfies our condition is 2 times the area under the curve to the right of T_n (B) for our given data set. This is equivalent to the p-value of the test.
Now you observe a new data set and compute a new value of the test statistic, which we denote by T_ n'. Suppose that T_ n' < T_ n, i.e., the test statistic has a smaller value than from before.
Will the new p-value be larger or smaller than the p-value from the previous data set considered in this problem?
If T_n' < T_n (i.e., the new test statistic value is smaller than the previous test statistic value), then the new p-value will be larger than the p-value from the previous data set.
The p-value represents the probability under the null hypothesis of observing a test statistic as extreme as or more extreme than the observed test statistic. When the test statistic is smaller, it means that the observed data is closer to the null hypothesis, and therefore, the p-value is larger. A larger p-value indicates weaker evidence against the null hypothesis and suggests that the data is more likely to be consistent with the null hypothesis.