I have 28 participants. Each completed five tests. The maximum score a participant could receive on any one test was 4/4. The lowest score anyone could receive was zero.
I will do a Friedman test for this paired data.
My question: given that I have a very large number of identical ranks (mostly participants getting zero in all tests) can I still go ahead? I just want to know if there is a significant difference in the medians.
Thanks!
Yes, assuming the zero was a valid score, as the person being tested didn't know the subject matter. If the zero is a zero for other reasons (absent, being hardheaded and refusing the test, or similar), then the conclusion on difference is likely to be invalid.
I am reminded of instances were kids put their name on answer sheets, and turn it in as an act of rebellion. You can't make a conclusion about the meaning of scores in that case.
Thanks very much Bob. I much appreciate your taking the trouble to respond.
All the best!
Colin
After reading Bob's comment, I can't help but remember the best French student I had purposely getting himself a zero but not only putting a wrong answer but the very worst answer! The principal was so angry he wanted to suspend that student but instead I persuaded him to just delete that score, because I knew why he had done that! I'm happy to say that student went on to be a wonderful surgeon.
Sra (aka Mme)
To determine whether you can proceed with a Friedman test despite having a large number of identical ranks, we need to consider the assumptions and requirements of the test.
The Friedman test is a non-parametric test used to analyze paired data when the dependent variable is ordinal (ranked) and the data are not normally distributed. The test compares the medians of different groups or conditions.
In your case, you have 28 participants who completed five tests, each with a maximum score of 4/4 and a minimum score of 0/4. However, you mentioned that most participants obtained a score of zero in all tests.
When conducting a Friedman test, it is important to have variation in the ranks. If most of your participants received zero in all tests, there might not be enough variation to evaluate the differences in medians among the tests accurately. In this scenario, the Friedman test may not be appropriate, as the lack of variability in the data could affect the statistical analysis and the validity of the results.
A possible alternative could be to use a different statistical test that is more suitable for your specific data. For example, if you are interested in comparing the medians of the tests, you could consider using a non-parametric analog to the one-way ANOVA, such as the Kruskal-Wallis test. This test can handle multiple groups or conditions and does not require as much variation in the data as the Friedman test.
Before proceeding with any statistical test, it is recommended to consult with a statistician or an expert in your specific field to ensure the appropriateness of the chosen method and to discuss potential alternatives based on your data characteristics.