Introduction to Hypothesis Testing

In an article entitled “Science and ethics in conducting, analyzing, and reporting psychological research” in the journal Psychological Science (1994), psychologist Robert Rosenthal examines the relationship between a researcher’s methods and the ethical implications of these methods. In particular, he examines what he terms "causism," defined as “the tendency to imply a causal relationship where none has been established (i.e., where the data do not support it)” (p.128). He goes on to state that causism can arise from the language chosen to describe the results of hypothesis tests, especially when authors use words such as "consequence" or "cause" instead of "related to" or "inferred from." He argues that the stronger language can be misleading, causing the result to “appear more important or fundamental than it really is” (p. 128). This, in turn, misleads the public into drawing conclusions or implementing policies that could be based on faulty assumptions. “As shown in the previous section, our results were significant at the .05 level (p = .047, n = 23). These highly significant results prove the research hypothesis that there is a higher incidence of Birth Defect X in babies of mothers who consume prenatal vitamins daily versus babies whose mothers do not consume such vitamins. Thus, we can state with some certainty that taking daily doses of prenatal vitamins during pregnancy can cause the birth defect in question in many cases. This is a landmark study, the first of its kind that examines the link between prenatal vitamins and the consequent appearance of Birth Defect X” (Dewey, Cheatum, & Howe 2000)., discuss the errors in the conclusions the authors have drawn from their statistical tests.

This should help:

http://drdavespsychologypage.homestead.com/Proving_a_Point.pdf

What are some of the errors

The authors in the provided example have made errors in their conclusions by misinterpreting the results of their statistical tests. Let's break down the errors in their conclusions:

1. Inappropriate language: The authors use strong language such as "prove" and "cause" to describe the relationship between prenatal vitamins and the birth defect. This language implies a causal relationship where none has been established, which is known as "causism." The use of such language goes beyond the evidence provided by statistical tests and can mislead readers.

2. Misunderstanding statistical significance: The authors state that their results were "significant at the .05 level," but this does not mean that the findings prove a causal relationship. Statistical significance implies that the observed effect is unlikely to have occurred by chance alone, but it does not establish causation. It only indicates that the data provide evidence to reject a null hypothesis, suggesting a relationship exists. However, other factors could be influencing the observed association.

3. Small sample size: The authors mention a sample size of 23, which is relatively small. Small sample sizes can lead to less reliable results and a higher likelihood of obtaining false positives or false negatives. With a small sample, it becomes more challenging to generalize the findings to a larger population, and conclusions should be drawn cautiously.

4. Overgeneralization and exaggerated claims: The authors claim that their study is the "first of its kind" and that it provides definitive evidence for the relationship between prenatal vitamins and the birth defect. Such claims should be supported by multiple studies and robust evidence, not solely based on a single study. Overgeneralization and exaggerated claims misrepresent the significance and implications of the research.

To summarize, the errors made by the authors include the use of inappropriate language, misinterpreting statistical significance as proof of causation, relying on a small sample size, and making overgeneralized and exaggerated claims. It is essential to interpret statistical results accurately and avoid drawing unwarranted conclusions beyond the evidence provided.