suppose that the teacher of this class graded solely on a curve - 10% get A, etc.

How might this effect students getting into into study groups to prepare homework or tests?

It depends on the mind set of the participants in each group. If they see themselves competing against other groups, they will do their best to help each other. However, if they view themselves as competing with others within the group, there may be sabotage or at least a lack of cooperation.