1. She studied hard, only to fail.

2. She studied hard, but failed.
3. She studied hard, but she failed.

(Does #1 mean #2 and #3? Are they all grammatical? Do we have to put a comma before only?)

All are correct, and all mean about the same thing. And, yes, you need the comma.