1. They went over to the other side of the river in a boat.

2. They went over to the other side of the river on a boat.

3. They went over to the other side of the river by boat.
==============
Are they all grammatical? Which one is commonly used?

All three are correct. See my answer to the previous question. :)