1. He is looking at the baby happily.

2. Happily, he is looking at the baby.
2-2. He is happily looking at the baby.
3. He looks around happily.
4. They are talking with each other
happily.

(Are they all grammatical? What is the difference between #1 and #2?)

All are fine.

There is no real difference between 1 and 2 except the placement of the adverb.