1. When I came home, I saw a stranger near my house.

2. When I was coming home, I saw a stranger near my house.
=================================
Q1: Which one is a common expression?
Q2: What does 'near my house' modify? Does it modify 'a stranger' or the verb 'saw'?

Remember that English likes to have modifiers as close to what they're modifying as possible. So "near my house" is modifying "stranger."

Both sentences are grammatically correct. The first uses fewer words, thus is used more often.