a^2-b^2+8bc-16c^2

25-m^2-12mn-36n^2

x^2-a^2-y^2-2ay

I know that you have to group them 3 and 1 and then you have to do difference of squares but I am utterly lost how to go about it. Please help!

You just need to recognise the square of a sum of two terms like (x+y)^2 via the term 2 x y. If you do that, you will find all the squares and then you can see the difference of the squares.

So, in the expression:

a^2-b^2+8bc-16c^2

you have the term 8 bc. So, there must be a linear combination of b and c that is squared. We also have the terms -b^2 and -16 c^2 = - (4c)^2, so we can cpmbine all these terms into:

-(b - 4c)^2

The remaining term is a^2, so we have:

a^2 - (b - 4c)^2 =

[a - (b-4c)][a + (b-4c)] =

(a - b + 4 c)(a + b - 4c)