(5x^-3)^2

just multiply exponents when raising an expression to a power:

(5x^-3)^2 = 5^2 (x^-3)^2
= 25 x^(-3*2)
= 25x^-6

recall that (x^-3)^2 is just x^-3 * x^-3 = x^(-3 + -3) = x^-6