solve for x:

(x^2-2x)^2 -11(x^2-2x)+24

if you mean

solve (x^2-2x)^2 -11(x^2-2x)+24 = 0, then let u = x^2 - 2x and you have

u^2 - 11u + 24 = 0
(u-8)(u-3) = 0
now plug back in for x:
(x^2-2x-8)(x^2-2x-3) = 0
(x-4)(x+2)(x-3)(x+1) = 0

That help?