56x(3)y(5)-40x(2)y(6)+8x(2)y(3)

Online, use "^" to indicate an exponent.

Is this what you mean?

56x^3(y^5) - 40x^2(y^6) + 8x^2(y^3)

What do you want to do with this formula?