Use the half-angle formula to find the exact value of cos(x/2) when tanx = -5/8 and x is in quadrant IV. (| is a radical)

==> I got sqrt((89 + 8sqrt(89))/178). Is this correct? I hope my answer is easy to read..

The answer is correct.

In fact, you have nicely rationalized the denominator, which is expected of students.
You could also express the answer as:
cos(x)=sqrt((1/2)+4sqrt(89)/89)
if it appeals to you.