integration of x^2 * cos^2(x)

|x^2*cos^2(x) dx =

=|x^2*(1/2)(1+cos(2x)) dx
=(1/2)|x^2 dx + (1/2)|x^2*cos(2x) dx
=(1/6)x^3 + (1/2)|x^2*cos(2x) dx

Using integration by part:
u = x^2
du = 2x dx
dv = cos(2x) dx
v = (1/2) sin(2x)
|x^2*cos(2x) dx =
= (1/2)x^2*sin(2x) - |xsin(2x) dx

Again, using integration by part:
u = x
du = dx
dv = sin(2x) dx
v = -(1/2)cos(2x)
|x^2*cos(2x) dx =
= (1/2)x^2*sin(2x) - {-(1/2)x*cos(2x) + (1/2)|cos(2x) dx}
= (1/2)x^2*sin(2x) + (x/2)cos(2x) - (1/4)sin(2x) + const

Finally, we have:
|x^2*cos^2(x) dx =
= (1/6)x^3 + (1/4)x^2*sin(2x) + (x/4)cos(2x) - (1/8)sin(2x) + const