Using the "completing the square" technique, the maximum value of the function (x)=sin^2(2x)-2cos^2x is:
f(x) = sin^2 (2x) - 2cos^2 x
= 1 - cos^2 (2x) - 2cos^2 x
I know cos (2x) = 2cos^2 x - 1
then 2cos^2 x = cos (2x) + 1
f(x) = 1 - cos^2 (2x) - cos(2x) -1
= -cos^2 (2x) - cos(2x)
let a = cos(2x) , then
f(x) = -a^2 - a
I guess this is where the "completing the square" part kicks in
f(x) = -(a^2 + a + 1/4 - 1/4)
= -(a + 1/2)^2 + 1/4
So f(x) has a max of 1/4 when a = -1/2
that is, when cos(2x) = -1/2
2x = 120° or x = 60° which is π/3
max is 1/4, when x = 60°
graphing y = sin^2 (2x) - 2cos^2 x
on Desmos confirms this:
www.desmos.com/calculator