d/dx 1/cos[3cosx-2sinx)/sqrt3]

unbalanced parentheses, and an odd function. Is there an arccos somewhere in there?

If you meant

d/dx 1/cos[(3cosx-2sinx)/√3]
d/dx √3 sec(3cosx-2sinx)
= √3sec(3cosx-2sinx)tan(3cosx-2sinx) * (-3sinx-2cosx)

Not much that can be done with that.