Vector A has magnitude 3.87 units; vector B has magnitude 5.63 units. The angle between A and B is 60.0°. What is the magnitude of A and B?

You have already said what the magnitudes of A and B are. Call them |A| and |B|. I assume you want the magnitude of vector sum A + B

Use the law of cosines.

|A+B|^2 = |A|^2 + |B|^2 - 2|A||B| cos 60