That day in June he had swum in the ocean. Is swum the proper verb for this sentence?

Yes... that is correct.