what kind of grammatical construction is "whom he had previously enjoyed'?

It's an adjectival dependent clause.