The patient was advised to return to the clinic (immediately) for any redness or swelling.

immediately is that an adverb?

yes, it modifies "to return," a verb. ("to the clinic" is a prepositional phrase)