Rewrite: I went to town.

Start with: Went.....

English has a normal word order:

subject ... verb ... [indirect object] ... direct object or other type of complement.

To start with the verb is not normal English word order. Please re-think this.