1. Hamburg is the second largest city in Germany.
2. Hamburg is the second-largest city in Germany.
Which one is correct? Do we have to use "-" or not? Are both acceptable?
Both are used, yes. I'd use the second one because two adjectives are being put together to be used as one adjective, thus the hyphen.
Hyphens have other uses
~ creating compound words, particularly modifiers before nouns (the well-known actor, my six-year-old daughter, the out-of-date curriculum
~ writing numbers twenty-one to ninety-nine and fractions (five-eighths, one-fourth)
~ creating compounds on-the-fly for fly-by-night organizations
adding certain prefixes to words: When a prefix comes before a capitalized word or the prefix is capitalized, use a hyphen (non-English, A-frame, I-formation). The prefixes self-, all-, and ex- nearly always require a hyphen (ex-husband, all-inclusive, self-control), and when the prefix ends with the same letter that begins the word, you will often use a hyphen (anti-intellectual, de-emphasize), but not always (unnatural, coordinate, cooperate). By all means, use a good dictionary when in doubt! For further information about compound nouns and compound modifiers, see the separate section on Compound Words.
from http://guidetogrammar.org/grammar/marks/hyphen.htm