Would you put a comma between huge glass windows?

Yes, a comma is necessary between "huge" and "glass windows" as it is separating two adjectives. The correct phrase would be "huge, glass windows."