Their isolation towards others expresses that them growing up in a different environment affects their social skills.

Would I use affect or effect

affects

You need a correction, though: Use "their" instead of "them" in front of "growing."
http://grammar.ccc.commnet.edu/grammar/gerunds.htm#possessive

I'd also omit "towards others" since isolation takes care of the idea.