Their isolation towards others expresses that them growing up in a different environment affects their social skills.
Would I use affect or effect
affects
You need a correction, though: Use "their" instead of "them" in front of "growing."
http://grammar.ccc.commnet.edu/grammar/gerunds.htm#possessive
I'd also omit "towards others" since isolation takes care of the idea.