Both of the girls insisted that they try to reach the end of the trail by themselves.

I know they and themselves are pronouns. I know both can be used as a pronoun, but I am thinking no here. Am I right?

Both are pronouns, yes ... and are used as pronouns in this sentence.

? So Both at the beginning of sentence is also a pronoun

Both of the girls

http://grammar.ccc.commnet.edu/grammar/pronouns1.htm

Yes, "Both" is an indefinite pronoun.
And "they" is a personal pronoun.
And "themselves" is a reflexive pronoun.

Be sure to read through the different types of pronouns on this webpage.

No, in this sentence, "both" is not being used as a pronoun. It is being used as a determiner.

In the sentence, "both of the girls insisted that they try to reach the end of the trail by themselves," "both" is used to indicate that both of the girls (people) insisted on something. It is modifying the noun "girls" and functioning as a determiner, specifying a specific group of people.

Pronouns generally replace nouns and can stand alone to refer back to a previously mentioned noun or nouns. In this sentence, "they" and "themselves" are pronouns because they are used to replace the noun "girls." "They" is a pronoun that functions as a subject, and "themselves" is a pronoun that functions as an reflexive object, referring back to the subject "they."