f(x)=4sq.rt(x+5), g(x)= (x^2+5)

g of f =

g o f = g(f(x))

=g(4√(x+5))

So replace x by 4√(x+5) in the expression for g(x).

See also
http://www.jiskha.com/display.cgi?id=1296238552