The Zambezi River runs through the easternmost part of this country...is it Zambia?

Yes.

http://www.britannica.com/EBchecked/topic/655540/Zambezi-River