Why does the potential inside a solid conductor not change?

If there were a potential difference current would flow.

i = V/R
where V is the potential difference
and R is the resistance
In a perfect conductor the resistance is zero and and V ( potential difference) would lead to infinite current to wipe out the potential difference.
Imagine taking the water out of one side of a lake. Then you would have quite a (hydraulic) potential difference
between one side and the other. If you removed your barrier, the water would flow like crazy until the water level (the potential) was the same both sides of where your barrier was and the flow stopped.