A pendulum will have minimum potential energy when the displacement from its equilibrium position is at its maximum. In other words, the pendulum will have minimum potential energy when it is at the highest point of its swing, where it has maximum height and zero velocity.
