A man hires a car with the fuel tank 5/8 full. He travels 180 miles and is left with 2/8 of a tank. How much fuel will he need to put in the car at the start of his return journey of 200 miles to ensure he has 5/8 of a tank when he gets back with no wasted cost?

fuel used to go 180 miles = 5/8 tank - 2/8 tank

= 3/8 tank

so to go 200 miles, he would use
(3/8)(200/180) tanks
= 5/12 of the tank

total used = 3/8 tank + 5/12 tank = 19/24 tank
so he must top up his tank by the total he used
which is 19/24 of the tank.
but he has 1/4 tank left
1/4 + 19/24 = 1 1/24 tank, HE OVERFILLED it
There is a flaw in this question, he cannot put in enough
at the end of the first trip to last the return trip and have 5/8 of a tank left

Check:
suppose his full tanks holds 96 L (I picked that since all my denominators divide evenly)

so he starts with 5/8 of 96 or 60 L
he goes 180 miles and has 24 L left , using36 L
consumption is :
180 miles --- 36 L
1 mile --- 36/180 L
200 miles ---- (36/180)(200) L = 40 L

so he used 36 L + 40 L or 76 L
But he can't put in 76 L because he still has 24 L and the tank would have to be 24+76 or 100 L
(If it did hold 100L then he would use the 40L on the return trip and be left with the necessary 60L)

All he could do is fill the tank with 72 L at the end of the first trip obtaining 96 L , then use 40 L to have 56 L
Now at the end you top it up with 4 L more
notice the total added would be 72 + 4 or 76 L