An open box will be made from a rectangular piece of cardboard that is 8 in. by 10 in. The box will be cut on the dashed red lines, removing the corners, and then folded up on the dotted lines. The box needs to have the MAXIMUM volume possible. How long should the cuts be?

Let the cut-outs be x by x in.

length of box = 10-2x
width of box = 8-2x
height of box = x , clearly 0 < x < 4 to have a box

volume = x(10-2x)(8-2x)
= 80x - 36x^2 + 4x^3
d(volume)/dx = 80 - 72x + 12x^2 = 0 for a max of volume

12x^2 - 72x + 80 = 0
3x^2 - 18x + 20 = 0

solve using the formula, use the x that fits 0 < x < 4