how do I figure out if one million dollars in 100 dollar bills can fit into a briefcase measuring 17" by 12" by 3.5"

10,000 bills

A $100 bill is .0043 inches thick.

Find the other dimensions.

Take it from there.