find the area of a cardboard wasted if a sector of maximum possible size is cutout from a square card board of size 24cm

That depends. Which part is wasted - the part cut out, or the part left behind?

There's some missing information here, like relative orientation of the squares, etc.