Cultural diffusion and population shifts