The shrinking American backyard is getting even smaller, and it partly has to do with mortgage rates.