No, people did. Slavery was abolished in 1865, about 160 years ago. Slaves also didn’t really build that much, they worked on plantations. While there were definitely some contributions by slaves in the past, this country was primarily built by non slaves.
Or, it’s the fact that we can’t afford...