From Apple Maps, it looks like it’s on a mostly undeveloped island, with the nearest town, Cape Canaveral, over ten miles away. I can’t tell how far away Orlando is, but much further.
Compared to this Chinese test site, it looks like population centers are 5x - 10x farther, plus you have an entire ocean to blow stuff up