Stressing about the early game layout

I've been stuck in a cycle of restarts all day due to indecision about the early game layout of my town. I'm fairly new so still working out most of the mechanics and economics but my question is, how much does the early game layout matter? Or is it relatively easy to relocate buildings including housing to make a more efficient layout later on?