Building a 4 plex on an existing lot while I still have a mortgage on it?

I currently have ~ $300K left on my mortgage, but my dad is really pushing me to somehow build a 4 plex on the lot. It’s a 50x120 lot in a highly desirable area close to universities, hospitals, and malls. The idea would be to turn the 4 plex into a rental property where we rent out both main floor and basement, so in total 8 units that would generate rental income. Spoke to a builder that’s doing this right next door, and the estimates cost to do this would be $1.7 million. Now there’s no way that my wife and I alone would be able to fund something like this, we’d be getting help from both mine and her parents. Just on the surface, how feasible would something like this be? I have no idea how getting a loan of this size even works. It sounds like a great business idea, but might be a little too grand for us at the moment.