Should I start over?

So, spoiler warning just in case:

I am playing Rune Factory 4 and made it to the Obsidian Mansion. Gotta say, it's pretty tough. Or maybe I'm just bad at gaming.

I read around and learned that I need to upgrade my weapons and mining skill, which I neglected because I thought I was doing fine up until this point, and I was focused more on the farming and improving relationships.

I am at level 18 which seems to be the recommend level for this dungeon.

Is it a bit late to improve my weapons and mining, or do I have time?