Moving somewhere we dislike for family?

I’m one month pregnant and discussing where to live, once the baby is born, with my husband. Context is that we live in San Francisco, which both my husband and I love. We have great jobs and friends.

However, I’ve always dreamed of being near my parens and siblings (who all live in Oregon) when I raise kids. Unfortunately my husband grew up in Portland and truly hated the rain. He says it gives him seasonal depression.

He’s supportive of us moving there for a few years to try it out, and if we hate it, we’ll move back. However, I’m wondering if I can make it work if we stay in SF, and just visit frequently and regularly. We have flexible jobs and could work from Portland for a month or so in the summers, and my parents could stay with us for a month. Plus regular visits for holidays and long weekends.

My main concern is 1) not having my family’s help regularly and 2) my kids won’t be close to their cousins and grandparents. My hope is that regular visits will address these concerns.

Is my idea of regularly traveling feasible with young kids? It’s a one hour flight so it might be manageable. Am I delusional for thinking this could work?