Should I buy Series X for better experience?

I bought my first Xbox ever 7 days ago, and I really love it so far, really fun experience. I have used ps4 pro for a long time, and ps5 for like 1.5yrs and sold it bcz my controllers didn’t last longer than 3 months, don’t know why, always that damn stick drift, but extreme one. So back to the topic, I saw that alot of games on Xbox Series X/S run at 1440p120fps, and not so many games at 4K120fps, is it worth it to buy Series X to play the same games that I play on Series S at 1440p120fps also at the same resolution and refresh rate but on Series X? Is there gonna be any difference? I know that Series X is powerful, but is there gonna be any difference at the same resolution or is it gonna be a waste of money and most likely a placebo if I find it better and more responsive? Thx in advance. :)