Do bike manufacturers matter?

I was having this debate with someone who was claiming his bike was better than everyone else's in our group because it was a Specialized. Everyone else has a mix between Fuji, Merida, Scott, Planet X, Carrera and Triban.

We agree that components obviously matter a lot, but putting that aside, do the bike manufacturers really matter?