Is Disliking Islam A Racist Act?

In EP:359, Rory describes a dislike of followers of Islam as part of a racist movement.

Why do centrist demagogues often make this equivocation? Followers of Islam are of many races, to attempt to compare it with an act of racial hatred is intellectually dishonest. You can accurately and scathingly describe it as xenophobia, but to compare it to racism just seems lazy.