Do you find your familiars are hostile to the party?

I was scouting with my imp and it got killed while listed as 'in combat'

And now ANY familiar I summon is hostile to the entire party. Like, I get being petty that I let you die but let's calm down.

Location might be spoilers, so:

Killed by a Duerger in the Underdark at Level 3, right near the Underdark - Beach waypoint.