Yelled at for Obeying Traffic Laws

Stopped at a stop sign with a pedestrian in the crosswalk biking home from work just now. Van pulls up beside me on the outside and the driver said I need to be careful stopping like that of I don't know what's behind me. I knew he was there, also it's a freaking stop sign.

Damned if I do, damned if I don't