Truth be told, I'm probably annoying. I expect other people to work as hard as I do, which they generally don't want to do. I'm also rather frank. I also don't take a lot of crap from my superiors. I expect them not to yell at me, and when they do, I fight back. I don't like bullies. I would add that junior people whom I try to protect view my assertiveness as my strong suit.
I also think that once one decides the leave, colleagues stop biting their tongues and feel free to vent. One thing you can say about me, if I'm unhappy with you, I'm not going to wait for you to leave to let you know. (Hopefully, though, if you're doing something well, I'll tell you that, too.)
I would add that having high standards doesn't always make friends, either. When I got here, senior faculty were telling junior people to write two papers per year. That was a recipe for being on the job market after tenure review. I told them they should write 4 to 5 and told the senior faculty they should take the lead. Raising the bar probably didn't endear me to anyone.
Another part of this is that the moment one takes another job, there is an inherent conflict of interest between a faculty member and his soon-to-be-former department--namely, the latter wants to get its collective hands on the financial and other resources of the former. (I would note that I did manage to escape relatively unscathed financially this time around, though I ain't out of here yet. (My current chair asked that I clarify this--and not very nicely (see reference to stinker e-mails below), I might add. My center director, Linda Collins, has been just great.) I really got screwed on one of my moves.
I also think when you leave a place, the folks left behind always feel a sense of rejection.
Finally, though, I would have to admit that while I didn't invent the inflammatory e-mail, I've perfected it. In my own defense, I would note that no one ever seems to remember the stinky e-mails people send me. I could quote some here, but that would only throw gas on the fire. (I think a fair question, posed by Mr. Cynic, would be, "Gee, when the heck did that slow you down in the past?")