Well, I can't specifically narrow it down to "self-esteem" but there are a number of people, going all the way back to high school, with whom I've either reduced or completely stopped connection, because they were varying levels of bad for my mental health and other aspects of life.

I suppose it says something that many of them were (ex) boyfriends, and the worst was Buttmunch, whose manipulative button-pushing nearly got me fired.

Proud of it? It's not in the top ten of words I'd choose, but yes. Proud that I finally stood up for myself. Proud that I chose to work my way out of a dysfunctional situation. Proud that I could say "You will no longer take advantage of me. You will respect me as a human being."

And I've never regretted any of them :-)