As a queer white feminist still deep in the throes of self-education, I already find the ass-hattery of some people incredible when they're called out on their racism. I think a huge part of the problem is that explicitly we're brought up to believe 'racism is wrong' and it doesn't go any further than that. For me, my upbringing with white liberal parents involved being told that treating someone differently because of the colour of their skin is wrong, and went no further. There was never discussion about the systemic oppression of PoC, or the intrinsic attitudes with which we are equipped as white people. My peach coloured crayon was called 'flesh'; the plasters that matched my skin tone were called 'flesh coloured', and this was never noted or discussed. Whiteness was default and my parents, I suppose, fell into the common liberal fallacy of 'the world will be fixed when we all become colour blind and gender blind'*. I've had a lot of self education to do, and am still doing it, and probably will be for a long time.
But I think a lot of problems occur when people think, 'I'm an intelligent person and I know stuff about race, I can skip the definition of terms and jump into the discussion', and don't find out the givens that other people who are in the discussion are taking as such. They hear the term 'white privilege' and jump in with 'I'm not treated in a special way because I'm white!' More often, I see uses of the word 'racist' or 'racism' used with the assumption behind them that those reading them will subscribe to, or at least understand, the activist definition of it - that is, that racism is prejudice + power, that racism isn't limited to individual acts, but refers to ways in which PoC are abused by the system. This means that when an activist says, 'I find what you said/did racist', they may mean, 'I find your actions/words to be reflecting the ways in which PoC are oppressed by the system/building on power dynamics that oppress PoC, in ways which you didn't necessarily mean', while the culprit hears 'You are a bad person who deliberately and maliciously goes around hurting people of colour'. Disconnect. Breakdown in communication.
So, I think this is why the education part of becoming an ally/working in solidarity is so vital. It's so easy to respond with hostility to what you think somebody said, when there was no sense of that in their original comment, asshattery reigns supreme and everybody has a bad day. It's not necessary, and getting yourself on the same page before blundering in is so important.
*
This is a pretty good explanation for why colour-blindness is not the answer to ending racism.