Thoughts on the latest LJ wank:
So there is wank! Apparently LJ is listed on Spamhaus as they are having too much in the way of trouble with spammers recently, and this has potentially been impacting people's ability to receive notifications based on which email service they use. People are upset because they aren't getting their notifications, and because there's spam popping up here and there. Now there is an 'I'm a paid account and I'm mad about it!' and of course everyone is popping up to have a go at LJ without listening to any of that reasoned argument stuff so here are some thoughts from a dude who has adminned sites hit by small-scale persistent spammers, let alone the large ones.
* A Spamhaus listing is not, in and of itself, going to stop notifications from reaching you. It may make your email service automatically flag them as spam, and you may have to go get them recovered from your spambox, or they may be deleted out of hand by your email service. This is not a blanket immediate 'it is LJ's fault they're not reaching me' - I've been on Gmail, and they've been coming through just fine because a sufficient number of users have watched for them and indicated they're not spam. What I'm basically saying is that a Spamhaus listing is bad but is also far from inviolate; you are not receiving your notifications for reasons beyond Just The Fact A Listing Exists.
Edit: The thought also occurs to me that LJ have it pretty rough here. No matter how many steps they take to combat spam, they'll still have to email notifications to you... which will have to contain the content of the comment. Which will count as spam, which will get them a negative rep from Spamhaus. They'd have to edit the content of their emails in the case of suspected spam to get it to go through okay.
* Fighting back targeted spam on a website or service that prides itself as being open is very difficult. You can't just block IP addresses ahead of time because, I can guarantee, the spammers will have more IPs than you know how to catch. You can't do large blanket bans because if you do, you'll catch a lot of innocent people and get given hell for it. You can't filter content, because if you do, you'll catch a lot of innocent comments and get given hell for it.
* Pretty much the only foolproof way to deal with targeted spam before the fact is to block it in a recoverable manner, which is what their screen-links-from-non-friends was intended to do. You can't delete it out of hand because of potential false positives, so the best thing you can do is hide it, in much the same manner as a spam folder in an email account. Apparently, they fucked it up - I'm pretty sure it isn't catching links within image tags - but it was certainly an approach towards an acceptable first attempt. Yes, they should make it heuristic based upon the text or content of the comment, but anti-spam systems have been doing that for two decades and people still end up with lost emails and with spam in their main inbox.
* What are their options after the fact? Unfortunately, not many without relying on their userbase to report spam comments and/or lock down their journals. Ideally they'd close new registration and urge people to only accept comments from friends or registered accounts... but again, they'll catch hell if they do that. They're stuck, to an extent (I don't know about their internal politics, this is purely an external viewpoint) between a rock and a spammer-shaped hard place. If I were them I would start filtering comments based on the content of a href="" tags, comparing them to a blacklist of known spam targets... but even then, you're going to have false positives, which are pretty much an undeniable fact of life when it comes to spam, and you're going to require people to report comments for you rather than outright delete them. Since the latter option is the path of least resistance, this is usually what your users will do, which weakens the whole system to an extent unless you stay extraordinarily vigilant about what they're deleting... which gets you into a whole different big can of worms as regards Facebook-style 'nondeletion'.
* YES GUYS YOU ARE STILL GETTING EXPIRY EMAILS WELL DONE. I see this mentioned time and time again ("Well LJ are still trying to charge me huh I guess they know their priorities") and I kind of want to crush someone's head every time I see it. I do not claim to know LJ's internal architecture, I am not a staff member, but I would bet very good money that the expiry emails are going to be sent out specifically by the billing and account servers rather than the general comment database servers, which will have the important distinction of being much less overloaded and, now that I come to think about it, probably won't originate from the same IP address as the Spamhaus-registered one.
* Random thought: how many emails end up in your spam folder in gmail? Youtube comments that get flagged as such? Compare that to how many spam emails you get total on LJ, autoscreened or non-autoscreened. As much as spam has been on the increase on LJ, I'm still finding (at least personally speaking) that it's far from the end of the world here compared to those services we're used to seeing spam on.
* A Spamhaus listing is not, in and of itself, going to stop notifications from reaching you. It may make your email service automatically flag them as spam, and you may have to go get them recovered from your spambox, or they may be deleted out of hand by your email service. This is not a blanket immediate 'it is LJ's fault they're not reaching me' - I've been on Gmail, and they've been coming through just fine because a sufficient number of users have watched for them and indicated they're not spam. What I'm basically saying is that a Spamhaus listing is bad but is also far from inviolate; you are not receiving your notifications for reasons beyond Just The Fact A Listing Exists.
Edit: The thought also occurs to me that LJ have it pretty rough here. No matter how many steps they take to combat spam, they'll still have to email notifications to you... which will have to contain the content of the comment. Which will count as spam, which will get them a negative rep from Spamhaus. They'd have to edit the content of their emails in the case of suspected spam to get it to go through okay.
* Fighting back targeted spam on a website or service that prides itself as being open is very difficult. You can't just block IP addresses ahead of time because, I can guarantee, the spammers will have more IPs than you know how to catch. You can't do large blanket bans because if you do, you'll catch a lot of innocent people and get given hell for it. You can't filter content, because if you do, you'll catch a lot of innocent comments and get given hell for it.
* Pretty much the only foolproof way to deal with targeted spam before the fact is to block it in a recoverable manner, which is what their screen-links-from-non-friends was intended to do. You can't delete it out of hand because of potential false positives, so the best thing you can do is hide it, in much the same manner as a spam folder in an email account. Apparently, they fucked it up - I'm pretty sure it isn't catching links within image tags - but it was certainly an approach towards an acceptable first attempt. Yes, they should make it heuristic based upon the text or content of the comment, but anti-spam systems have been doing that for two decades and people still end up with lost emails and with spam in their main inbox.
* What are their options after the fact? Unfortunately, not many without relying on their userbase to report spam comments and/or lock down their journals. Ideally they'd close new registration and urge people to only accept comments from friends or registered accounts... but again, they'll catch hell if they do that. They're stuck, to an extent (I don't know about their internal politics, this is purely an external viewpoint) between a rock and a spammer-shaped hard place. If I were them I would start filtering comments based on the content of a href="" tags, comparing them to a blacklist of known spam targets... but even then, you're going to have false positives, which are pretty much an undeniable fact of life when it comes to spam, and you're going to require people to report comments for you rather than outright delete them. Since the latter option is the path of least resistance, this is usually what your users will do, which weakens the whole system to an extent unless you stay extraordinarily vigilant about what they're deleting... which gets you into a whole different big can of worms as regards Facebook-style 'nondeletion'.
* YES GUYS YOU ARE STILL GETTING EXPIRY EMAILS WELL DONE. I see this mentioned time and time again ("Well LJ are still trying to charge me huh I guess they know their priorities") and I kind of want to crush someone's head every time I see it. I do not claim to know LJ's internal architecture, I am not a staff member, but I would bet very good money that the expiry emails are going to be sent out specifically by the billing and account servers rather than the general comment database servers, which will have the important distinction of being much less overloaded and, now that I come to think about it, probably won't originate from the same IP address as the Spamhaus-registered one.
* Random thought: how many emails end up in your spam folder in gmail? Youtube comments that get flagged as such? Compare that to how many spam emails you get total on LJ, autoscreened or non-autoscreened. As much as spam has been on the increase on LJ, I'm still finding (at least personally speaking) that it's far from the end of the world here compared to those services we're used to seeing spam on.