Our main mail/web server, lodestone, got rootkitted last night. Unless I've missed a security bulletin recently - which I could well have, given how things have been - it was probably a local exploit via a stolen user account. However, that's not guaranteed. (I have some suspicions about where to look; I saw a login yesterday that somewhat surprised me, but not enough for me to be alarmed. That was probably a mistake.) The rootkitting happened last night, mostly around midnight PDT.
The box has been fundamentally compromised; in addition to widespread webspace hacking, /bin and files in it have been changed. The box will need to be flattened and rebuilt. Unfortunately, this work cannot really begin until
annathepiper gets back. We are also in the middle of trying to move, making things much more difficult.
Assume we will be down for AT LEAST four days, probably longer. During this time, our net connectivity will probably be intermittant. I may be able to get a skeleton box online to catch incoming mail before then, but no promises.
When lodestone comes back up, we will be requiring new passwords from ALL users. ALL executable files in userspace should be deleted, and either recompiled or reinstalled. These were not expert crackers; the work was sloppy; they left fingerprints everywhere, and some of their attempts to modify binaries resulted in nonexecutable files, rather than trojaned files, which is good. But that doesn't mean they were totally inept. And they were certainly trying to do as much damage and backdooring as possible, so we need to react as if they were better than they are.
If there are any former Murknet admins who think they can help/do something useful/provide advice, let me know.