"couldn't retrieve anum for entry" error when exporting journals/comments

[khyron]

Greetings fellow permanent account holders! If you have a moment, I wonder if you'd mind helping me troubleshoot something? I've been running into a really frustrating LJ bug, and I know that many permies also value the ability to back up their LJ content so I thought this might be a good place to ask

Comments

[ron_newman]

I'm having no problems using ljdump on davis_square (where I'm one of the maintainers)

[khyron]

That's interesting, I haven't tested any tools out on a community. I've just been trying to back up my journal. Do you mind trying to run ljdump on your own journal real quick just for kicks? You don't have to let it drag on or anything, if you get even one entry processed it will be much further than I can get (the missing anum error is triggered with my journal immediately).

If this issue is only effecting certain journals, I wonder what the discriminating factor is and why it's started recently?

[ron_newman]

It backs up my journal too, but I don't have much there.

[khyron]

Fascinating. One theory that some folks have advanced is that the error is in fact "real," which is to say perhaps there isn't a bug in ljprotocol, but rather that problematic entries are to blame (which have been somehow saved to the database without an anum value, by client mishaps or LJ screwups or something). The fact that you can back up one journal while others can't certainly seems to lean in that direction...something journal-specific.

Of course the problem is that LJ gives us no access to see whether our data is messed up, and there appears to be no way to contact a support person directly to have a dialogue about which entries in one's journal might or might not have weird data in them (gawd, at this point if I could narrow the problem down to just an entry or two I'd freakin' delete them!).

I wish there was something more I could do about this, I feel like my hands are tied all of a sudden.

[bandicoot]

Given the sudden rash of problems occurring about the same time leads me to point at LJ (probably XML-RPC) rather than the individual third-party apps. For example, my LJArchive errors don't mention "annum", but talk about lack of server response.

One thing I haven't seen mentioned is the question of what LJ did in response to the recent DDoS attacks - would any of those measures also have broken their API? Especially since I'm sure any measures they implemented were done in great haste.

Oh - one more thing. I have no problem importing entries, while one friend can't import those either. I just can't get comments.

I dunno...

[khyron]

For the journals this effects, it happens using absolutely any tool that talks to the XML-RPC interface (as opposed to the old "flat" method). It's definitely on LJ's side of things, there's no question about that. The way that XML-RPC works, these errors are being reported back to the backup/export tools from LJ, basically saying "hey I tried to handle the contents of this journal but I couldn't find what I was looking for and I'm punting now, sorry but I don't care to tell you what row I was on."

I guess the million dollar question is, what exactly changed to cause this to start happening?

[av8rmike]

It's just a shot in the dark, but I looked through the code base and saw this change near line 3952. If "anum" didn't exist in the log2 table for a record (entry), the script might throw an error like people are seeing.

[khyron]

That exact diff was one of the first things I found when I started investigating the error (back when I didn't even know what an "anum" was). It seems very likely to me that the source of the error is related to it, but what remains to be seen I guess is why the case of a "missing" anum isn't being handled better (after all, it's trivial to just go ahead and add one to a row that didn't get one for some reason, and then go on about your business). It seems like a better way to handle the case would be to just repair what's wrong with the database row, then proceed to return the requested results to the client. Maybe I'm missing something.