Comments | lj_dev: Multiple LJ backup software has a syncitems bug.

njyoder in lj_dev

Multiple LJ backup software has a syncitems bug.

Apr 03, 2008 06:06

Many LJ backup software, including the official jbackup.pl client, have a bug when doing syncitems where the first batch of items is all comments (i.e. no posts). LiveJournal Backup / Search Tool and ljArchive also appear to the susceptible to this bug ( Read more... )

client: sync, client: export, client, code: perl

Comments 5

ghewgill April 6 2008, 09:53:25 UTC

I had a look at my ljdump backup program and it appears as though it should handle this case properly. In particular, the lastsync state variable is updated for each item in the list that syncitems returns, instead of just for journal entries.

I wonder whether you could give ljdump a try on your journal and see whether it works for you.

njyoder April 7 2008, 18:29:43 UTC

I'll check it out. How well developed is its library for general use (in other programs)? I was thinking I'd probably use ljsm because it can backup any journal, but I might be inclined to use another one in Python instead of Perl if it wouldn't be too time consuming for me to add that.

Oddly enough, I couldn't log in with ljsm and it kept returning some relatively useless error message. I found out that the cause was "BROKEN_CLIENT_INTERFACE" being set to 1, so I disabled it (set it to 0). I just found that really funny, even though I know it's actually set to compensate for sessiongenerate not returning cookies. Did LJ's sessiongenerate not work properly at some point?

ghewgill April 8 2008, 08:11:18 UTC

ljdump is pretty specifically tailored for its single purpose, that is, it's not designed as a library. But the code isn't terribly long or complex so it could certainly be used in another context.

njyoder April 14 2008, 10:42:54 UTC

That's good to hear. I'm sorry to bother you again, but after I tried it, for some reason it seems quite slow compared to other backup utilities.

Are you downloading posts one by one? It seems like that and if so, it's inefficient when you can posts month by month, just in case you didn't already know (I'm not meaning to patronize if you already did). I don't know if this is normal, but it did go over the posts/syncitems more than once even though it's a first run, does it create and update the same post more than once under any circumstances?

It takes over 30 minutes, probably closer to 60, to download the about 1,800 posts and 2,000 comments in my journal. It takes only about 10-20 with other utilities.

It did download everything correctly, it was just slow at doing it.

Thread 5