[tech, lj] Distributing LJ
Ever since the Strikethrough of '07 -- actually, ever since I realized that LJ was something of an attractive nuisance of basket in which to store eggs, way back when -- I've been thinking about how one would go about turning LJ, the software, from a client/server model to a peer-to-peer model. That is, how to make LJ distributed
( Read more... )
I suspect there are some mild scalability issues WRT aggregating the flists. Nothing really *hard*, but pulling the flist together from disparate sources is a much bigger operation than doing the DB merging on a single site. (Even worse than ordinary RSS aggregation, because the number of readers per feed is much lower.) This mainly implies that folks have to remember that the amount of hardware required for 10k LJdists is much, much more than for 10k ordinary LJ handles. Fortunately, it's distributed.
In general, I would broaden the concept slightly. It sounds like you're focused on manage-your-own-LJ, which I honestly think is a non-starter: there just aren't going to be enough people who are willing to go to the work to run their own site. But a medium-level distribution, that is focused on co-operative sites of *any* size, is more likely to be practical in the long run. You might wind up with a few dozen medium-sized providers serving anywhere from dozens to tens of thousands of users, as well as a few thousand hardy souls who really do wing it themselves. Get the architecture right, and those can all play together pretty seamlessly, and serve a variety of interests. (This is really where my musings have been focused: getting a network of sites, so there's less of a single point of failure.) And in the *long* run, when personal servers are more common, full peer-to-peer may become more practical.
The economics could be iffy, and that's nothing to sneeze at. LJ's finances have always struck me as a bit questionable to begin with, and I'm concerned that it would be fairly easy to accidentally kill it before the network was really up and running properly. (And of course, those economic issues provide disincentive for 6A to play nicely.)
I do think that, to *really* work well, you have to solve the friending thing. Requiring someone to have an LJ account in order to read flock'ed posts is a short-term hack. In the longer run, this all wants to be tied into a proper distributed-identity system, so that you *can* simply provide the credentials to do friending correctly. Fortunately, there are people dealing with such problems now, and the infrastructure is evolving. (Frankly, OpenID still strikes me as one of the more primitive solutions: despite being Microsoft-supported, I think that the WS-Trust model is better-designed.)
It's an interesting problem, and not *too* hard -- probably around the same level of complexity as the stuff I do at work. But the economics are going to be the killer here. Almost by definition, it's a hard model to make money on, so not many companies are likely to support it. And while the current system of separate islands is *conceptually* very shaky, it works well enough that folks don't have a lot of incentive to put the free work in on the distributed system. So my suspicion is that it's not going to really happen outside of speculation until we get a disaster, and one of the major social networks collapses, driving the point home...
Still, I do think it's going to happen eventually. And not just for LJ: all the same arguments apply to Second Life and suchlike systems. Of course, I've been arguing *that* for well over 10 years now, and so far it hasn't happened...
Reply
Leave a comment