Shyam's weekly musing

[bjornick]

Finally launched! W007. This still doesn't make my life any easier. I am sure to find an infinite number of bugs in my system. To be honest, I am still not completely happy with the system when it launched, but in the end most users won't know the difference anyhow. Crap, I'm sounding like a crappy software apologist now don't I? Anyway at

Comments

[friedfly]

a) So, am I allowed to ask if this is you?: http://arstechnica.com/news.ars/post/20060326-6460.html

b) Batch jobs are always going to be more reliable. And in huge massive volume, what could matter more?

[bjornick]

That wasn't me. I (worked/am still working) the crawler. I don't expect much improvement right now, but we are set up to really fix some of our crawling problems pretty soon.

[bjornick]

If batch jobs really are more reliable, I would hate to see real-time attempts at this stuff.