I just finished chapter 10 of Bostrom's Superintelligence, on Oracles, Genies, Sovereigns, and Tools. It has been one of the more disappointing chapters of the book. Since I've followed along these sorts of debates for a long time, I've never really been convinced that the "tool AI" approach was "bad" in the way Bostrom and Yudkowsky seem to claim ( Read more... )

I'm reading Nick Bostrom's "Superintelligence," which overall I've been appreciating (though much of the material so far is a condensation of common wisdom in the LW-sphere). I've noticed my first major issue with the book in chapter 4, in box 4 regarding the kinetics of an intelligence explosion. Here Bostrom sets up an equation for the rate of ( Read more... )