Prey
I read something this christmas break which peaked my interest, but i've decided to type it up here because it kind of relates to some things that have been annoying me recently.
"There's one problem with all psychological knowledge-- nobody can apply it to themselves. People can be incredibly astute about the shortcomings of their friends, spouses, children. But they have no insight into themselves at all. The same people who are coldly clear-eyed about the world around them have nothing but fantasies about themselves. Psychological knowledge doesn't work if you look in a mirror. This bizarre fact is, as far as I know, unexplained.
Personally, I have always thought there was a clue from computer programming, in a procedure called recursion. Recursion means making the program loop back on itself, to use its own information to do things over and over until it gets a result. You can use recursion for certain data-storing algorithms and things like that. But it's got to be done carefully, or you risk having the machine fall into what is called an infinite regress. It's the programming equivalent of those funhouse mirrors that reflect mirrors, and mirrors, ever smaller and smaller, stretching away to infinity. The program keeps going, repeating and repeating, but nothing happens. The machine hangs.
I always figured something similar must happen when people turn their psychological insight-apparatus on themselves. The brain hangs. The thought process goes and goes, but it doesn't get anywhere. It must be something like that, because we know that people can think about themselves in definitely. Some people think of little else. Yet people never seem to change as a result of their intensive introspection. They never understand themselves better. It's very rare to find genuine self-knowledge.
It's almost as if you need someone else to tell you who you are, or to hold up the mirror for you. Which, if you think about it, is very weird.
Or maybe it's not.
There's an odd question in artificial intelligence about whether a program can ever be aware of itself. Most programmers will say it was impossible. People have tried to do it, and failed.
But there's a more fundamental version of the question about whether any machine can understand its own workings. Some people say that's impossible too. The machine can't know itself for the same reason you can't bite your own teeth. And it certainly seems to be impossible: the human brain is the most complicated structure in the known universe, but brains still know very little about themselves.
For the last thirty years, such questions have been fun to kick around with a beer on Friday afternoons after work. They were never taken seriously. But lately these philosophical questions have taken on new importance because there has been rapid progress in reproducing certain brain functions. Not the entire brain, just certain brain functions. For example, before I was fired, my development team was using a multi-agent processing to enable computers to learn, to recognize patterns in data, to understand natural languages, to prioritize and and switch tasks. What was important about the programs was that the machines literally learned. They got better at their jobs with experience, which is more than some human beings can claim." - Michael Crichton, Prey