Tim Lee and Robin Hanson have had a good time going back and forth on this and I added my two cents as well, but I think I'll let the topic go...well, almost...
One last thought (building off my last two posts) on modeling the human brain with a computer: why would anyone want to?
Perhaps one could argue that an accurate brain model on computer would allow us to create and study various illnesses that occur in the brain, and understand them better. We could "age" our computer brain, and learn just what happens to a brain as it gets old, and hopefully develop methods to keep the biological counterparts sharp to the last.
Those are noble wishes, and I can't say I disagree with them. But this entire argument stemmed from the idea that we'd all "port" or "emulate" ourselves onto machines and the Singularity would come.
This, then, begs a serious ethical response. Should it be legal to clone a copy of yourself and keep it under medically induced coma, in case you are injured and need spare parts? I think the issues here are the same. A clone of myself would come in handy if I got shot in the chest, or fell off a building. A clone of myself would be nice if I were called to fight in war, and needed replacement limbs after IEDs blew the originals to shreds. But a clone of me, a fully functional human being...is a human being nonetheless, and therefore has rights.
What does this have to do with computer-brains? Well, the answer is simple. If I were able to port my brain, exactly as it is, into a computer, the first act upon completing this task would be CTRL + C. The second act would be CTRL + V, CTRL + V, CTRL + V, CTRL + V, CTRL + V, CTRL + V, CTRL + V...
If one of me is fun (I like myself quite a bit) then surely eight of me would be even better! But now I've got eight sentient Alex Wallers floating around in the the ethereal reaches of cyberspace. Which one is me? Which one has rights? Do any of them? Do all of them? How does my lovely wife know if she's interacting with the original me? Would it matter? Copying our brains to machines seems like an incredibly reckless and society-destroying plan.
I know this sounds ridiculous, but what is to stop me, upon downloading my conscience to cyber space...from completely filling every hard drive I can with copies of myself...like some viral Agent Smith, filling The Matrix with me, me, and more ME! I digress. The point is, while developing the technology to create a brain-simulator seems like a worthy and admirable goal...developing the technology to create a brain-emulator seems like a dangerous and perilous goal. That said, I have argued already that I see that day coming. Heaven help us if it comes before our species is ready.
This Is Your Brain On Sleep
21 minutes ago