Creating friendly AI is a popular topic among singularitarians. For example, writes:
As best as we can currently figure, the amount of effort needed to create a Friendly AI is small relative to the effort needed to create AI in the first place. But it’s a very important effort. It’s a critical link for the entire human species.
Elsewhere, he writes:
It seems like there are several fundamental, indispensible insights into the structure of this problem, which has been called “Friendly AI”. Oddly, it seems like they were uncovered all at once by a single guy, Eliezer Yudkowsky, who is possibly the only person on the planet who has thought about the problem long and hard enough to be qualified to talk about it. In most areas like this, there are multiple experts, each with something worthwhile to say, but when it comes to Friendly AI, people just seem to fall flat.
In fact, AI researchers have been thinking about this issue for some time. I list a few papers below. Interested readers can find more papers by searching for citations of these papers.
In Turney (1991), the solution that I prefer is programming AI with a sense of ethics, based on empathy, rather than hard-coded ethical rules. I still support this idea, but I also hope that there may be a scientific basis for ethics, which could contribute to the development of friendly AI. I have in mind research such as Robert Axelrod’s The Evolution of Cooperation and more recent work on understanding cooperation.
Barley, M., and Guesgen, H.W. (2002). Safe Learning Agents: Papers from the 2002 Spring Symposium. (Edited collection.) Technical Report SS-02-07. American Association for Artificial Intelligence, Menlo Park, California.
Gordon, D. (2000). Asimovian adaptive agents. Journal of Artificial Intelligence Research, 13, pages 95-153.
Turney, P.D. (1991), Controlling super-intelligent machines, Canadian Artificial Intelligence, 27.
Weld, D. and Etzioni, O. (1994). The first law of robotics (a call to arms). Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 94), pages 1042-1047.
Filed under: Computer Science, Evolution, Philosophy of Mind, Philosophy of Science, Political Science | Tagged: AI, cooperation, ethics, friendly AI
I think that the (contemporary) TV series Battlestar Galactica is an interesting reference here (stay with me for a minute). In this series, the Cylon (AIs) are at war with human beings and are winning. However, it is not clear whether they are immoral or evil. In fact, they are being depicted as being religious and as having rather deep philosophical issues. For them, at least initially, destroying the human kind is somehow the right thing to do.
When asked why, one Cylon responded, in effect, “you are not asking the right question, you should ask why you desserve to survive”. And indeed, there is strong evidence that the human beings are not especially kind or ethical compare to the Cylons. While this is not outlined in the TV series, we can safely assume that had the human beings been able to, they would have wiped out the Cylons without any remorse. My general feeling is that the Cylons appear as the bad guys mostly because they have such overwhelming military power. It is easier to appear evil when you are powerful.
Could it be that, at some point, it would be ethical for such AIs to wipe out humanity?
http://en.wikipedia.org/wiki/Battlestar_Galactica_%282004_TV_series%29
Daniel,
Good points about Battlestar. That is the problem with “ethics”. They are fads or arbitrary values.
I’ve written my thoughts ( nanofuture2030.com ) which are basically we need to limit AI minds by designing them to focus on one task and to ignore everything else. If you design them to be like us, then they will be just as untrustworthy as we are. And if they are also smarter, that is a bad thing. So, don’t create creatures smarter than us if you also designed them to be interested in human power competition and personal survival.
That is the problem with “ethics”. They are fads or arbitrary values.
Our ideas about what is ethical have evolved over time. The evolution is ongoing, and it may give the misleading impression that ethics are fads or arbitrary values. However, there is a general trend, over the long term, towards improved ethics. See the links above (e.g., Axelrod) and see also:
http://www.nonzero.org/