In a recent blog post, Daniel Lemire writes:
Hamming, the famous scientist, once suggested that researchers should focus on the most important problems in their field … What are the important problems in 2007? The ones we should all be working toward? Any ideas?
I happen to be working on the most important research problem. First, I should explain that Artificial Intelligence is the most important research field. Second, within the field of AI, the most important problem is programming computers to think with analogies. This is the problem that I am working on.
Filed under: Computer Science, Philosophy of Science | Tagged: AI, analogy, research
Could there be any such thing as “General Artificial Intelligence”, the problematic word being “general”?
Quoting a recent paper by Yoshua Bengio and Yann LeCun
“Scaling Learning Algorithms towards AI”
To appear in Large Scale Kernel Machines, MIT Press, 2007.
“Although the human brain is sometimes cited as an existence proof of a general-purpose learning algorithm, appearances can be deceiving: the so-called no-free-lunch theorems [Wolpert, 1996], as well as Vapnik’s necessary and sufficient conditions for consistency [Vapnik, 1998, see], clearly show that there is no such thing as a completely general learning algorithm. All practical learning algorithms are associated with some sort of explicit or implicit prior that favors some functions over others.”
Doesn’t this mean that any “intelligent investigation” of whatever problem or topic is NECESSARILY “tainted” by some unavoidable prejudices?
Bottom line: a “smarter than you” argument doesn’t cut it when it comes to conflicting interests.
What do you think?
Let X be any algorithm that learns, searches, optimizes, or makes guesses or predictions. Informally speaking, the No Free Lunch theorems say that X must necessarily have a bias (a “prior”), and this bias will be helpful in some universes, but will be detrimental in other universes. Averaged over all possible universes, no one bias is better than any other bias.
It’s notoriously difficult to express the No Free Lunch theorems in informal terms. What I said in the preceding paragraph is bound to be misleading, so look at the more formal discussions here:
http://en.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization
http://en.wikipedia.org/wiki/Inductive_bias
One interpretation of this is that an algorithm that is “intelligent” in one universe may be “stupid” in another universe; no algorithm can be “intelligent” in all possible universes. That’s fine; I’m interested in this particular universe, in which we find ourselves now; not in all possible universes.
Another way of looking at this is that “intelligence” consists of a match or correspondence between an algorithm’s bias and the universe in which the algorithm “lives”. We say that an algorithm is “intelligent” when its bias works well for this universe.
In everyday usage, “bias” is bad, but in technical usage, in machine learning, “bias” is inevitable. There is no such thing as a learning algorithm without bias. A bias is good when it is suited to the environment and bad otherwise.
When people talk about “General Artificial Intelligence”, they mean an algorithm that would work well for a wide range of interesting problems in this universe. It would not make sense to define “General Artificial Intelligence” to mean an algorithm that would work well in all possible universes.
There is an interesting and deeper issue about the nature of intelligence. In IQ testing, it is assumed that there is a single underlying factor, called the g factor, that explains the correlation in IQ test scores. But the argument for g is actually rather weak:
http://en.wikipedia.org/wiki/G_factor
http://cscs.umich.edu/~crshalizi/weblog/523.html
If there is no g, then it would seem that we have no basis for reducing intelligence to a single number, such as IQ. So we should not say that one person is “smarter” than another, because (perhaps) intelligence has many different aspects (factors), and who is smarter depends on how you weight those aspects. (I say “perhaps” because we really don’t know one way or the other. That is the point of Cosma’s post above.)
It seems entirely possible to me, even likely or necessary, that AIs (if and when we manage to create them) will be “smarter” than humans in some ways and “stupider” than humans in others. This is a good thing. Diversity brings robustness and strength. Assuming we can solve the cooperation problem.
“no algorithm can be ‘intelligent’ in all possible universes. That’s fine; I’m interested in this particular universe, in which we find ourselves now; not in all possible universes.”
This is where I depart from your optimistic line. Of course we don’t need to care about “all possible universes”, only this one, but relying on “our universe” uniqueness surreptitiously assume some form of Platonistic view. That is, you need to equate the “objects” in the universe with the concepts/objects names of your epistemology. Since any epistemology actually depends on a peculiar observer there *are* infinitely many epistemologies, which is just about as inconvenient as infinitely many universes.
What I want to suggest is that instead of taking for granted that “objects” or “concepts” exists just on a whiff of “intuitive evidence” and then haggling endlessly about the pros and cons of their “existence” we would probably make a better use of our (limited…) brain powers by looking *how* we come up with “objects” and *what use* we make of the “existence” (or not) of such objects.
Peter, thanks for the link to Gentner paper in our mail exchange, I will try to elaborate on my original reply in my blog.
Hello Peter,
I have always been puzzled by IQ testing. In fact, I find it somehow insulting to intelligence to think that it could be characterized by a series of tests.
As far as AI is concerned, it seems that the “intelligence” partis a bit of a moving target. Every time machines do out-perform humans on some “smart” task (chess obviously, but also some OCR or stabilizing various flying contraptions), human intelligence die-hards would be happy to explain that this task did not require any smarts after all, just number crunching.
Soon the only sure signs of intelligence will be messing up long divisions and reading distorted letters to post blog comments! (not here, thankfully)
Cyril > “human intelligence die-hards would be happy to explain that this task did not require any smarts after all, just number crunching.”
That’s a “reasonable” definition of Artificial Intelligence : anything that the computer is expected to do but isn’t doing yet. How would *you* define Artificial Intelligence? Or even just Intelligence, BTW, since you seem to have some uncertainties about how to characterize it.
I would myself define Artificial Intelligence as the ability for the computer to come up with the same smart solutions as we do, not just “some” right solution. Chocolate bars anyone?
Kevenbuangga:
Well, let’s say I don’t think I need a good definition of either intelligence or AI for the stuff that I do.
I would be happy with AI as “attempts to impart machines with capabilities usually associated with intelligence” or something like that. Which 1) is not so far from your definition, apart from the imitation part, and 2) as you rightly point out, moves the focus to defining “intelligence”.
The key in your definition is in the “smart…as we do”. Well, I’d be happy to call a machine smart if it does it better than we do.
I like chocolate bars, and the puzzle, very much.