I believe that math is very important: My first paper was mathematical (How many ways can an N-dimensional hypercube be unfolded into (N-1)-dimensional space?) and my most recent paper was mathematical (How can a very large tensor be decomposed with limited RAM?). But medan agan: everything in moderation; nothing in excess. In machine learning and artificial intelligence, I sometimes worry that there is an excess of math. Others have complained about “the use of unnecessary and obfuscatory mathematics to improve the author’s chance of publication”, and I agree that this is a problem, but it is not the problem that concerns me here. My worry is that our research is biased towards heavily mathematical algorithms, but perhaps “intelligence” (whatever that might mean) is only moderately mathematical.
I don’t have a solid argument here; only a vague intuition. What is the nature of the algorithmic processes that make humans smart? Are the algorithms more mathematical, like support vector machines, or are they less mathematical, like genetic programming? I am tempted by the beauty of the math behind support vector machines, but my intuition is that humans use algorithms that are less mathematical. For example, Bayesian inference is an attractive mathematical approach to reasoning, but Kahneman and Tversky and others have shown that human reasoning is not quite so rational. This does not imply that machine learning and AI researchers should avoid Bayesian approaches, but it does suggest that there are more important things for intelligent behaviour than getting accurate probability estimates.
What are those things? I think Dedre Gentner is pointing us in the right direction. However, my aim here is only to suggest that we should be moderate in our use of math; my aim is not to endorse any particular alternative approach. As researchers, we are exploring a space of algorithms, searching for “intelligent” algorithms, and it seems to me that our exploration is heavily biased towards more mathematical algorithms, but there is no evidence to support such a strong bias.
It looks like physics envy to me. I admit I’m a physics groupie. The first step to rehabilitation is admitting that you have a problem.
Filed under: Computer Science, Philosophy of Mind, Philosophy of Science | Tagged: AI, complexity, math, moderation
My reading of the heuristics and biases school is that, if we want to descriptively model human behavior, we have to get away from the normative assumptions of rationality.
That doesn’t mean abandoning math, but it does pose some challenges. For example, if, as prospect theory holds, people make decisions relative to a frame of reference rather than based by maximizing expected utility, then we should reconsider any modeling that assumes the latter. But that means we have to model what, at least to my math-trained eye, is human inconsistency.
Should we be making our rational machines ape human irrationality? If we want accurate simulation, then I think we must. But if our aim is to create better tools for problem-solving, perhaps we’d do well not to corrupt our robots’ brains. After all, they don’t need to be optimized for the environment in which homo sapiens evolved.
But if our aim is to create better tools for problem-solving, perhaps we’d do well not to corrupt our robots’ brains. After all, they don’t need to be optimized for the environment in which homo sapiens evolved.
I agree completely. I am an AI researcher, not a cognitive scientist. I aim to discover intelligent algorithms, and it doesn’t matter to me whether they descriptively model human behaviour. However, I look towards cognitive psychology for any hints that I can borrow and use in my own work. One hint I see is that humans seem to get by reasonably well without following our normative models of rationality. This suggests to me that our normative models of rationality are not as important to the goals of AI research as we think they are. Yes, let’s make our robots more rational than us, if we can. But it seems that normative rationality is a nice optional extra feature, rather than an essential feature. So what are the essential features that we’re missing? And should we expect to find them in the more heavily mathematical subspaces of the space of algorithms?
Yes, let’s make our robots more rational than us, if we can.
WHY? Because “rationality is good”? Isn’t there some Petitio Principii lurking here? Could the “right question” be, instead, when do intuition, guesses, randomness, etc., beat rationality (for all practical purposes) and why?
Smarter Isn’t (necessarily) Better: “If you’re using your intelligence to outsmart your group, then there’s an arms race,” Dr. Kawecki said. “So there’s no absolute optimal level. You just have to be smarter than the others.”
Could the “right question” be, instead, when do intuition, guesses, randomness, etc., beat rationality (for all practical purposes) and why?
I think I agree with what you’re saying, although I would phrase it slightly differently: Evolution is the final judge. It is not rational to behave in a way that leads to extinction. Normative rationality must take into account the computational and informational resource bounds of an agent.
Smarter Isn’t (necessarily) Better: “If you’re using your intelligence to outsmart your group, then there’s an arms race,” Dr. Kawecki said. “So there’s no absolute optimal level. You just have to be smarter than the others.”
This article simply points out that learning can be costly. This has been known since 1896 (if not earlier), when Baldwin published his theory on the interaction between learning and evolution, now known as the Baldwin effect.
Our saving grace as human beings is surely our ability to manage decently with bounded computational resources. Sure, our satisficing introduces systematic biases, but all in all it seems we’ve done okay for most of our existence as a species (though I’m not sure what benchmark to use for comparison).
I suspect that what our algorithms are missing reflects the limits of our ability to articulate and even understand our own knowledge. Consider the gap between an expert and an expert system. The latter avoids susceptibility to cognitive bias, but almost surely fails to capture all of the relevant knowledge of the expert it is intended to model.
“Applied Mathematics” is much less useful than people may think. This is a case of misperception on our part.
While elementary mathematics is very useful and omnipresent in all long lasting scientific theories, our attempts to “mathematicize” problems rarely work.
In the few instances where sophisticated mathematics proves useful, it is very impressive and leaves a long lasting impression on us. So we tend to try to reproduce it elsewhere, and we fail.
It is not so much that we should stop using Mathematics, but we should be critical of its powers.
So, PageRank uses Markov processes to compute the authority of a Web page. Very impressive, is it? But is it any better than computing the number of inbound links? Maybe. Maybe not. Using fancy Mathematics should not grant you immunity.
Disclaimer: I have a B.Sc., a M.Sc. and a Ph.D. in Mathematics.
Related: Change the algorithm, not the dataset.