Scientific Productivity, Age, and Field
I once saw a graph that plotted scientific productivity as a function of the scientist’s age, with different curves for different scientific fields. I remember that the curve for mathematics peaked between the ages of 20 and 30, but the curve for chemistry peaked somewhere around 50. There was no curve for AI researchers, and I occasionally wonder where such a curve might peak. This ties into the neats versus scruffies debate. The neats probably peak early, whereas the scruffies peak late. A good strategy would be to be neat early in life, and switch to scruffy later. Today I went searching for a copy of that graph. I didn’t find it (let me know if you can find it), but I found a lot of interesting studies on the topic.
Dean Simonton has been doing research in this area for a long time, but I am troubled by his emphasis on genius. It seems to me that the concept of genius is closely connected to the heroic theory of scientific development. Instead of trying to measure genius, I would rather focus on productivity, measured by number of publications. There is too much room for subjective bias in a study of genius.
Satoshi Kanazawa has made a bit of a splash with his theory that young male scientists, like young male criminals, make an extra effort in their chosen fields, in order to attract mates. Once they get married, their productivity immediately drops. I am not convinced by his study.
Svein Kyvik finds that productivity in the social sciences is not particularly sensitive to age, but productivity declines with age in the natural sciences. He claims that older scientists cannot keep up with the fast pace of progress in the natural sciences, but progress is slower in the social sciences. I am skeptical.
Paul Allison and John Stewart suggest that there is a kind of feedback effect in a scientist’s career, where success leads to greater productivity and thus greater success, but lack of success leads to lower productivity and thus less success. Therefore there is more variability in the productivity of older scientists than in the productivity of younger scientists. This complicates the analysis of how age affects productivity. Looking at the average productivity can be misleading; we need to look at the variance of the productivity distribution.
Hall, Mairesse, and Turner examine the problem of separating the effect of a scientist’s age from the effect of the time period in which a scientist lives. We would like to be able to compare scientists who have the same age in a certain time period, but were born in different time periods, but this is difficult to arrange. If competitiveness increases steadily with time, then older scientists may appear less productive, but it will be due to the time period in which they worked, rather than biological or psychological effects of aging. Most studies do not attempt to separate these two factors.
My conclusion: more research is needed.
Filed under: Computer Science, Philosophy of Science | Tagged: research, AI, productivity, age
I don’t know about AI research, but I can tell that software development does peak. At 60, I really don’t feel like I could be doing the kind of hairy stuff I did 20 or even 10 years ago. I find this is due to a drop in concentration abilities and also some (relative) lack of motivation, definitely a plainly age-related decay, I think.
But it is not certain that the number of publications is the best metric for measuring scientific productivity, see Productivity at elite universities. Even using this criterion, there are large differences in the various fields of research (from Boosting Productivity) and gender may have its role too: Gopnik’s Learning Curve.
As for Satoshi Kanazawa, maybe his theory has some merit along the lines of evolutionary psychologist Geoffrey Miller, but he is not the best example of a credible scientist, according to Cosma Shalizi, who cannot be denied to be himself a credible scientist.
I do not entirely buy the Allison and Stewart hypothesis even though it is the most interesting of the lot. Of course, if you do well early on, you will have more opportunities and resources to grow. But not everyone grows. It seems that we improve if we think that our abilities are not innate, see this post of mine: Thinking intelligence is innate makes you stupid.
That is probably how I would cluster the researchers. Group all researchers who think that their skills are innate. Then you will probably observe that they do not grow much over time. Those who believe that their skills can grow, will tend to grow more often.
Of course, you have to be motivated. Once you have a steady job, the benefits you derive from a great paper are not as significant as they were when you started out. I would guess that the best motivation is your love for science.
One factor that may explain why many scientists reach a plateau is that success is a bad thing in science. It tends to bring money, responsibilities, people to supervise, more meetings, and so on. As graduate students, we used to refer to this effect as “the accident”. Whenever a professor was highly successful, you would find he was “out” (confused about new ideas, mostly reacting to new results, never coming up with anything new, and so on).
[quote]
I would rather focus on productivity, measured by number of publications.
[end of quote]
I’m not certain that the number of publications is the best metric for measuring scientific productivity. As Kevembuangga pointed out, “it’s easy to optimize for number of publications”. Number of publications is a bad measure for individual productivity evaluation.
However, the fact that some people may artificially boost their number of publications is probably irrelevant when trying to get the big picture of productivity by age. I’m tempted to say that there are as many young researchers who do it as older researchers; it simply lifts the curve up for all age groups.
Consider this analogy to information retrieval (IR): counting “words-matching-a-query” is not sufficient to find “documents-relevant-to-a-query” and, the same way, publication count is not sufficient to approximate individual scientific productivity. In both cases, raw count is easy to manipulate and often varies for reasons unrelated to document relevance / scientific productivity.
In IR, we must look at links to documents (in-links) and, the same way, we should look at number of citations (impact?) per publication.
In IR, we must also look at in-linked document credibility (pagerank) and, the same way, we should look at citation credibility (recursive measure of impact).
Regarding Least Publishable Units, I think it is possible that LPUs are a good thing, in terms of memetic evolution. When memes are bound together in one publication, they may live or die as a unit. One bad meme might spoil the bunch. Breaking the memes into LPUs may accelerate memetic evolution.