In January 2017, there was a panel discussion between 5 economists (including 4 Nobel Prize recipients) talking about economic journals, citations and the incentives of academic economics. I highly recommend watching this video:
link. The conversation included: James Heckman, George Akerlof, Angus Deaton, Drew Fudenberg and Lars Peter Hansen. This post is a collection quotes from the video.
[Copyright American Economics Association; reproduced with permission of the AEA]
James Heckman (2000 Nobel Prize recipient)
"When I go to dinner with a group of young pre-tenure PhD economists or talk to graduate students who are thinking about doing research, I find increasingly, and even with post-tenure economists, there seems to be an obsession on ranking themselves by the number of top 5 journals articles published."
"If people are really aiming to go just into the top 5 journals and their goal is to publish papers as opposed to contribute to basic knowledge, I think that's a limited vision of economics and what economic research should be."
George Akerlof (2001 Nobel Prize recipient)
"Economists know there's a basic problem with any fixed reward system: people work to the test."
"People divide their effort between tasks according to the respective rewards of those tasks. If the rewards are to publish in the top 5, people will put their effort into whatever it takes to publish there. But that may not be the effort that makes them the best economist. Furthermore, in real life the game does not stop there. There's a doubly bad equilibrium because those who are successful at the task are those who then set the reward system in the future."
"If there is low power of tests for what is good or bad science, there are equilibrium traps. In these traps, new and better paradigms will fail to be adopted. On the contrary, such bad equilibrium traps disappear if there are high power of tests."
"There's almost a total disconnect between sociology and economics. But are they always wrong and we always right? Isn't it some combination of the two more likely to be correct and also relevant for the types of situations we as economists tend to look at?"
"There's a whole type of economics that economists simply don't do that doesn't occur in our journals. For examples economists did not provide detailed reportage of what was happening in the financial system [in 2005]."
"The authors should own the paper. When we submit the paper it should be our paper. We shouldn't have to rewrite it for whatever the referee thinks the paper should be."
"What I'm worried most of all, is what we don't see. I'm worried about the analysis that is never seen, that never becomes a paper and it doesn't become a paper because it can't be a paper. It can't become a paper because that's not what a paper in economics is all about."
"You usually don't know such vacuums exist. But there's actually a signal that such a black hole does exist because there wasn't anything in the economics profession that was detailing what was going to happen when we got into the great recession."
"I believe we should allow stuff that is much fuzzier so people can say what's in their hearts and that way may fill in some of these vacuums that are now black holes."
Angus Deaton (2015 Nobel Prize recipient)
"The top 5 journals are not really much about communicating new findings, that's a shame in some ways."
"There's been a large increase in evaluation by metrics, whether within departments or across departments by government agencies. In those, international journals are given very high weight."
"When somebody tells me, 'We'll that's your favorite candidate for bringing here, but we don't like their work at all' and you can try to explain but often you get nowhere at all because there isn't a common ground for communicating that information... That's why these algorithms, and this very silly algorithm (top 5) has become very popular."
"I think more sophisticated algorithms, if departments used them, deans made departments use them, would actually weaken the position of top journals. So what we have now, counting top journal citations is really stupid I think."
"I think that thinking about algorithms would be really helpful, even if they are very much a second best solution."
Drew Fudenberg
"We're not alone in this, we didn't just for some bizarre reason switch to a bad equilibrium. For some reason parallel things are happening in other fields."
"Since I joined this field, people have complained that papers are too long and are getting longer."
"Another trend in biology, which I think is going on in economics, is more co-authors per paper."
"I'd like to see journals have more papers in their lead journal but also have more subsidiary journals. I believe that's one way forward."
Lars Peter Hansen (2013 Nobel Prize recipient)
"This reliance on referees leads to a much more conservative strategy. I think it works against novel papers that cross subfield boundaries and that makes it all the more challenging. Basically it makes the simplest path to publication in the top 5 journals to be high quality follow up papers."
"I think it's hard to use citation counts in sensible ways that are applicable for young scholars."
"Sometimes there are papers that basically close a field they are so exceptional. They basically clear up everything so they don't have a whole lot of follow up work and therefore lose the citations."
"I don't see simple algorithmic alternatives as a way to evaluate young people, but I do think figuring out ways to encourage synergistic activities across subfields and to figure out ways that recognize creativity and possibilities of it at early stages of a career is absolutely vital and important."
