Published November 28, 2013
Continuing the discussion of citations, I thought I would report on the results of the first paper that I had checked for the accuracy of the citations. (I won’t, of course, indicate the paper or the authors of the paper.) If you want to preserve your time so that you can stuff your turkey or whatever, you can stop reading if you just want the take-home message, which is that the first paper checked for the accuracy of citations was pretty darn accurate, with just a few questionable citations, but nothing really facepalmy or anything.
So. I assigned the work to a trusted research assistant (RA), who I asked to ensure that each citation in the text properly cited the source. That is, I asked her to check to see if the source cited actually said what the authors of the paper said that it said. She was able to check nearly all of them, with some exceptions. In some cases, papers cited in the text didn’t appear in the reference section, in a few cases making it impossible to determine what paper was being cited. In other cases, papers cited were unavailable because they were unpublished or in journals to which the University of Pennsylvania does not subscribe.
She reported that it took her roughly 11 hours to complete the task.
I should note that Elsevier, as part of its copy-editing process, already checks to be sure every work in the references is cited in the text and every parenthetical cite has a corresponding citation in the reference section. So, in the future, I think I’ll instruct the person doing the work that she need not do this. (Having said that, as an author, I think that I nearly always have a research assistant do this check before submission. It’s possible that some citations are missed, but generally, it seems to me that authors really ought to be taking care of this. There are always last minute modifications that lead to errors, but I think it’s more or less reasonable to ask authors to be careful about making sure that the parentheticals and references match. It’s a task that can be assigned even to undergraduates, a little dig I’m putting in here just to check to see if my undergraduate RA Molly is reading all my blog entries.)
The result of her check was that nearly every citation was, in fact, accurate. There were really only two cases in which my RA identified potential problems. One was a case in which she found the wording of a sentence a little misleading in terms of the words the authors chose to relay the idea in the source material. A second was a case in which a finding was reported without adding a caveat about the scope of the finding. There were a couple of other cases in which it seemed to me that the distance between the citation and the source were debatable and probably defensible. I passed the result of the check onto the authors, who I have asked to modify the document appropriately. I also asked them to send me a cover letter indicating the changes.
Overall, I think I would say that, based on N = 1, the results were better than I would have predicted. (This has nothing at all to do with the specific authors of the paper in question. My expectations are based on my overall sense of citation accuracy in journals.) That is, I would have guessed that there would have been a larger number of discrepancies, and that the citations would have been further off than it seems like they were. Still, I think it’s worth continuing the pilot program to see if this is a typical case. If it is, then I think that my view will be that it might not be worth continuing, or worth continuing only on something of a spot-check or random basis. If there are so few errors, working hard to find them might not be worth it. (Again, I’m open to discussion. Feel free to comment or send me a note directly.)
The other issue, which this pilot didn’t address, is page citations. On this issue there are some remarks in the comments section of the prior post and I have also received some email on the topic. My sense is that people are in favor of adding pincites, or, at minimum, asking authors to provide page citations when they refer to entire books or other very lengthy works. I think maybe I’ll defer this issue until the meeting of the editorial staff in Brazil for the Human Behavior and Evolution Society meeting.
To my American readers, happy Thanksgiving. To my non-American readers, happy Thursday. See you all on the other side of the holiday.
