What Would We Do Without Peer Review?

The Scientist; (free registration required)

Two researchers conducting animal studies on immunosuppression lied about experimental methodologies and falsified data in 16 papers and several grants produced over the past 8 years, according to the Office of Research Integrity (ORI).
The scientists, Judith Thomas and Juan Contreras, formerly at the University of Alabama at Birmingham (UAB), falsely reported that they performed double kidney removals on several rhesus macaques in experiments designed to test the effectiveness of two immune suppressing drugs — Immunotoxin FN18-CRM9 and 15-deoxyspergualin (15-DSG) — in preventing rejection of the a single transplanted kidney.
The experimental protocol was to remove one intrinsic kidney, replacing it with a transplant and starting the monkeys on immunosuppresants, and then remove the other intrinsic kidney a month later, according to Richard Marchase, UAB’s vice president of research. “What occurred in a good number of these animals was that [Contreras and Thomas] never performed the second surgery,” Marchase told The Scientist. In a statement emailed to The Scientist Marchase called the misconduct “a very serious offense.”
Thomas’s and Contreras’s research was funded with more than $23 million in grants from the National Institutes of Health. UAB officials learned that Contreras and Thomas had left one native kidney intact in at least 32 animals — which allowed those animals to live and inflated the apparent effectiveness of the drugs — on January 27, 2006, when Thomas reported that she found an experimental monkey with one of its native kidneys intact and blamed Contreras for the mistake.

Had they only had the good sense to channel their energies into climate research, all of this unfortunate “research integrity” wiffle piffle might have been avoided…

32 Replies to “What Would We Do Without Peer Review?”

  1. That Richard Marchase character sounds like a skeptic. He’s getting in the way of funding. Quick, somebody, invoke consensus!

  2. and they would have received a lot more money . . . they barely sucked the public teat for enough to attend a five star Bali conference or two.

  3. So what’s your point? Doesn’t the fact that the falsity was exposed by the ORI’s investigation indicate that there’s a transparency and accountability that’s eventually brought to bear on those attempting to dupe the scientific community?

  4. I know it’s been said before, but “peer review” is not what many people suppose it is.
    It is not a guarantee of correctness. Reviewers do not have the resources to repeat the experiments, nor do they go through the researchers’ notes looking for signs of malfeasance.
    At best a reviewer can ensure the paper is relevant, novel, significant, and well-written, follows scientific standards, properly cites and uses previous work, and so on. If the authors are lying through their teeth – and clever about it – there is only a small chance the reviewer will catch the mendacity.
    Eventually, if the work is important enough, other researchers will attempt to replicate it. Or perhaps the researchers will submit a paper describing something the reviewer him or herself has attempted previously. Questions then start to arise.
    I am shocked that Contreras was “excluded from government funding and PHS advisory roles for three years.” How about a life-time exclusion? How about a little jail time for this white lab coat crime?

  5. Red Tory, the point I think is plain for all to see. In the case cited in Kate’s link, when scientific shenanigans were brought to light, the scientists in question were rightly ostracized. Note the many words used to describe their actions: mistake, lied, falsified, deception, wasted research, misconduct.
    However, in the case of AGW, when shenanigans are brought to light (and many example abound), they are explained away, covered up, or just plain allowed to stand (such as the recent information about incorrect temperature readings in Hawaii). When do any of the above-quoted words ever receive play when describing AGW research in the main stream media?

  6. some Korean dude got busted for this, when? 2 – 3 yrs back? maybe less than that.
    fakery skullduggery and fraud woven thru the ivory towers. kinda like what the tobacco co’s ‘research’ showed.
    I wanna conduct research on how often research is fudged and Im willing to pay top dollar to get the results I want.
    there was a time when even the slightest oversight in crediting sources would result in the most serious of sanctions. now? pfft. who wants to cite suspect results?
    p.s. how is the investigation into coroner smith’s massaged findings going? ya, him. mr silver tongue charisma who sent hundreds of probably innocent people behind bars. that’d be the one.
    these people need to be dragged into criminal court for wreckless endangerment for the consequences a successful scam could have caused to transplant patients.
    anyone wanna buy some stock in a cold fusion startup endevour?

  7. Hey, in the gun control biz stuff like this will get you a bonus from the CDC. Arthur Kellermann ate well for 15 years on studies that were much worse crap than this.
    Probably got more people killed than a badly researched transplant drug too.

  8. Red Tory;
    Colin from Mission B.C. is right on the mark. A lot of AGW studies that have been put out have not been properly peer reviewed. In many cases there has been a refusal to share data so that the conclusions could be checked.
    The most famous case was the hockey stick graph which formed the backbone of the IPCC report that claimed humans were responsible for GW. The author, Mann, resisted all efforts to get at his data, and when he was finally hounded into forking it over, it was found that he had massaged the numbers to make the warming period a thousand years ago just disappear, thereby making the modest warming of the 20th century stand out. True to what Colin said, the press just let it fall off the radar screen without pointing out what a fraud the IPCC was promulgating.

  9. If you think this is bad, just read a bit about “creation science” over at LGF or other blogs.

  10. Actually the “hockey stick” matter was much more onerous than just Mann’s fudging. The journal science first refused to publish the critique of mcIntyre and McItrick and then they commissioned a “peer revue of M&M’s work by associates of Mann.
    This is sort of an indirect “pal revue” rather than a peer revue.

  11. I had a follow up remark but it was moderated/deleted. Yes, it’s quite clear that Kate was suggesting the whole concept of scientific “peer review” is suspect. If that bogus notion panders to your ignorant, faith-based conceits, well, there’s not much further that can be said about the matter. At least not here, where the discussion is circumscribed in a manner that some might describe as being little more than a “circle jerk”…

  12. This kind of data falsification is not that unusual though in this case, not doing the second surgery is pretty bold, but I don’t know how they thought they could get away with this.
    I review many scientific papers and what is more common is exaggeration of the effects, poorly chosen comparisons to be studied, and then just not telling or publishing all the data that is generated. No cheating per se, just the entire data set is not available to the rest of us….
    Many scientific studies are corrupted or else politicized. It is to weep.
    And I am not even talking fake science like AGW.

  13. Valencia:
    I review many scientific papers…
    Examples please. In what capacity do you review? What journals are you talking about? It’s much more useful if you can name these shoddy journals so that they can be avoided.
    Thanks so much.

  14. Posted by: bob c at July 13, 2009 10:37 PM
    bob c, you must be a mind reader. The Mann hockey stick was the most egregious example I was thinking about when I made my original post.

  15. Posted by: Red Tory at July 13, 2009 11:33 PM
    Red Tory. Straw Man. Straw Man. Red Tory.
    Here, I’ll leave you two alone, as I’m sure you have much to catch up on.
    Oh, and Red Tory, I’ve had my fair share of posts “held back” for moderation that never reached the light of day. Sometimes certain words get flagged by the filter, and with the amount of traffic on this blog, it’s likely Kate or the few volunteer mods simply never get around to seeing it. Few, if any, of my posts would reasonably be construed as offensive.
    Occam’s Razor. Look into it.

  16. curious article in the lastest ‘foreign policy (july – august) about research that flops. that is, a legit study that fails to get the expected results; has to do with the learning process.

  17. No offense to any scientists, but how do you expect to be taken seriously when you believe we evolved from apes – I think it time we move forward from 1860.

  18. Peer Review is only a review for proper data handling and methodology, correct citations and a validation of an original work. Peer Review is a artifact of publishing research, not a “agreement of your peers in support of your conclusions”.
    If for example a Scientist never published his works, and hence did not participate in peer-review, that does not invalidate his work.
    Also research that is submitted for publishing and by definition peer reviewed and rejected hence not published is not neccessarily invalid.
    Nor is any published and peer reviewed research necessarily “more weighted” than non-published or self published works because there is no validation of the conclusions. So a peer reviewed paper is the exact same as a non-peer reviewed paper in as far as its conclusions being correct.
    Peer review has not been and never has been a meaningful guage of the validity of scientific conclusions. It has been hijacked to stifle debate and filter out conclusions that disagree with a particular dogma.
    To a real scientist it is meaningless.

  19. All you need to know about scientists comes from this survey – 55% self identify as Democrat vs 6% Republican. Remember Democrats will go to any lengths to achieve their agenda including blatant lying.

  20. “No offense to any scientists, but how do you expect to be taken seriously when you believe we evolved from apes – I think it time we move forward from 1860.”
    Incorrect, but our evolutionary tree highly suggests that we have the same ancestor. All current evidence points towards that, and new evidence is popping up all the time that seems to fit perfectly.
    In other words, the “theory” has predicted “facts” that come true in the future.

  21. “Posted by: Colin from Mission B.C. at July 13, 2009 11:54 PM”
    I think you’re being disingenuous or just obtuse if you don’t think obvious implication of Kate’s tone is to smear the validity of peer review. Red Tory is absolutely correct.

  22. I don’t know what is in all the preceding comments, but it is clear to me that these charges of fraud all arise from prejudice. One is a woman, and we all know how badly used are women in academia. The only way to get money is to lie–otherwise the old boys’ network just keeps ’em down. Just like that Kevane woman at Montana.edu As for the other researcher, he is obviously some species of Hispanic, and therefore American racism once again rears its ugly head. It is high time that you critics out there begin to study your inner selves and to face up to the hypocrisy and racism and sexism embedded deep down in your souls.

  23. Red Tory spluttered: “Yes, it’s quite clear that Kate was suggesting the whole concept of scientific “peer review” is suspect.”
    It -is- suspect, you idiot. Peer “review” and editorial bias gets all kinds of thing published where they don’t even get the freakin’ math right, much less the study design.
    My areas of study are physical therapy and the medical literature on gun control, both are more than replete with examples of peer review gone astray, in some cases gone rotten. Climate science appears to be much worse, given the US weather station situation.
    In fact, I have a citation for you: “Guns in the Medical Literature: A Failure of Peer Review” March 1994, Journal Of The Medical Association Of Georgia, 83(13).
    And here is the paper in its entirety:
    http://homepage.usask.ca/~sta575/cdn-firearms/Suter/med-lit.html
    Enjoy. Idiot.

  24. Red Tory’s complaint belies the leftard’s large ego. Everything is about “me”. No other explanation need be examined.
    I, too, have had many comments simply disappear into cyberspace. Apparently RT thinks someone is sitting at the switch 24/7 and is speed reading everything that passes by, catching those which don’t conform. Such is the mindset of those idiots. He probably believes in AGW, too. At least his statement posted in this thread is indicative of the same fuzzy logic.

  25. Many studies have shown that peer review does not improve the quality of scientific papers. Scientists themselves know it doesn’t work. Yet the public still regards it as a sign of quality… as if that meant something. It doesn’t.

    the ultimate lesson is that science isn’t special – at least not anymore. Maybe back when Einstein talked to Niels Bohr, and there were only a few dozen important workers in every field. But there are now three million researchers in America. It’s no longer a calling, it’s a career. Science is as corruptible a human activity as any other. Its practitioners aren’t saints, they’re human beings, and they do what human beings do – lie, cheat, steal from one another, sue, hide data, overstate their own importance, and denigrate opposing views unfairly. That’s human nature. It isn’t going to change.
    Michael Crichton

  26. Academics has degraded itself just like when the NHL expanded beyond the talent pool and the whole sport became watered down.
    There are too many “scientists” unworthy of the name. The universities are degree factories who no longer pretend to be elite in any way. In fact, they shun the idea of being elite.
    Oh, and a Red Tory is a liberal too stupid to know where they belong and who doesn’t have the mental capacity to see that “red tory” is an oxymoron.

  27. Valencia:
    I review many scientific papers…
    ….
    Thanks so much.

    Let it be noted that there was no answer to my request. I call bullsh*t.

  28. Jason:
    I know for a fact that peer review improves the quality of papers – indeed, it would be a rare exception where it didn’t. But most of the improvemens are in the writing and organization, not in the quality of the underlying science.

  29. rabbit,
    So peer review won’t catch shoddy work but will catch shoddy grammar? Great. A more literate fraud… lol.
    The problem with peer review is that anything worth publishing is new. You aren’t reviewing stuff you find in textbooks (to point out the obvious.) It means that the author is the first to do something and the reviewers are not usually in the exact field of study (which are normally extremely narrow.) Since peer review only amounts to a quick once-over by people whose expertise is merely “similar,” it will never find anything but the most glaring omissions or errors.
    Any scientist interested in making false claims or fraudulent work will find that it is easy to do for as long as it takes for competing teams to attempt a real replication of the work. That may take years if it happens at all.

  30. rabbit, peer review is -supposed to- catch errors in study design, method, conclusion not matching data, all that stuff. The reviewers can’t go and comb through the guy’s records, but they can use their supposedly large brains to see the obvious.
    And indeed, in reputable journals review works as it is supposed to, catching all manner of goofs, flubs and damn foolishness.
    Unfortunately, many formerly reputable journals have tossed aside any pretense of scientific rigor to become flat out propaganda organs for various Left wing social causes. Like gun control, like AGW, like socialism.
    What’s unusual here is that the scientists got called on their BS. Normally in medical journals even the most heinous junk science gets published free of any uproar at all.

Navigation