en.wikipedia.org A reviewer at the American National Institutes of Health evaluates a grant proposal.

Ethan Barton on April 20, 2016at The Daily Caller News Foundation

National Institutes of Health (NIH) experts can’t predict which studies they fund will produce the best results despite a rigorous peer review process, and switching to a lottery system could be equally effective and cheaper, according to a study posted on the agency’s website.

“Peer review is widely used to assess grant applications so that the highest-ranked applications can be funded,” the study, originally published in February by eLife Sciences, said. “A number of studies have questioned the ability of peer review panels to predict the productivity of applications.”

The report, which extensively details flaws with NIH’s grant awarding process, was only recently brought to The Daily Caller News Foundation’s attention.

“This underscores the limitations of peer review as a means of assessing grant applications,” though the process effectively prevents money from going to applications with “serious flaws,” the study continued.

Peer reviews are assessments of completed studies by experts doing work in the same field. The study on the NIH site concerned only the use of the peer review process to screen applicants for study funding.

The research cited other reports that collectively suggest “expert reviewers cannot accurately predict” a grant recipient’s productivity through the peer review process.

“This may contribute to a pervasive sense of arbitrariness with regard to funding decisions and dissatisfaction with the peer review system,” the study said.

“Furthermore, the best science is not necessarily receiving support under the present system,” the study stated. “The manner in which scarce research funds are allocated deserves greater attention.”

Entering grant applications into a lottery after peer reviewers pick the top candidates could be an equally effective method of awarding funds and would save NIH money, the study said, noting that the New Zealand Health Research Council had already adopted such a system.

The report based grant productivity on several factors, including the number of publications that resulted from the federally-funded research.

A NIH spokeswoman provided a statement from NIH directors Michael Lauer and Richard Nakamura:

An article based on a large, long-term sample of grants that was published in Science last year did find a correlation between peer review scores and grant output,” they said. “We have commented in detail on how findings similar to those in the eLife paper should be interpreted. For example, one precaution is that raw numbers of citations is not the goal of research.

“In addition, the initial round of peer review is part of a larger process in how NIH makes funding decisions to support research,” they continued. “NIH is using pilot programs and rigorous evaluation tools that take into account the multi-faceted, complex nature of assessing scientific impact and address the entire breadth of NIH activities, including peer review, that support scientific research and its impact on health.”

NIH’s peer review and grant awarding process also makes it difficult for younger scientists to secure funding, causing them to leave the research field, Johns Hopkins University President Ronald J. Daniels previously wrote.

Both the media and Congress have criticized the studies various government agencies have funded.

Most notable was a study that ran shrimp on a treadmill, which was recently funded for a second experiment, TheDCNF previously reported. The research received $1.3 million from the National Science Foundation (NSF), which holds the “gold standard” for its review process, according to the agency’s website. (RELATED: Feds Hide Why Shrimp On A Treadmill Cost $1.3M)

Additionally, Arizona Republican Sen. Jeff Flake and former Oklahoma Republican Sen. Tom Coburn released numerous reports pointing out questionable research. Flake recently highlighted $5 million in NIH grants used to study the effects of alcohol on birds’ songs, for example.

Plagiarism and falsified data at NSF has recently skyrocketed, though the agency refused to provide statistics on how much of that misconduct has been prevented through the peer review process, TheDCNF previously reported. (RELATED: Feds Give Billions To Research Based On ‘Falsified Or Fabricated Data’)