Significant Changes in National Institutes of Health Peer Review Process

Front Page

Full Story

Implementation of Significant Changes in National Institutes of Health Peer Review Process Now Underway

As calendar year 2009 begins, the massive grant review system at NIH will be undergoing conceptual and operational changes.  The goal of the changes is to fund research that advances the most important biomedical research while attempting to minimize the inherently conservative and bureaucratic aspects of such a group process.  Assuring the entry of more early stage investigators into the community of funded NIH grantees is also a goal.  The scope of the changes was announced in September 2008, but detailed guidance and firm implementation dates were not in place at that time.  Now that further details have been posted, below is a summary of the changes and an analysis of the issues researchers and sponsored program officers should consider going forward.

ENHANCING THE PEER REVIEW PROCESS

In a series of new notices posted on the "Enhancing Peer Review" website in early December, the first operational steps are explicitly described.  These changes apply to research grants and cooperative agreements submitted for funding consideration for fiscal year 2010 and after (due dates of January 25, 2009 and following).  In most cases, the first round of study section meetings that implement the new policies will occur in May/June 2009.  There may be some exceptions to this implementation timeline, but these will be addressed in the individual funding opportunity announcements for specific program announcements or Requests for Applications (RFAs). 

• Resubmission applications.  In October 2008, NIH announced that, for due dates of January 25, 2009 and beyond, NIH would accept only a single amendment to the original application if it did not receive funding after the first passage through the system.  Submission of second amendments (A2s) will not be allowed.  This policy applies to ALL applications. Currently, Requests for Applications (RFAs) do not permit amendments, unless the funding opportunity announcement explicitly states so.  Applications that were submitted before January 25, 2009 will be permitted two amendments (A1 and A2), but NIH will not accept any of these 'grandfathered' A2 applications after January 7, 2011.  The notice on resubmissions is at http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html.  
• New review criteria.  NIH is not deviating from five core review areas established in October 2004 (significance, investigator, innovation, approach, and environment).  However, NIH has articulated more explicit statements of how reviewers should assess these topics along with additional review considerations (animals in research, budget, resource sharing, etc).  A description of these new 'enhanced review criteria' and a side-by-side comparison with the old criteria are posted at http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-025.html and http://grants.nih.gov/grants/peer/side_by_side_comparison.doc.  However, updated detailed guidelines for peer reviewers are not yet available. 
• New scoring procedures.  As with the new enhanced review criteria, implementation will occur with the May/June 2009 round of review meetings.  A 9-point rating scale, with 1 being exceptional and 9 being poor, will be used.  Individual reviewers will assign an overall impact score of 1 to 9, as well as separate scores using the same metric for each of the five core review criteria.  The priority score for any given application will be determined by averaging the reviewers' overall impact scores and multiplying that average by 10.  Hence, applications can receive scores in the range of 10 (best) to 90 (poor).  Percentile ranking calculations will begin anew with the May 2009 cycle of review and will be reported to the nearest whole number.  The notice about scoring procedures is posted at http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-024.html.  
• Terminology changes.  Some terminology changes occur in conjunction with the new policies.  Applications formerly termed 'amended' will now be termed 'resubmissions' (i.e. A1s).  Applications submitted to continue a previously funded project, formerly termed 'competing continuations', will now be designated as 'renewals'.  Applications requesting supplements for expansion of scope will now be termed 'revisions'.   

A summary timeline for these, and other anticipated changes in the future, is at http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-023.html.

NEW AND EARLY STAGE INVESTIGATORS

In another October 2008 notice, NIH described its new approach to enhance the grant funding success of new early stage investigators, including some procedural changes in peer review frameworks.  The R01 research project grant mechanism was affirmed as the most feasible mechanism of entry for individuals seeking first-time NIH funding.  That notice is at http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-013.html.

PERSPECTIVES FOR RESEARCHERS AND SPONSORED PROGRAM OFFICES

A number of aspects are likely to cause confusion and uncertainty for both researchers and reviewers of grant applications during the two-year period of transition between peer review processes that will occur throughout calendar years 2009 and 2010.  Given the schedule outlined above, there will likely be study section meetings in which study sections will be reviewing both 'grandfathered' resubmissions and resubmissions of post-January 2009 applications.  

• Unlisted criteria in funding decisions.   NIH notes, 'the new scoring system may produce more applications with identical ("tie") scores'.  Hence, the funding institutes will have to take into consideration other factors such as mission relevance and portfolio balance when making funding decisions.
• Overlap between old and new grant applications.  The old criteria and processes are operative for grants to be considered for funding in FY 2009; this is particularly true for study sections that meet to review grants in February 2009.  At the summer meetings of study sections (May/June 2009), the new review criteria and scoring system will be deployed.  At that time, and for subsequent review meetings, a particular challenge will be assuring an equitable and fair peer review for both new system grant applications and those 'grandfathered' in, particularly as the transition period extends into late 2009 and early 2010.  NIH has not developed guidance as to how it will manage this 'dual' population of applicants. 

As a result of the above factors, as well as reviewers' increased awareness of the impending changes and the NIH philosophy behind those changes, 'grandfathered' applications may be at greater risk as the timeline toward full implementation proceeds.  Prudent principal investigators and sponsored program offices may wish to consider the relative merits of pursuing an A2 application versus completely rethinking the project for receipt dates in fall 2009 and beyond.  NIH program officers will be critical for guidance in these particular cases.  Reviewers should be assessing the intrinsic quality of the scientific work proposed and its importance in the context of the research field of endeavor, but they will be making these judgments in the context of  NIH's intent to foster cultural change in its peer review system.