Q: I’ve gotten my reviews back and contacted my program director to set up a time to talk about them. In her reply she suggested I make sure to read the context statement before we talk. What is that and where do I find it?

A: Each panel has a document that describes the panel process and outcome for that panel, called a context statement. While the panel summary and the reviews give you an idea of the response of the reviewers and panelists to your proposal, the context statement allows you to get some idea of where your proposal was ranked relative to the others. It tells you how many proposals the program reviewed and what proportion the panel(s) placed in each of the panel rating categories. Context statements are written for the program rather than panel because the decisions are made by the program and the numbers might reflect more than one panel especially at the preliminary proposal stage. The context statement can be found directly below the reviews in either Research.gov or Fastlane.

In this most recent round of full proposal panels in fall 2015, IOS adopted a new set of rating categories and definitions, moving from 4 categories to 5 categories. These new categories will be used for both preliminary and full proposal panels moving forward. As before, proposal placement is based on the subjective judgments of the panelists and it tends to be the case that they are relative judgments rather than absolute ones.

The categories are:

Outstanding

• Innovative, bold, imaginative Intellectual Merit
• Substantial Broader Impacts
• An important advance in the field (and possibly beyond) is likely

High  

• Significant and compelling Intellectual Merit
• Solid Broader Impacts
• Expected to have a noteworthy impact in the field

Medium

• Moderately exciting and likely to make a useful contribution
• Broader Impacts efforts at least satisfactory
• May have some minor deficiencies

Low

• Questions addressed may be important, but the feasibility may be in doubt
• Alternately, the importance of questions may be marginal
• Broader Impacts efforts satisfactory or insufficient
• Worthwhile project but requires major changes to be competitive

Not Competitive

• Intellectual Merit flawed in logic, feasibility or approach
• Efforts toward Broader Impacts perhaps not substantial or effective
• Unlikely to provide a useful scientific contribution

Rationale

Why did we make this change?

It had been more than 5 years since we had last revised the rating categories and in the interim, the Plant Genome Research Program which used a different set of categories had joined IOS. Additionally we had been breaking our prior categories into an upper and lower sub category (e.g. Medium1 and Medium 2) at panel but only reporting the main category to PIs. We found that we did not really need 2 parts to all the categories. So a new set with 5 categories gave the panel the means to spread out the proposals relative to each other and also would be more transparent. Additionally this new set, would allow all of IOS to use the same categories, making it easier for programs to share staff and hopefully easier for our applicants.

While the categories are slightly different, much else about the panel remains the same. Panels continue to be advisory to the Program Directors who may invite or recommend a proposal for funding from any category. Each program has a limited number of invitations based on their expected budget the following year and of course a limited budget to make funding recommendations from. In these tight fiscal times, Program Directors rely on clear advice from the panels and often find themselves with far more good science than they can invite or recommend for funding.