It is about time! I got a little distracted, but it is time to finish telling you all about my study section experience. In case you can't remember, in part 1 I wrote about how I ended up as an ECR on an NIH study section, and in part 2 went on about what I did leading up to the meeting. Now part 3 - the meeting!
You guys, the meeting itself was exhausting! We started at 7:30 am, and went until 6:00 pm. We were meeting in a conference room with a giant table. Everyone had an assigned seat, and you got a seating chart so you would know who everyone is. The first few minutes are milling about and getting coffee/pastries*, setting up your computer and getting it connected to the wireless, etc. Then the meeting starts with the SRO welcoming everyone and explaining the rules. You all check your COI form to make sure that it is correct. Then she hands the meeting over to the Chairperson of the study section.
The Chairperson then starts the review process. The first grant is announced, and anyone that is in conflict leaves the room. For each review, it basically went the way it is shown in the movie made by NIH. Each reviewer announces their primary impact score. Then the first reviewer gives a short description of the grant and what were the strengths/weaknesses that led to its score. Then the other reviewers add anything that they considered to drive their score. At this time, everyone else is listening and looking over the grant, particularly the Project Summary and Aims but you could pull up the whole grant on your computer.
After all the reviewers spoke, then anyone could ask questions. There were lots of questions. Questions could be about the science or the scoring. For example, you could ask a reviewer why they gave a grant a 1 but had many concerns about the approach. Or you could ask for clarification about what experiments they were (or were not) proposing. Or you could ask about the interpretation of the preliminary data. The reviewers answered the questions (which is why it is so important to make sure they have all the information they need to advocate for your project!). After 10 minutes or so the Chairperson summarizes the discussion, and asks the Primary Reviewer if there are any concerns about animal use or reagent sharing (seriously people, don't forget to fill these sections out). Then the original three reviewers announce their final impact score, which may (or may not) have changed. This sets the "range". The Chairperson then asks if anyone will be voting outside the range, and you have to raise your hand if you are. If you didn't say anything at all during the discussion but then indicate you are voting outside of the range, the Chairperson may ask why. They mostly want to know if it was something in the discussion or something else. Then everyone enters their score on the online form. And then on to the next grant.
The order that the grants were reviewed was based on preliminary impact score. The R01 from ESI were the first group, with the "best" scored grants first. We only discussed the top half, the rest were "not discussed" (aka triaged)**. But every grant was announced and we all had to agree to triage, so if one person wanted to discuss the grant, then it was discussed. After the ESI R01, we went through all the top half of the rest of the R01 applications. Midway through this group we got a 30 min break for lunch (seriously, it was a LONG day). Then we started on the R21 and R03. There were so many R21. SO MANY. After about 10 hours of this, we stopped and all of us went to dinner, which was pretty awesome. The SRO made sure I got introduced to everyone, especially the BSD in my field that were there, which was awesome. The next morning we met again to finish the R21. We worked through lunch, and finished early in the afternoon. It was crazy.
The whole process was fascinating, really. We all know that there is no formula that derives an impact score, but it is really amazing how each grant was evaluated differently. For some grants, the approach was KEY. But there were some where the reviewers noted significant concerns about the approach but still gave it a very good impact score because of the investigator, or vice versa (for instance). Every so often the SRO would interrupt the discussion to make sure that we were following the rules. For instance, we were not allowed to say "This grant would be better if they had controlled for widget frequency". You could only raise concerns. This was to prevent people from assuming if they did everything the reviewers "suggest" that they would get a better score. Because who knows? There were also some times that the SRO would ask someone to clarify their score, if it wasn't clear that they were following the scoring system rubric. This was to try to avoid score compression. There was also a program officer there, but she didn't say anything during the meeting, just watched.
Being on study section was really interesting experience. I think that I knew many of the concepts of what was happening (thanks for folks like DrugMonkey and PhysioProf, etc.). But seeing it in person really brought it all home.
Now all I have to do is translate this experience into my own successful grant!
Do you guys have questions, or requests to expand on something? I don't have a plan for a Part 4, but I am open to suggestions.
*apparently this is the last meeting that will have these, though. New rules prevent the NIH from buying food or coffee for these meetings.
**You get notified that day if you are triaged, based on my personal experience. But you don't get to see your preliminary scores for a while after that.