Is it cheating to propose experiments in a grant that are "done"?

Sep 27 2012 Published by under academia, grants

If you have already FINISHED the studies, why would you propose them in a grant? I know that there is a common meme that "you have to have done the work to get it funded", but I have never behaved that this is an actual real expectation.

This comes up because Fred, a new PI, left a comment over at Dr. Becca's place*:

...we are seriously thinking about publishing most of the preliminary data (and data not shown in the grant) about 1-2 months after submitting the grant. This manuscript would show we have completed more than half of proposed studies in the grant.

Fred wanted to know whether it is a good idea to publish the paper. But I am wondering WHY that was the grant that was written. If you had done the experiment wouldn't it be cooler to talk about what new awesome thing you could do next? I get that you need preliminary data for a grant. I have always written my grants with preliminary data that showed I was able to do the experimental procedure. I have always written my grants using Preliminary Data to demonstrate feasibility. But the reason I needed money was to actually DO the experiment. I may "know" the answer (or at least think that I do), but I haven't actually done the experiment.

It feel like holding back data, and proposing experiments that you have already done is kind of sketchy. And I don't understand the motivation. Is the idea that you will be able to show fast progress on the grant (if it's funded) and that will be awesome?

Help me out, guys. Am I totally off base here? What am I missing?

Also, FTR, in my n=1 experience on study section, if there is evidence that you have already published the results of the proposed research it is viewed poorly by the reviewers.

 

*I started to comment over there but realized that I was going off on a tangent.

30 responses so far

  • Bashir says:

    My impression was that many people do this to some degree. Maybe not quite on the level as described. Don't they say that the best way to get a grant is to have already done the work?

  • Namnezia says:

    I've come across a situation where I submit a grant and it doesn't get funded, and by the time it was resubmitted several times and finally picked up by the funding agency, a bunch of the experiments were already done. Would it make sense then that by every resubmission you start removing aims because you've done them and proposed totally new thing so you can get dinged again for not having enough preliminary data? I don't think so.

    • gerty-z says:

      No, I agree that doesn't make much sense. Do you include the new data in the grant when you resubmit? Or do you hold it back so that it looks like you haven't done so much yet? And, FWIW, Fred was talking about a first submission (at least that is how I understood it)

      • Fred says:

        Yes, it's a first submission. This is a project I started with my n00b start-up funds. We've been very productive in the 6-12 months it's taken us to develop it. A lot of 2 aims are near complete, but I'm proposing to "finish them off" during the grant period, with Aim 3 as the one I need to fill in more holes.

  • Dr kalmia says:

    This happens in pretty much all fields and is pretty common from what I understand. The idea behind it is that you can then use the grant $ to start the next project so you have solid "preliminary data" for next grant.
    I can see how this cycle works well and improves funding rates for established land (like my PhD lab) but this game absolutely sucks for young scientists and new labs who have to write proposals and compete with projects that are already 1/2 done.

  • Ed says:

    Sydney Brenner had an excellent column musing on the question of preliminary data years ago, and came up with a wonderfully novel, and typically clever, solution:
    "All the world's a lab ... then the assistant professor" http://www.sciencedirect.com/science/article/pii/S0960982200002177
    Another reason people write already knowing the answer is that it makes hypotheses much easier to frame. You've already worked through the problems and considered alternatives in real life, and just have to retrospectively present them in the proposal as what you'll consider in the future.

  • drugmonkey says:

    I'm on both Gerty-Z's and Namnezia's sides on this. I generally worry most about technical feasibility data and much less about robust hypothesis-test-feasibility data in my proposals. It is also true, however, that science progresses as you deal with the rounds of review/reject/revise. And it often becomes a close game of what to hold back so as to not look like you've made too much progress. My bias is, however, to keep publishing and adjust the Aims or experiments in the revisions of the grant to accommodate any progress we've made. There is no rule against including published figures as Preliminary Data and I've done that with regularity. The key is to describe where you are going to go from the current state of your pubs....

    With that said, Fred's comment

    The first Aim would be mostly complete, and good portions of the other 2 aims would be complete.

    concerns me slightly. The idea that the lab has one manuscript that covers this much of the proposal suggests that the proposal (an R01) is far too limited and/or conservative. If there is no room left for additional investigations under the Aims, then perhaps these are not really Aims but more like Experiments?

    • Fred says:

      I should also say the reason this manuscript COULD be ready to submit in 1-2 months is because:

      (1) the data are that good and flowing like butta,
      (2) my postdocs are m-o-t-i-v-a-t-e-d, and
      (3) some of the studies via collaborators who have their shit together and git-er-done

      So to make the nay-sayers happy, I should tell everyone to stop working ... I guess ... for a few months?

    • gerty-z says:

      No, you shouldn't have folks stop working. But have you considered expanding the Aim so that there is more room to do new awesome stuff, building on your preliminary data?

  • drugmonkey says:

    I'm proposing to "finish them off" during the grant period

    I would say you never, EVER want the reviewer to think that is what any of your Aims are about.

    • Fred says:

      I know someone who got a grant funded (below 10%) who did just this

    • gerty-z says:

      maybe this is field specific, because IME if your Aims are just "finishing up" then you get dinged for "innovation" or else your work is perceived as "incremental". All of these are not good.

      • Fred says:

        i guess i don't see how almost completing an Aim makes it less Innovative or makes the work Incremental. if you propose to study something super kick ass, submit it as an Aim in a grant, and in the next few months happen to have had a lot of success and mostly finish the experiments, how does that make the work less Innovative or more Incremental? from what you describe, it sounds to me like these study section members are too fixated on their lack of progress and preliminary data vs. someone else's.

        • gerty-z says:

          well, it seems to me that if you proposed a real, meaty Aim then there should be no way that it was "completed" in a couple of months. But maybe if you present a whole crap-ton of preliminary data then it will make reviewers think you are Serious. I only have n=1 experience on study section, and maybe it was unique to the panel I was on, but there were grants where someone raised the concern that the application didn't seem relevant/important because the work had pretty much already been published. The small bits that remained were "clean up" work that was incremental. For your first submission it may be that a JIT new publication could be a good thing. But you should also, then, expect that if you have to revise and resubmit that you will have to change the Aims to reflect that one of them is "done".

  • Boehninglab says:

    I have had the best luck with grants when a good paper with portions (or most) of the preliminary data come out between submission and peer review. The scope of my grants are generally MUCH larger than the preliminary data presented (sometimes to a fault). Like drugmonkey, I often include published data in the prelim data section if it supports the hypothesis and/or feasibility. Lately I have found a very strong correlation between innovation and overall score (disproportionate to other scored criteria), so I have tried to focus on the innovative aspects of my projects and data to support my claims/hypotheses.

  • Dr24hours says:

    My mentor always said that a good grant application reduced everything to labor. Meaning: all of the thinking is done. All you need to do is physically produce the things you say you'll do. But the labor should remain to be done.

    • Fred says:

      I think what I said has been misinterpreted ... the labor isn't done now, before the grant is submitted.

      After this grant is submitted in less than one week, we are still going to be working on it--very hard--and at this rate it is likely at a good chunk will be done in 2 months time and ready to write up.

      These are not the same:

      1. the Aim is done at the time of grant submission (not the case here)

      2. the Aim has important experiments left to complete at the time of grant submission and, after submitting the grant, hell yes we are going full stream ahead and if all goes well our team (note: this is not just one person developing this project) will complete most of the studies in this Aim

      So is one to propose studies that one would only perform 9 months after the grant is reviewed and funded? That makes no sense.

      Are people supposed to stop working on proposed projects while they are under consideration? I didn't think so.

  • becca says:

    It's certainly not 'cheating' by any reasonable moral standard I can see, but I haven't a clue if it's good strategy.

  • proflikesubstance says:

    I have a project that has bounced around DC for some time. It's always right on the edge of funding, but hasn't yet made the jump. In its original incarnation we proposed three Aims that, at this point, are done. But what I'm not quite understanding from the discussion above is why anyone would resubmit the proposal without making changes to reflect the progress.

    None of the original aims persist in my proposal anymore. The data from those Aims has been incorporated as preliminary data to expand the project and take it much further. No, you should never stop working on a project while waiting to hear back, but nor should you project be so limited that it doesn't continue to expand as you produce more data.

  • DrLizzyMoore says:

    This might be field specific, but in my experience, the BSD labs generally put in grants where the Aims are largely done. I don't think my generation of PIs will get away with this.

    To me, Preliminary data can be published (or on the cusp of publishing) but the teeth of the grant needs to be done and should go well beyond prelim data.....of course what do I know. Year 3 just started. It's nut up or shut up time......

  • Crystaldoc says:

    Given the 5 year scope of an R01, I've received advice, and makes sense to me, that the reviewers should see the proposed work under best case scenario leading to 6-10 papers. Maybe 3 aims, each with 2-3 sub aims, each of which can be envisioned as a self-contained research paper that might appear in a medium impact society journal. Shouldn't this be the expected outcome? Given the 12 page length, you are better off if you can cite your published work for feasibility and methods, and use the space to explain proposed experiments, and even more so your rationale and why the work is important.

  • Cynric says:

    This debate rages in the UK too. It partly depends on the stated aims of the grant scheme, of course, but the workhorse Research Council grants are supposed to be for future research. I have been contacted before now querying how I managed to publish the first paper attributed to the grant so quickly. But as Fred says, the time between submission and award is so long (in science progress terms), that we'd been working on aim 1 in the interim.

    I've always been struck by the need to serve different masters: to get the grant awarded, you need to persuade your reviewer peers that the work is feasible (with lots of solid preliminary data), but the Research Council staff who assess progress post-award stick to the letter of the terms and conditions.

    All that said, everyone I've dealt with at the Councils has responded well to carefully justified explanations. Presumably, NIH would be the same...?

Leave a Reply