Last August we provided a short post looking at CAREER proposal submissions across BIO and what effects the preliminary proposal system may have had on those numbers. In response to a question on our FY 2014 Wrap-Up post, we’re taking this opportunity to share a little bit more about the CAREER program.
Please read the previous CAREER post if you need an intro to the program.
Overall CAREER Trends
As we discussed before, CAREER submissions in DEB increased markedly from FY 2007 to FY 2009, creating a new normal over the last 5 years; annual submissions are now roughly double what they were in the early 2000s.
At the same time, the annual number of CAREER awards in DEB has also increased. However, CAREER awards have always been a small part of the DEB portfolio— a single digits percentage of total awards for many years.
While these award numbers seem small, CAREER awards have increased since 2008 relative to the total number of awards made to our core programs.
This pattern results from a combination of an increase in absolute numbers of CAREER awards and declining total numbers of other core program awards.
The funding rate (Number Awarded/Number Reviewed) for DEB CAREER proposals has been fairly flat in the long term but with some wild swings concurrent with the recent high submission counts and budgetary changes. Over the 13 year period examined here, there were 914 CAREER proposals reviewed in DEB and we made 108 awards. This gives a 12% funding rate.
Even though the overall funding rate has been on a downward trend for the DEB Core Programs (see FY 2014 Wrap-up), CAREER proposals have suffered less and have closed the gap with regular proposals that was evident before FY 2009 (the spike in 2009 was due to ARRA – the stimulus funding). In the long run, the CAREER proposal funding rate is subject to the same external drivers as the overall success rate, submission counts and appropriated budgets, which adds a great deal of uncertainty to any prognostication based on current trends alone.
Under the preliminary proposal system (FYs 2013 and 2014), CAREER proposals fare about as well as regular proposals going through the entire two-stage process (7-8% funded). However, they fare much worse than the invited full proposals with which they share our fall panels. This backs up a frequent comment we hear from our fall panelists that the CAREER proposals don’t seem to do very well. As a whole, we would expect CAREER proposals to fare worse on average than the pre-screened set of invited full proposals in the same panel because the pre-screening wasn’t applied to the CAREER projects. At the end of the entire process though, the system is identifying and prioritizing for funding high-quality CAREER projects at the same rate as regular projects. This suggests that, on the whole, the project ideas in the body of CAREER proposals fare no better and no worse than the ideas in regular proposals through the DEB review system. Neither the myth that CAREERs are “easier” for eligible faculty to obtain nor the opposing claim that they more prone to being trashed in review are supported by the funding trends.
We can break down the numbers of CAREER proposals and awards by cluster. Below, we’ve put the proposals and awards on separate figures to allow more vertical detail and show the contribution of each cluster to the total counts of CAREER proposals and awards.
Submission numbers for the CAREER solicitation tend to mirror the distribution of Core Program submissions in general: Population and Community Ecology (PCE) receiving a plurality, followed closely by Evolutionary Processes (EP), and then Ecosystem Science (ES) and Systematics and Biodiversity Science (SBS). Prior to 2007, ES represented a larger portion of CAREER submissions than at present but has since been eclipsed by growth in submissions to PCE and EP. Our interpretation of this is that because CAREER proposals are all single-investigator projects they exhibit less potential for growth in this community (ES researchers) that places a strong emphasis on collaborative projects.
Award numbers also provide for some interesting discussions within the limits of such small numbers. The growth in DEB CAREER awards is primarily due to increases in the number of awards made in the EP and PCE clusters. While the PCE growth has been consistent with the submission growth (12% funded prior to 2009, 12% funded since 2009), the awards numbers in EP have grown more than the submissions (6% funded prior to 2009, 16% funded since 2009). There has been little to no change in ES CAREER award counts, matching their lack of submission growth, and the same can be said for SBS.
Pivoting from the cluster view, we also wanted to present some numbers relating to who applies for and who receives CAREER awards.
|Times (re)submitted, 2002-2014||PI Count||Awarded PIs||Success Rate|
The first thing is that the plurality (if not majority[i]) of CAREER awardees were funded on their first submission. But, a majority of applicants also try only once for a CAREER. The success rate for these 1-timers is ~14%. However, the PIs who come back and try again do better (21%). While fewer PIs succeed at each resubmission, the number attempting resubmissions drops off much more quickly meaning those PIs are funded at much higher rates. Someone on a 3rd attempt at a CAREER is more than twice as likely to be funded as someone on their 1st attempt. (On a side note: to the extent we can do so, we observe a similar pattern in regular proposals too. But, it’s much harder to accurately track “resubmissions” due to PIs juggling multiple proposals, and changes in personnel, titles, and programs; thus the magnitude of the effect for full proposals is uncertain.)
Also, successful CAREER PIs are not necessarily those who are the newest faculty.
Given the requirements of the CAREER solicitation, it can be advantageous to already have a lab up and running and to have gotten into the groove of being a researcher/educator. Also, prior experience with writing a successful grant helps when you’re attempting to write a successful grant for the challenging criteria of the CAREER program.
One of the important messages here that is tough to illustrate directly with charts and tables is that CAREER Awards have a different purpose than regular research grant awards, and this distinction sometimes eludes PIs and reviewers. CAREER projects emphasize the integration of research and educational activities; this requirement is highly selective for a subset of PIs who are passionate about this professional nexus. In fact, these requirements can be so organizationally difficult to execute that we often advise the newest faculty against applying by this route when they are just getting their first lab together. A major failing of many declined CAREER proposals is inadequate integration of research and educational activities: it might have been a competitive regular proposal but because the PI selected a higher bar, they didn’t pass it.
As hard as it is to conceive of and present a strong case for a CAREER proposal, it can also be difficult for reviewers to rate these proposals when they themselves are not experts on the educational integration side of things. That is a good argument for allowing CAREER proposals to come in directly as full proposals because they can benefit from selection of ad hoc reviewers with strengths in those aspects otherwise on the periphery of expectations for a regular proposal. And, journeying into the realm of speculation, one might wonder if the increase in submissions actually contributes to an increased funding rate because reviewers then have a larger pool on which to base their expectations for these projects. Perhaps we hit some critical mass of understanding by panelists that helps overcome misperceptions. Either way, supportive panel reviews are an important part of justifying CAREER award decisions and the charts above should make it clear that we fully expect CAREER awardees to be able to pass that (elevated) bar.
While they are a valuable facet of our award portfolio that we support and try to give every consideration, we don’t have quotas and don’t provide a free pass to funding for CAREER proposals. Any decision to apply for a CAREER should be based on a weighing of your own variables: personal preferences, professional preparation, research interests, practicability, but not simply because someone said it would be “good to do”. To put it another way: CAREER proposals aren’t for everyone and you only get three attempts at it. If this is something you both want to do and feel you can do, that’s great. But, bowing to pressure to submit a CAREER project when it’s not a good fit for you or before you’re ready can distract from and delay pursuit of other potentially successful proposals.
Now that we’ve left you pondering whether it makes sense to submit at all, we want to try to close with something more positive for potential CAREER applicants to consider. Think about submitting a preliminary proposal before you submit a CAREER proposal. This is essentially a “no risk” proposition. Regardless of outcome, you will get some feedback on your ideas. This feedback may or may not be directly applicable to a CAREER proposal given the differences but you’ve gained useful experience. Even if you get a preliminary proposal rejection, you wind up with additional data that you can consider to make an informed (not reflexive) decision as to whether you can improve the work and present a strong case for an integrated educational component. And, you might just get an invitation for a full proposal. In that case you have not only gained feedback but a choice as to whether you want to pursue a full proposal or attempt to turn that idea around and transform the project into a full-fledged CAREER.
[i] Caveat: because we didn’t track individuals back before 2002, those who appear once during the early part of the data could be on a 2nd or 3rd submission.