DEB Numbers: CAREER Submission Data

This is a quick numbers post while we in DEB pivot from summer research and meeting outreach to fiscal closeout and autumn (full proposal) panel mode.

CAREER proposals in BIO were due on July 21, 2014. These proposals will be reviewed this fall and become part of the FY 2015 decision-making process. In this post, we take a look at the trends in submission of CAREER proposals through the current competition. We aren’t looking at funding rates or outcomes – those are beyond the scope of today’s post.

If you have no clue what a CAREER proposal is, skip to this short primer.

DEB CAREER Submissions Trend

The CAREER program is a long-standing, foundation-wide opportunity for junior faculty. In DEB, CAREER proposals can provide eligible PIs an additional submission above and beyond the 2 allowed annually in the core programs.

Some DEB staff were concerned that we would see a substantial increase in CAREER submissions after the launch of the preliminary proposal process for the core programs. The reasoning was that CAREER-eligible junior faculty who were not invited to submit a full proposal would submit a CAREER instead. Alternatively, others pointed out that a competitive CAREER proposal differs substantially from a regular research proposal and that one does not simply flip a project idea from one to the other. Going in to the preliminary proposal system, it was an open question as to how CAREER submissions would respond.

CareerTrend1

Between 2002 and 2007, we received 40-60 CAREER proposals per year. There was rapid growth from 2008 to 2009 and 2010, well before the preliminary proposal process was launched, but that trend has not continued over the 5 subsequent deadlines. There is a jump in 2013, the first CAREER deadline after the switch to preliminary proposals, but that level has not been maintained. It appears that, 3 years in, we have not experienced a large shift of DEB proposals into the CAREER mechanism. Anecdotally, a few comments we’ve heard suggest that CAREER proposals continue to be seen as prestigious awards with high expectations at the review stage so many eligible PIs may be hesitant to apply. Supposing this view is sufficiently widespread it could easily suppress the numbers of PIs who would otherwise turn to this program after or instead of a core program submission.

Elsewhere in BIO

While there was little change in CAREER submission numbers after the switch to the preliminary proposal system in DEB, the story is somewhat different in IOS (which instituted preliminary proposals at the same time) and MCB (which instituted a single deadline and PI submission limit of 1). Both of these Divisions also started from a very different place than DEB with respect to CAREER submissions. IOS and MCB received larger numbers of CAREER proposals than DEB in the years immediately prior to their respective review process changes and there are less tangible but important cultural differences in community expectations and approach to CAREER proposals. MCB was coming down from a submission peak (217 CAREER proposals) in 2006 and IOS was at a plateau ~110 following substantial submission growth in the early 2000s.CareerTrend2

*The 2015 numbers may be revised if some submissions are returned as incomplete or out of compliance.

Both IOS and MCB experienced actual and proportionately larger initial jumps in CAREER submissions post-2012 than DEB. And, neither IOS nor MCB experienced a drop of similar magnitude to DEB during the second or third year since their solicitation changes. It’s not clear why the reaction in these two Divisions differed so much from the DEB communities. Potential reasons include: larger communities with larger potential for increases, influx of PIs from the biomedical community seeking funding, and differing perceptions and expectations for CAREER submissions relative to regular proposals.

Summary

Even though three years is a short run, CAREER submissions from 2013 onward in each of the three Divisions appear to be relatively stable. Concern about overwhelming growth in CAREER submissions has not been borne out. However, it is unclear why numbers went up and remained there in MCB and IOS but not DEB and what that means for us. Are DEB PIs and/or reviewers much more selective as to what they will put forward for a CAREER award? Do the demographics differ between the fields enough that we were already saturated with CAREER submissions while the other Divisions were not? Or is the variation stochastic? If anyone has data that might shed some light on these questions, we’d be happy to know.

CAREER primer:

CAREER is shorthand for the NSF-wide Faculty Early Career Development Program. The CAREER program is distinguished from other NSF opportunities by being exclusively for pre-tenured faculty and specifically focused on excellent educational aspects integrated with an outstanding research program. In addition to being considered a prestigious award in its own right, receipt of a CAREER award is a prerequisite of eligibility for the Presidential Early Career Awards for Scientists and Engineers (PECASE).

CAREER program submissions have a single annual deadline in mid-to-late July (exact date varies by directorate) and are reviewed by the most relevant disciplinary program. In DEB, CAREER proposals are taken to review in our fall full proposal panels, alongside invited full proposals, OPUS, and RCN submissions, and co-review proposals as we have previously illustrated. While not the normal practice in DEB, it is fairly common across NSF to have panels exclusively for CAREER proposals in programs where there are large numbers of CAREER submissions and/or no other close deadlines. [back to top]

DEB Live! 2014: ESA in Sacramento, 10-15 August

DEB is coming to the 2014 Annual Meeting of the Ecological Society of America in Sacramento, California.

We are expanding our formal presence at this year’s meeting by:

(1) Holding our regular lunchtime “Town Hall” session on Tuesday, August 12 from 11:30-1:15 pm in room 202 of the Sacramento Convention Center.

(2) Hosting a Table in the Exhibit Hall  with general information about NSF and the Biological Sciences Directorate, including funding opportunities in and beyond DEB (as of press time we expect to be Table B). Expect 1 or 2 representatives at the table throughout the day.

Our ESA 2014 Table. Stop by and say "hi."

Our ESA 2014 Table. Stop by and say “hi.”

(3) Program Officers will also generally be available for questions at the NSF exhibit table during the late-afternoon poster sessions: Monday, Tuesday, Wednesday, and Thursday from 4:30-6:30 pm.

(4) If you wish to arrange a special time to meet with one of these folks you can contact them by email.

 

Below are individuals from NSF who will be in attendance at the meeting:

Penny Firth, Division Director
Division of Environmental Biology
pfirth@nsf.gov

Alan Tessier, Deputy Division Director (Acting)
Division of Environmental Biology
atessier@nsf.gov

Henry Gholz, Program Officer
Division of Environmental Biology, Ecosystem Science Cluster
hgholz@nsf.gov

Peter Alpert, Program Officer
Division of Environmental Biology, Population and Community Ecology Cluster
palpert@nsf.gov

Linda Deegan, Program Officer
Division of Environmental Biology, Ecosystem Science Cluster
ldeegan@nsf.gov

Betsy Von Holle, Program Officer
Division of Environmental Biology, Population and Community Ecology Cluster
mvonholl@nsf.gov

John Schade, Program Officer
Division of Environmental Biology, Ecosystem Science Cluster
jschade@nsf.gov

Alan Wilson, Program Officer
Division of Environmental Biology, Population and Community Ecology Cluster
aewilson@nsf.gov

John Adamec, Program Analyst
Division of Environmental Biology
jadamec@nsf.gov

Cheryl Dybas, Public Affairs Officer
NSF Office of Legislative and Public Affairs
cdybas@nsf.gov

 

NSF.gov Web Survey: How well is the official site meeting your information needs?

There’s a survey up for public input/comments related to the material and organization of sites on the official NSF.gov homepages for Directorates and Divisions (e.g., the BIO Directorate and DEB’s Division page).

From the NSF.gov website management:

Dear Colleagues,

The National Science Foundation (NSF) is working to improve our web site, NSF.gov. We are initially focusing on the pages that provide Directorate- and Division-specific information. NSF is divided into seven Directorates that support broad areas of science and engineering research and education. Each Directorate is divided into several Divisions, which focus on more specialized disciplines (see the NSF Organization List).

Help guide the future direction of the NSF web site by completing our online survey at https://www.surveymonkey.com/s/nsfwebsitesurvey by August 19. It should take no more than 10 to 15 minutes to complete and is anonymous. The OMB Control Number for this information collection is 3145-0215.

Thank you for your participation. We look forward to receiving your feedback on how our web site can serve you better.

DEB Live! 2014: Botany in Boise, 26-30 July

Botany 2014 will take place in Boise, Idaho from Saturday July 26 through Wednesday July 30.

There are several chances at this conference to interact with DEB and other NSF/BIO representatives.

  1. We will have an information booth in the Exhibit Hall, (#24 on the map).
  2. On Monday 7/28, we will be having a Lunch and Learn at the Exhibit booth from 12:00 – 1:30pm
  3. We will be available at the booth during the Poster Sessions on Monday evening from 5:30-7:00pm
  4. We are hosting a brown bag information session on Wednesday July 30 from 12:00 – 1:30pm at the Eyrie Room/Boise Center
  5. If you wish to arrange a special time to meet with one of the NSF attendees you can contact them by email.

 

Below are individuals from NSF who will be in attendance at the meeting:

Michele Dudash, Program Officer
Division of Environmental Biology, Evolutionary Processes Cluster
mdudash@nsf.gov

Simon Malcomber, Program Officer
Division of Environmental Biology, Systematic and Biodiversity Sciences Cluster
smalcomb@nsf.gov

Joe Miller, Program Officer
Division of Environmental Biology, Systematic and Biodiversity Sciences Cluster
jtmiller@nsf.gov

Roland Roberts, Program Officer
Division of Biological Infrastructure, Research Resources Cluster
rolrober@nsf.gov

Irv Forseth, Program Officer
Division of Integrative Organismal Systems, Physiological and Structural Systems Cluster
iforseth@nsf.gov

 

International Research Experience for Undergraduates (IREU) Supplements: Notes from the field

An International Research Experience for Undergraduates (IREU) supplement is a modification of the NSF-wide Research Experience for Undergraduates (REU) program. These IREU supplements are available exclusively to PIs in DEB. Traditional REU supplements available to NSF funded investigators support mentorship for undergraduate students to conduct independent research.

DEB has developed a partnership with CAPES, a funding agency in the Brazilian Ministry of Education. Through this partnership, DEB-funded investigators are eligible to apply for an IREU supplement to NSF in parallel with a Brazilian colleague who can apply for undergraduate funding through CAPES. If awarded, DEB funds the US student, and CAPES funds the Brazilian student so both students have an opportunity to conduct research at home and abroad. Over the course of the supplement, the two undergraduates overlap in the laboratory or the field and conduct a veritable student exchange.

This type of experience can spark a passion for science and research in undergraduate students. While the IREU supplement opportunity is still nascent, it has already provided numerous students the opportunity to conduct international research. Furthermore, programs like this allow international funding agencies to make use of their aligned interests and provide a greater funding impact through coordination and cooperation.

We had an opportunity to catch up with one of these students and her mentor as they shared some of their experiences in the IREU program:

~~~~~~~~~~~~~~~

Kendra Avinger

Undergraduate Student, Rutgers University

US Mentor: Dr. Siobain Duffy, Rutgers University

Brazil Mentor: Dr. F. Murilo Zerbini, Federal University of Viçosa

Duffy_IREU_student

Kendra Avinger, a DEB IREU student.

Tell us a little bit about your IREU experience thus far?

Surreal is the closest word to describe my IREU experience thus far. My mother is a second-generation Puerto Rican who raised my sister and me. The IREU experience is the type of horizon broadening opportunity she has always hoped for me.

In Brazil I am studying Begomoviruses, a group of viruses that cause disease in dicotyledonous plants, such as Tomato golden mosaic virus (TGMV) and East African cassava mosaic virus (EACMV). Begomoviruses were reported have major agricultural impacts with losses of 40 to 100% in Southeastern Brazilian states. I am learning to collect, catalogue, amplify, and sequence samples of Begomoviruses, as well as assisting multiple graduate students in the lab with their Begomovirus projects.

Have you participated in international activities before?

This IREU is the first international activity I have been involved in, in any capacity.

How did you prepare for your trip?

Mental preparation was key for me before arriving in Viçosa, Brazil. It is important to accept the overwhelming feeling of a new culture, location, and language without allowing it to overwhelm you and consequently, your work. To academically prepare for these differences, I enrolled in a Portuguese course tailored to speakers of the Spanish language, as I am fluent.

Tell us about working with your Brazilian counterpart?

My Brazilian counterpart, Hermano Pereira, and I overlapped at Rutgers for a week before I left for his University. Although our communications were filtered through two languages, we were inspired by our shared connection as young scientists.

Tell us about your mentors?

My mentor in Brazil is the phenomenal Dr. F. Murilo Zerbini. I am the first student to be sent to Brazil from the Duffy Lab, yet Professor Zerbini has had 3 students travel to Rutgers so far.

My US based mentor is Dr. Siobain Duffy. She understands that thriving graduate students are born from efficacious, confident undergraduates. She has helped me to realize that I have as much creative power as any professor, and to view myself on the same playing field. She gives me the confidence to move forward and share my own ideas.

What are your future professional/academic plans?

With my future, I hope to effect change eclectically. Public speaking, presenting, technical writing, and life sciences pull at my strong points thus far. I am not sure where I would like to end up long term, however, a short-term goal of mine after graduation is to manage a lab somewhere that is warm year-round, explore other interests, and eventually apply to graduate school.

~~~~~~~~~~~~~~~

Dr. Siobain Duffy

Assistant Professor, Rutgers University

How did you first get involved in conducting international research?

PI and IREU mentor, Siobain Duffy.

PI and IREU mentor, Siobain Duffy.

I am lucky that evolutionary virology is a very international field. Not just because cataloging viral diversity is a global pursuit, but some of the best experimental viral evolutionary biologists are in Europe, and some of the computational tools we use on a daily basis were developed in Africa.

My closest international collaborations both came from introductions made by a mentor of mine, who is very active internationally herself. As I was starting out as an assistant professor, she invited me to collaborate on some large multi-institution proposals. Some of them succeeded, some didn’t get funded, but the emailing back and forth created the personal connections required to start research collaborations.

Science has always advanced most quickly when ideas are shared internationally, and email, online dissemination of journal articles and VoIP technology [e.g. voice over internet technology] have made it trivial to collaborate with people on the other side of the planet.

What advice would you give other investigators who are considering applying for an IREU supplement?

My best advice is to start everything earlier than you think you need to – visas to the US take time, and undergraduates are less likely than grad students and postdocs to have gone through these processes before. The PIs on both sides will have to walk the students through obtaining housing, making sure their health insurance is set up, making sure they have working cell phones, etc.

What advice would you give to undergraduates, who may be inspired by Kendra’s work, and who are interested in getting involved in international research?

There are many more opportunities for international research exchanges than undergrads realize there are! If you are working in a lab with international collaborations, ask if there is a chance to participate, and if there is time to write an IREU proposal for your work. Look for REUs with an international component! Look for post-baccalaureate programs, especially in countries where you are already proficient in the main language. A former undergraduate researcher in my lab is working in a German science lab doing exactly this.

And don’t sweat it if your research doesn’t turn international at this stage – if you stay in science, there’s a very good chance you can work and live abroad for a while.

Has the IREU supplement impacted your collaborations with international investigators?

NSF’s partner in this exchange, CAPES, has been so pleased with the project they are funding additional Brazilian undergrads to come to my lab this fall. Because I have visited my collaborator’s lab in Viçosa, I know these students, and they already have some sense of who I am and what their time in the US will be like – which makes it much less intimidating to get on the plane. I will be looking for ways to write more IREU students into projects in the future.

 

DEB Numbers: Where do the various official funding rate numbers come from and why are they different from what your PO tells you?

We have recently received several questions by email from PIs wondering how the funding rates shown in official reports can be so different from what they experience in DEB. We thought that it would be good to post a response here on the DEBrief.

What are the official NSF funding rates and where does one find them?

Official funding rates published by NSF vary depending on the program(s), division(s), or directorate(s) being examined. These numbers are calculated in the same way across all of NSF.

You can find official funding rate data here: http://dellweb.bfa.nsf.gov/awdfr3/default.asp and can use the tool to drill down and look at different divisions and programs. These numbers should match the ones you see publicized in official statements and on the nsf.gov site.

As you may notice, the posted funding rates are several times higher than what you hear in discussion with your PO, during panels, or in our context statement.

For instance, according to the official numbers, DEB in 2013 had these stats:Numbers.biis.1

That’s 1,751 proposals, 409 awards, for a 23% funding rate. The mean time to a decision was 4.65 months. An “average award” in 2013 had a duration of just under three years and is funded at ~$80K per year.

Why do these numbers differ from the ~3-7% funding rates you’ve heard from various personal sources?

Missing from the Denominator:

Well, for starters these numbers don’t count preliminary proposals. The official numbers are based only on the proposals that lead directly to funding decisions. You can see that if you look back a couple of years in DEB:Numbers.biis.2

According to these numbers we had ~1000 fewer proposals in 2013 than in 2010. That’s not an error, we did have fewer full proposals, but it is misleading because the source of the count isn’t detailed here. In DEB,when we report funding rates to you, we report separately the pre-proposal invitation rate, and the full proposal funding rate, or we report a single funding rate for the whole process[i].

Notably, this artificial dip in submissions/spike in funding rate can actually be seen on the top-line NSF numbers:

Numbers.biis.3The drop in submissions from 2011 to 2012 is largely because DEB and IOS Preproposals aren’t counted there, resulting in an apparent increase in the NSF-wide funding rate while the number of awards didn’t change much. However, the effect of sequestration (on top of losses from run-of-the-mill inflation and risings costs of research) can be seen from 2012 to 2013 in the drop in awards.

Proposal duplication:

The official numbers count each proposal (“award jacket”) separately. These counts do not combine multi-institutional collaborative proposals into a single unit. This has a sizeable effect on the numbers for DEB, especially within the core programs. Many of our awards involve two or more collaborative proposals reviewed as a single unit; we feel that most of you would consider that such a unit should be counted once instead of each component counting separately and that’s what we do on this blog. However, in the official numbers, a three-partner collaboration counts as three separate proposals and three separate awards or declines[ii]. Generally, collaborations counted in the official manner inflate the funding rate by a few points compared to the individual programmatic reality.

Lumping proposal categories:

We discussed in a previous post the organization of core and special programs in DEB. These are all lumped together in the official DEB funding rate calculation. For the most part, items like RAPIDs, EAGERs, and Conference support are a relatively small piece of the total and are only responsible for about a point of the official vs realistic funding rate discrepancy. You can drill down in the official numbers to get a better look at program-level outcomes. As you can see, there’s much variation here[iii]:

Numbers.biis.4

The two most important messages are a bit buried amid all those lines.

First, the “special programs” are not generally inflators of the funding rate. Dimensions of Biodiversity was at 14%, CNH 6%, and EEID 4%: cumulatively they represent about 20% of the official proposal count and only 10% of the official award count. These programs do not provide an advantage to submitters[iv]. They may be desirable to you for other reasons – interdisciplinary content and potentially larger awards than a disciplinary core project – but they are not an easier route to funding.

Second, the biggest inflator of the funding rate is not obvious to see, but a careful reader might have already picked up on it. Take a close look at the median annual size column, anything stand out? Do the median award sizes for Ecosystem Studies, Evolutionary Ecology, Evolutionary Genetics, Phylogenetic Systematics, and Population and Community Ecology seem a bit small?

A part of this is a reflection of what we already discussed above: each proposal is counted separately so a big $650K project may be comprised of three jackets each receiving $72K per year for 3 years which shifts the median size downward in the official count. But that doesn’t bring it down to $12K or $15K in some of those programs. The main reason official rates appear so much higher than reality is that they lump Doctoral Dissertation Improvement Grants (DDIGs) into the same count as regular research proposals.

Doctoral Dissertation Improvement Grants at ~15K cost just a fraction of a full project: it would take 20, 30, or more DDIGs to account for the cost of a single regular research project. Thus, a little bit of money goes a long way in funding DDIG proposals[v]. The result is that DDIGs can be relatively numerous (50%+ of award jackets in some programs) and enjoy a relatively high funding rate that when lumped into the official count, results in misleadingly high success rates for DEB as a whole.

 

While the topic of funding rates may be a bit confusing, we hope that this post has shed some light on why you may be seeing different funding rate numbers across NSF and DEB. As you all know, with summary statistics the numbers all depend on how the data are being analyzed.

 

[i] These don’t combine exactly because there are proposals like CAREERs and co-reviews that are part of the funding milieu but skip out on the preproposal stage; we covered that previously, here.

[ii] If in our full proposal panels we report funding 4 of 20 projects when counting collaborations only once each for a 20% success rate, the official count would reflect something like 7 of 30 jackets funded for a 23% rate.

[iii] Please note, these program bins don’t actually identify pots of money, they are organizational labels. Those few with only a handful of proposals and high success rates are mostly flukes of labeling. Some, like AToL are old codes that are being retired; others, like LTREB and LTER, are special programs that weren’t entertaining entirely new submissions but other sorts of awards like renewals, and workshops.

[iv] Dimensions might look relatively good, but you have to correct for collaborative projects in the official count here too. The rate as far as whole projects are concerned is ~10%.

[v] In 2013, DEB made 134 DDIG awards. If we never had this opportunity (and ignoring any resulting increases in student support requests), we could have funded approximately 4 additional regular grants, 1 per cluster, bringing the 2013 core award count from 121 to 125 and increasing the funding rate less than 1%.

Assessing the Value of the Doctoral Dissertation Improvement Grant

Caveat: This post is based on the research and analysis of Kara Shervanick, a 2013 Summer Student in DEB. She did valuable work but her time was relatively brief for this complex information gathering and analysis process. This work does provide some context for understanding DDIG program outcomes, however, we point out that the small sample size limits the power of these analyses.

See our other recent posts on the DDIG program here and here. Continue reading