Helping the Minoritized Achieve in Academic Science

Posts tagged ‘grant reviews’

Re-Evaluation

2015-06-23 12.31.18I have served on a lot of grant panels. In the last year, I served on a grant review panel – a small one for a small foundation that you probably didn’t personally qualify for. I was the only woman of three reviewers on the panel, and each grant had a number of ad hoc reviews provided by experts. For each grant, I submitted made a review, and it took a couple weeks to get them all done. Before I submitted my reviews, I re-evaluated all the proposals to make sure I wasn’t systematically biased against anyone. Because we are all gender biased and racist, I paid particular attention to checking myself on this. And, I ended up changing some of the scores based on the re-evaluations to level them out. Of course there are many reasons with scores could have been different, including if I was tired or hungry when I read the proposal, so I didn’t want those dumb reasons to affect the scores. This final step is one that most people don’t do because of time, but I wanted to do a good job. I did end up raising the scores of more women and of more people with foreign names. I felt like I did a good job, and was proud of my work.

And you know what? I am glad I did the re-evaluations. Here are the reasons:

  1. More women were in the upper half of proposals than would have been if I hadn’t re-evaluated them. Thus, we spoke about more women’s proposals in more detail.
  2. The two very nice gentlemen with whom I was serving were so unconsciously biased against women, it was effing ridiculous. Let me give you an example (the information has been changed to protect the innocent):

Grant 12345, a man:

Scores: Man1: 3, Man2: 3, WOS: 3.

 Comments from the dudes included things like, “Not exactly sure how this will be carried out. Needs more details on the experiments. I cannot tell if this will work.”

Grant 98765, a woman:

Scores: Man1: 3, Man2: 3, WOS: 5.

Comments from the dudes included things like, “Well-described methods, clear proposal, looks like PI will be able to secure federal funding.”

I was shocked. These two proposals had vastly different comments from the two dudes, yet, they gave them the exact same RANKING numbers! On two separate occasions, I convinced one or both of them to change their scores during the discussion. For this example, I actually flat out said, “Please look at the written comments for this person. What exactly is wrong with their proposal? You have nothing negative. So, why is your score the same as the last person who you did have negative comments about?” They could not deny that their rankings were illegitimate, and they changed them.

At another point in the discussion, I pointed out that the ad hoc reviews for a particular woman were biased. For them, this statement went too far. They did not respond well to that. They pointed to a woman ad hoc reviewer to prove to me that the ad hoc reviews were not biased. RED FLAG! If you cannot justify something, pointing to another biased review to justify yourself is not a scientifically good way to prove your point. I told them that it didn’t matter if the reviewer was a woman. Women are just as biased against other women as men. Being a woman does not protect or shield you from sexism. They were not convinced. In the end, I had to write the panel summary, and I had a very hard time. Why? Because they could not point to one thing that they came up with as a panel that was wrong with the proposal. This, to me, stinks of bias. You don’t think anything is wrong, but you just go along with what other people say? That is not the scientific way. All their issues were direct echoes of the ad hoc reviews, which I thought were biased. In fact, they only said positive things in their own personal opinions. Luckily, I did get that person to the funded category, because writing a, “Sorry you didn’t get $$, but we can’t figure out what is wrong,” panel summary is difficult and stupid. At least the letter, which doesn’t say anything is wrong, also doesn’t say, “And you don’t get money for not being wrong.”

There were several other times, I was looking at the comments and thinking, this rating doesn’t jive. More than once, dudes with many negatives were given the same score as women with only positives or far fewer negatives. Dudes just got a leg up. One woman was working with her former advisor still, and that got the old chestnut, “Is this person independent?” but the next dude who was working with their former advisor got, “This is a positive because they will be more likely to be successful working with this bigger group.” These were too far apart in the discussion to successfully combat with logic and reason as I did above, and it wouldn’t have made a difference, but that kind of stuff burns me up.

All in all, the women had to be way better than the men to be ranked equally by these guys, so my re-evaluating of the women only counteracted the lunacy. Also, I was the easy grader for most proposals. I was already seen as the person who gave the highest score (about 1 point higher out of 5 than the guy who gave the lowest scores) on all the proposals, so they didn’t suspect or know that I had re-evaluated and subsequently upgraded a lot of the women.

Based on this experience, I am going to call to all my WomenOfScience friends and male allies: Consider re-evaluating and seeking out your own biases against women and minorities when you review their manuscripts, proposals, or whatever. If you were already going to be nice, act like they walk on water. If you were going to be mean, be 20% nicer. It probably won’t move someone from the unfunded pile to the funded (it didn’t in my case), but it could move someone from the edge into funded, or give someone just a slightly nicer review – you can still be critical without being a total douche.

I will continue to re-evaluate at the end whenever I can, because I think it is the right thing to do. I wish more people would at least be mindful of their biases. To get funded today, you need a champion in the room. Each funded proposal has one. I vow to be the champion in the room for women and under-represented groups. That is what I did. I could have chosen to preferentially fund proposals about beavers (there were no beaver proposals) or a particular school, let’s say Ole’ Miss (there were no Ole’ Miss proposals in the panel), but I chose women. I will continue to choose women. Until there is actual equality.

What do you think? Comment or post here. To get an email every time I post, push the +Follow button.

Category:

Applying, Career, Service, StereotypeThreat, WomenRock

Tagged with:

Networking at Grant Panels

WomanNetworkNetworking is so very important!! I cannot stress this enough. This is true at all levels. At early levels (student), it helps you to establish connections and can even get you a job (see this post). Pretenure, it is essential to get the word out that you exist and are doing things that people should pay attention to. You gotta go to conferences (old post) and network on campus (recent post). When you are senior, lack of travel and often result in lack of recognition, and getting back out there can be essential to re-starting after a long absence due to childcare or other issue (see this awesome post).

When you are a professor, another important place to network is on grant panels. Serving on grant panels is so important for so many reasons:

  1. You get to read grants. Good grants, crap grants, many in between grants. When I read grants, I not only try to evaluate the science, but I also use the time to think about how best to write grants. Of course, you have to get rid of the grants afterward, but you can think and even write down what was good about the writing, the style, the format. All these things matter to writing a great grant that gets funded.
  2. You get to meet other scientists. On grant panels, you spend an intimate 1-4 days with a group of scientists talking about science that can be funded, using your expertise, learning new things you never knew before, and basically interacting. You are also together at meals where you spend time talking about your family, your pets, your house, and all the other lifestyle stuff. Scientists have similar lifestyles no matter if you are from California, Texas, or Michigan. This is the networking. This is the close kind of network that you often only find at very small meetings. Grant panels are the smallest of meetings.
  3. You get to meet program officers. In addition to working with other scientists who may or may not be in your field, you also get to work with the program officers who will presumably have the opportunity to fund your research. You can figure out what types of science they like to find and how they like to interact with scientists. Different program officers like to hear more about motivation or technical stuff or diversity impacts. Plus, if you are already at a funding agency, you might be able to visit other program officers while you are there.

What is a grant panel like? I have a lot more experience serving on NSF panels and foundation proposal review panels, so that is what I will describe. If you have information about NIH, DOD, DOE, or other, please comment here! At NSF you have to come prepared and be early. Most program officers want you to have all your evaluations uploaded over a day early, so they can prioritize the discussion list. Be prepared – it takes over an hour to review a single proposal and write a review, so make sure you start early enough.

At the panel. The program officer will start with a little background or information you need for the panel. Good ones will describe implicit bias and how it is important to be aware of biases, so that you can avoid them.

Reviewing. The panel will begin to review each grant. Some panels prioritize the grants so that the obvious ones (all highly rated or all low rated) are discussed first and taken care of. Sometimes the bottom ones are completely triaged – not discussed at all. Most program officers will try to keep you on track by giving you only 12-15 minutes to discuss the proposal. One person will be the “lead” discussant and describe the strengths and weaknesses of the proposal. The second and possibly third reviewers will describe and additional and not previously described issues. Typically, a third or fourth assigned reviewer will serve as the scribe who will record what is said at the panel to give some inside information about what was said in the room and write up the panel summary that also goes to the proposers.

Serving as a virtual panelist. In a recent panel, I served as a virtual panelist. In this, I used my computer camera to interact with the panel. Frankly, I didn’t like it. It was harder to interact and network with others. I felt like it was also more difficult to be convincing. Most of the other virtual panelists had cameras, but not everyone, so I couldn’t use facial cues to help me be more convincing. Also, I realize that I typically use these meetings for networking – specifically with the women scientists on the panel. I am not sure if I will be a virtual panelist again.

Anything else I missed? Post or comment here. To get an email every time I post, push the +Follow button.

Category:

Career, Networking, New Job, Research, Service, Uncategorized

Tagged with:

Tag Cloud