Helping the Minoritized Achieve in Academic Science

Posts tagged ‘Manuscript’

PeerReviewZilla

PeerReviewZillaThere was a recent funny article on “How to be the Perfect Mother” from Huffington Post that was a hilarious look at how society tells us conflicting information about how we should act as mothers. You should go look at it if you are a mother, know a mother, or have a mother. Just go see it.

This article, combined with two recent manuscript reviews coming back, got me thinking about how reviewers also often write conflicting advice for your manuscripts. So, I decided to write a satirical version of a manuscript review as an example.

***Note: any resemblance to reviews you may have received or written are purely coincidental.

Enjoy!

We have read and reviewed the manuscript, “This Science Thing is Important for this Other Thing” by Prof.Science. This manuscript investigates the ScienceThing and its interactions with OtherThing, a very important and understudied topic. This group performed many new experiments that had never been done before and had 6 figures each with A-J panels. Their work was executed well and revealed new information about the interactions of ScienceThing with OtherThing that we never knew before. Their clearly written manuscript had a simulation that modeled the results and showed similar trends suggesting a mechanism.

 

Major Concerns:

In performing these experiments, they used well-tested experimental methods along with specific tests to control for errors. They have used these methods to test for effects of ScienceThing on OtherThing and have quantified the effects. Since these methods are well-tested and accepted in the field, they are not novel. We want only novel experiments even if we cannot interpret the results we get from them. Thus, we suggest that the authors perform all new experiments. Further, did the authors investigate how ScienceThing affected OtherThingII? Only one paper on OtherThingII exists, from the OldFart Group, but it is clearly more important than OtherThing, and it should be explored even though almost no reagents exist for OtherThingII. Unless OtherThingII is also investigated, I do not think this paper is very worthwhile.

The authors display histograms of their work and how ScienceThing affects the OtherThing. It is important to be quantitative and have numerical data. For each histogram, they fit to a Gaussian and report the R-squared value of the fit to the data. They use these fits to discuss the results. Why do they do this? Why not use a simple p-value to the data? Isn’t a student’s t-test done on everything? It is clear that the two distributions do not overlap, so they should report the p-value.

The authors used a toy model to show that the ScienceThing behavior that they see could be due to a minimal number of simple rules. Being quantitative and having models is important. We want more quantitative work and models in this field of science. The simulation has the same trends as the experimental data, but it does not exactly match the data, so the model must be worthless. Why did these authors have a model? They are not theorists or modelers; they are experimentalists. They should remove the model, it detracts from the data.

Without the model, the authors do not have a mechanism. We want all science to be mechanistic. It is not good enough to simply observe something and report what happens. For instance, although their toy model uses 3 simple rules and has the same general trends as the data, they cannot rule out a model with 10 complicated rules. Thus, they have not revealed the mechanism behind the results they see, and thus the impact of the work is lower in my opinion. Until their work becomes more mechanistic, their results are purely qualitative, and the work is not work publishing.

 

Other Issues:

There are a number of sp errs in this manuscript. Don’t they care how they present thmseves? Its like thei didn’t even porrof read before they sent it out. They need to really fix this. There are way too many issues for me to helpfully point out.

They are missing a number of very important citations particularly from the OldFart group, “Science Stuff: A novel Regulator of Nothing,” JSS 1979; “Science Stuff Moves Science Thing,” JSS 1998; and “Science Stuff to Science Thing,” AJSS 2000. These important references about how ScienceStuff is connected to ScienceThing are important and should be added.

Their experimental methods are not good. They didn’t even present them! I suppose they could be in the supplement, but I didn’t read it, so I wouldn’t know. Even if they are in the supplement, they need to have them in the main text. Maybe, once they take out the model, they will have room in this 5-page paper to have detailed methods.

 

In conclusion, after having read this paper, I feel that these results were obvious and could have been guessed from deductive reasoning. Thus, the experiments were not necessary and the results are not novel. Further, to make the results important and novel, the authors would need to perform a number of extra experiments that were not in the original 60 plots presented, and they would need a mechanism, which they have not proven. Overall, it is clear that this study has no value and, thus, I recommend that this paper be rejected.

Anything to add? Post or comment here. Maybe we can add more examples? If you want to get an email every time I post, push the +Follow button.

Category:

Research

Tagged with:

On the Road Again

Travel-smallI am wrapping up the end of a couple weeks of travel. At first, it was nice to have a break from my gigantic class and my crazy household and the snow snow snow, but I am ready to go back home and get into a comfortable groove again. I am sure my HusbandOfScience will also be happy to have my help around the house again, too.

HusbandOfScience and I seem to do a lot of traveling. This semester, I am traveling for 3 conferences and 3 seminars. That doesn’t really seem so bad, unless you add in the travel of HusbandOfScience. If we each travel once per month, than someone is traveling every other week!

I think I should start phasing out most seminars, but people come back and ask every semester. You feel flattered and you feel guilty for saying no. The requesters come back and ask you again so nicely and politely.  So you make a date for a year in advance hoping that FutureYou will be in a better position than PresentYou as far as traveling. But FutureYou has always been in a worse position because PresentYou is an a-hole who keeps schluffing her travel onto FutureYou. This is all screwed up even more because some fancy, exciting invitations might come up on shorter notice than one year causing all wave function to collapse into a coherent travel particle.

But, here is a problem with phasing out going to give seminars – we get evaluated every year on our research accomplishments that help to determine the measely 0.5-0.7% raise you might get. People who don’t travel are dinged because they must not be important enough. Seminar invitations count toward your visibility as an academic, so you don’t want to ignore them completely. So, what is a happy medium? How many seminars is enough to make sure you are getting out your research message, but no so much that you drive your wonderful SignificantOther bonkers because you are never there? And is it really important to travel at all?

Some of my WomanOfScience friends who have new babies are having trouble getting back into the groove of travel. They are saying “no” and feeling guilty because it is for family reasons. Again, I ask if this is hurting their career? It seems to me that if they get out a reasonable number of scientific papers and other written works each year (say, 2-5?) then their scientific research “cred” is really not in jeopardy. I think not getting manuscripts submitted and published is a more negative issue for your career than the number of talks you give. So, I don’t think these MothersOfScience should worry about getting back to the swing of travel so fast, if they want to take a few years to settle into motherhood before hitting the road again. For when you are ready to get back, I have a lovely post contributed by one of my mentors and AllAroundWomanOfScience about this topic called, “Back in the Saddle” you might want to check out.

One thing I do want to say is that I get a lot more manuscript writing and submitting while away traveling and on the road (actually mostly on the airplane) than I do at home. So, for me, keeping up that paper count may depend on my seminar schedule being full.

What do you think? How essential is travel? Does it matter more before or after tenure? Comment or post. Push the +Follow button to get an email every time I post.

Category:

Career, Networking, Research, TimeManagement

Tagged with:

Writing a Draft Manuscript

typewriterI have been working on a manuscript about some pretty nifty science. I have been working on it for some time, and I finally just submitted it last week. As part of the “how to write” series, I thought I would give my personal process for how to write a manuscript. I am sure there are as many ways to write a manuscript as there are manuscripts that have been written and published. This is my way, and it also how I instruct to my students to help them get over the hurtles of writing.

1. Make figures. Science is all about the data. The data is the story, so the first thing I do is make figures. This can be pretty challenging in and of itself. When working with students, I often have them make the figures – or at least the first draft of the figures. They often don’t know exactly what to do – even if they have presented some nice plots and graphs in group meetings. I usually sketch out the figures with them on the white board before I set them to work getting the data into figure form. Also, I try to get the students to think about the figures as they are taking the data. I usually sketch out plots when we discuss the experiments. If you are already thinking about your data in figure format as you take it, making the figures becomes much easier.

If you figures are missing data, if often become clear at this step. It is obvious that something – some data, some plot – is missing in the figures. We don’t go onto the next step until we have all the data in figure form.

Just FYI, I use Illustrator to make my figures. I hate Powerpoint. It does not allow very much control or good resolution. I know some people love Powerpoint for figures, but I think it is clumsy compared to Illustrator.

2. Long figure captions. After the figures are all made – with ALL the data – we write long figure captions. The figure captions include how we did the experiments, what the figure results show, and what the results mean. The point of a long figure caption is to have an easy way to move from the data to the rest of the manuscript. The “how we did it” becomes part of the materials and methods. The “what the figure results show” become the results section. The “what the results mean” becomes part of the discussion section. The point of the long figure captions is just to help students get over their fear of writing. It is much easier to write while you have a figure to look at and to write about.

3. Methods. I find that the methods are often one of the easier parts to write – especially for students. This is where they get to say what they did. One issue I find, especially when working with undergraduate authors, is that this section can be harder to write than it seems. Yes, it is just what you did, but it has a specific style. For instance, it is not helpful to describe the volumes used in your assays – it is important to describe the concentrations of the reagents. The experiment *should* work the same if you mix up 1 ml or 100 ml, as long as the concentrations are the same.  I have a few favorite papers of my own or from others that I think have particularly good methods sections. I often give these to students to read to help them get the style and tone in their heads before they write.

4. Results. I, personally, find the results easiest to write – especially if you have the figures made up already. Often, each figure is a section of the paper’s results section. Again, this is where one can just describe what the data says. I write as stream of conciousness from the images and describe the data. Doing it this way tends to make the paper a bit long-winded, and it needs a lot of editing, but it is better than to stare at a blank page. It is always easier to cut and edit than to come up with perfect words the first time. A lot of times, I end up writing implications or how the data relates to other work in this section. These ideas are really better suited for the discussion, but it is easier to write them here and move them, if they belong somewhere else.

5. Discussion. After writing the results, I write a discussion section. Sometimes, the results and discussion sections are melded together, and for each figure, I write results and then discussion/implications. If they are separate, I still often write discussion-like ideas in the results and just cut and paste them into the discussion afterwards. These are the starting point, and I expand the discussion from there. At this point, I often have to do additional reading of the literature in order to put my work into context. I add in the references as I think of them with some sort of demarkation that my citation software can recognize. I use {curly brackets} and author name and year. If I know I need a citation as I am writing, I might highlight {cite}, so I can go back and find it later.

6. Introduction. I write the introduction after the discussion. How else will I know what topics I need to introduce and background literature to ground the work until I know the results and implications. I often need to do more reading at this point to make sure I have good and correct references. I insert the citations as I describe above in the text as they come up in the introduction. I often have to read other introductions, especially if I am stuck for words. By this point, I have a good idea of where I am sending the manuscript, so I read introductions from published papers in the same journal to get the style in my head.

7. Abstract. After the paper is pretty much written, I then write the abstract. The abstract is difficult because you need to be brief, for some journals less than 150 words! But, you have to get all the information of why it is important, what you did, and why is matters to the field. Again, I often write something much longer and have to cut cut cut. I think of an abstract as an inverted pyramid – start broad and focus down. You also want to hook the reader – tell them why this work is important early on so they want to read the paper.

8. Other stuff. This stuff I kind of do when the whim hits me. If I am having a hard time writing, then I might write the Acknowledgements section. This takes looking up all the funding agencies and getting the numbers. I had it on a sticky note (an electronic one on computer – not an actual sticky note), but I started being more organized with it all in an excel spreadsheet. Something like the title might change as the paper is being written and different important parts come into view. I like it to represent the results or the implications – it depends on the journal. The authors and author order are usually obvious and depend on the amount of effort, type of person, and the field you are in.

9. Cover letter. The cover letter. I think this is very important, and I think I have been doing it wrong until recently. Once you know the story, the cover letter is the place where you sell it to the editor. The higher “impact” the journal, the more important the cover letter is. This is especially true if the scientific editors are professional science editors and not Principle Investigators acting as editors. You have to educate the editor about how important your field is and why your work is an important piece of the puzzle that was missing until now. You have to discuss what your results are in laymen’s terms – like when you write a proposal for generally educated scientists who might not be right in your field. It must be clear and convincing.  I hope my recent cover letter works. It is far superior to any cover letter I have even submitted before now.

10. References. This is the absolutely last thing I do. Many of the references are being inserted in {curly brackets} along the way, but some will still be missing. They will also be missing from my reference software, so I have to spend some time getting all the citations into that, too. I personally do not use EndNote, but many people do. I use a program called Sente. Either way, I have to get the references into this software system to get inserted at the end. Once my references are inserted in the journal’s style, I cannot modify them or add more, so this is why I wait until the very very last to insert them – otherwise, I am wasting my time.

Before putting in final references, I have my entire lab read and edit the paper. They edit for typos, grammar, spelling, and tell me if things are not clear or confusing.

After all these steps, then I spend a lot of time submitting. The submission processes online are actually easier now than when I was a graduate student, but they still take several hours of inputting names by hand and getting the figures, tables, and writing all uploaded. So, even when you are done with the manuscript, you aren’t quite done.

And there it is, now you have a submitted manuscript! Easy, right? No, but it works, and it is relatively painless – at least for me. So, what are your tricks for writing manuscripts? Comment or post here.

Category:

Career, Research, TimeManagement

Tagged with:

Sticking Up For Yourself

FEMEN_Calls_for_Sex-BoycottAs I have discussed in prior posts, academic science is full of criticism. Most of the time, the criticism is important and helps you to make your science better. Sometimes, not so much. I still contend that women take much more criticism than their equally-qualified (or less-qualified) male contemporaries, but I haven’t seen any specific studies on it. Have you?

Either way, part of what you have to do in academic science is to stick up for yourself. Whether criticism is constructive or mean, we all need to learn to stick up for ourselves against it. Here are a few places where you have to stick up for yourself and some advice on how to do it. Disclaimer: I am not perfect at this and would love comments or posts with more information from others.

Response to Reviewers of Manuscripts: Reviews on papers are a good place to start with this topic. You will get criticisms in reviews – even National Academy members have criticisms and must respond to them. Here is what I do to respond to reviewers. I find this method both cathartic and productive.

Step 1, I print the reviews. I have a harder time reading things on the screen – especially critiques. As I read the reviews, I write whatever the first response to come to my head is. Sometimes it is easy, like, “Cite this paper,” or, “Emphasize this more.” Sometimes they are more elaborate like, “We can do these experiments: one, two, three, and they will probably take 2 months.” Sometimes my responses are just plain rude, such as, “Did this reviewer even read the manuscript?” or even that old chestnut, “F*ck You!” Yes, I write these all on the paper – dirty words and all. They are my first responses – whatever pops into my head – and they are very useful.

Step 2, I meet with the research team, and we all go over the responses and what we need to do to fix up the manuscript. This is usually major issues, like new experiments. I have others give me their first responses, too, if they want to. We air out everything and figure out how to address all the critiques.

Step 3, I identify the locations in the paper that require correcting and updating and set to do it as soon as possible. This is revising the manuscript. This is the obvious step.

Step 4, I write a response to reviewers. The actual response is very different from the responses in Step 1. For each criticism, I write a point for point response. Sometimes it is easy, like, “We cited this paper to address this concern,” or “We re-wrote this section to emphasize this point and clarify our reasoning.” The hardest responses are ones where you need to rebut the reviewer. None of us is perfect, and sometimes we need to let the reviewer know that their point was actually not correct. No big deal, right? It happens. But, reviewers are in a power position over the authors, and you don’t want to rub them the wrong way. When I have to rebut the reviewer, I make my case very strong with a lot of evidence and references. I have even been known to consult other experts of the field to have conversations about these issues. This is important if the reasoning behind something is unpublished “common knowledge” of the field. Every field has this common knowledge, but most of the time it isn’t published nowhere you can point to easily. If the reviewer is not in the exact same field as you, they might not be aware, so you have to inform them. I don’t just ask them to take my word for it, but I reference real conversations with other experts of the field who I asked if I could name in the response. No other expert has ever said no. I have talked to others, and I think this approach is unusual. No editor or reviewer has ever said it was wrong, so I will keep doing it, if I need to.

Critiques on Grant Proposals: These are more difficult to stick up for yourself because you don’t get to respond to reviewers. But, if you are resubmitting the proposal, you need to go through the reviews and respond to them implicitly within the new proposal. I basically go through the same method of printing the reviews and responding. I also have had co-PIs do the same exercise in a multi-PI proposal, and many found it fun and helpful and emotionally de-stressing. Then, I identify the areas of the proposal that require the changes and re-write based on the reviews.

On my first panel where I served as a reviewer, one of the proposals I was reviewing actually quoted their prior reviews and directly responded in their proposal.  At the agency I was reviewing for, the panel changes every time, so the new reviewers (me) would have no idea that they were responding to critiques except by this method. It was very effective. I haven’t employed this myself, but I do respond to the critiques of prior reviewers.

Even if I am not resubmitting because they do not take resubmissions or the research fit is not right for that program, I still read and try to understand the critiques. Very rarely, I get a really rough review. My first year as an AssistantProfessor, I wrote a grant to a foundation for a young investigator award. One of the reviewers said, “It remains to be seen if [WomanOfScience] can even successfully start a research program.” Ouch! It was really harsh, but it was reality. It was my first year, and it did “remain to be seen,” but it wasn’t for the reviewer to judge in the proposal review. That is what the tenure evaluation is meant to judge. In order to stick up for myself, I called other near peers and had a conversation about it. Swapping stories of jerky reviewers on proposals always makes me feel better before I start the task of trying to make productive lemonade out of their nasty lemons. It also helps me to decide if I should try again at a certain funding agency with the certain idea, or if it is time to move on.

Letter Writers for Promotion: We hope that all our promotions will be wildly successful, and we will all sail over the bar to get tenure, become full, and all other evaluations. But, this really just isn’t the case all the time. These critiques can kill a career, so sticking up for yourself is imperative. Once you go up for tenure, that might be all she wrote, although I know several people who have come back from failing tenure and went back on the market after unsuccessful tenure bids, so it can be done. Most jobs have a “mini-tenure” review process, and this is the time to identify issues and address them. Much like the case against the reviewers of papers, you need to understand the issues, determine how best to address them, and build a case in your favor to make sure you do better at the next promotion evaluation. I suggest doing basically the same process as above.

Step 1 helps you to identify what went wrong and come up with gut instincts as to how to correct them. Step 2 should be done with a team – hopefully you have mentors who can help you. They do not have to be in your department or even at your college. In fact it might be better if they are away from you. It will help you to get an outside prospective from someone who will not also suffer if you do not get tenure and promotion. Your university invested heavily in you, and they want you to succeed. But that also puts pressure on them, and they can lose effectiveness as mentors under this pressure. Going to outside mentors takes bravery. You have to expose your soft under belly to your mentors to allow them to help you. You will have to be vulnerable. So, these are people you will have to trust to have your best interest at heart. Then you will have to go through step 3 – and make the changes and implement the solutions your team have come up with. There isn’t really an mandatory equivalent to step 4 – writing the response – but it may be important to do just that if you don’t get tenure. Having a clear, thoughtful response to critiques is important if you want to challenge the decision or to try for new positions at different institutions.

Other Places: Sometimes we experience critiques in other places such as in email exchanges, in person at faculty or committee meetings, when getting critiques on a proposal or a manuscript from colleagues in person, or even in blog posts 😉 My advice, which I certainly could use a reminder of frequently, is to try not to react immediately. I advise that your initial reactions to bad news, critiques, or even personal attacks should be private. This is often hard to do, but good off-the-cuff reactions take practice to get them right. Public reactions should be carefully planned, if possible. I feel like this is where I fail most. I am obviously much better if I am rested and not hormonal, similar to the uncomfortable conversations post. But, considering that these things can strike without warning, it is hard to always be in the perfect condition to absorb negative comments.

Sometimes the best course is to ignore things. Ignoring something might go against the title of this post which suggests that you should defend yourself, but sometimes it is the best thing. For instance, I recently got an email from a colleague who scolded me because I used a public space for a laboratory end-of-semester party. The public space is adjacent to office space where his students sit. When we went to have the party (all 13 of us), there were only two students (of an office that seats 8) in the office space. Both of these students were  wearing headphones, and it was the second to last day of exam period at 4pm. I didn’t think I was bothering them, and no one said anything at the time when I could have changed anything. Yet, in an email later, I was scolded and told that I needed to ask permission of the students in the office to have my event. I should also mention that I have had these events there previously without scolding, and no one ever said anything before. My first reaction was to write back and defend my actions, but I decided against it. I just ignored the email. I figured if it was really a big deal, he could talk to me in person. I also suspected he threw out the email without much thought. I saw him multiple times the next day without a mention of it from him and with his usual nice self. I think ignoring the situation was the right thing to do there. Sometimes ignoring something is a statement in and of itself. You are saying, “This is not worth my time.”

So, what about you? Do you have any helpful hints on how to respond to criticism. To stick up for yourself in a good way? Any comments or posts would be greatly appreciated! You can receive notice when a new post appears by pushing the +Follow button.

Category:

Career, Self-Promotion, StereotypeThreat

Tagged with:

Tag Cloud