Tag Archives: reviewing

If you don’t have time to do it right, when will you have time to do it over?

One of my favorite expressions is “if you don’t have time to do it right, when will you have time to do it over?” I believe that the legendary UCLA basketball coach John Wooden was the first to use this expression. How does this apply to academic writing? After a little thought, I came up with these five examples of putting things off that would have been better accomplished had they been completed at the appropriate time:

John Wooden

John Wooden quote from http://www.brainyquote.com/

First, not doing a full outline before beginning to write a draft. Even in my senior honor’s seminar, open only to the best students in our program, many students give me a funny look when I ask them whether they do an outline before they began working on their papers. Okay, you say, those are undergraduates, what do you expect? However, when I ask the same question of graduate students and even faculty, many say they can’t be bothered, offering various excuses including “it takes too much time,” “I like to discover my central theme as I write,” and my favorite, “it hampers my creativity.” I liken this practice to hikers walking into one of our large national forests on a week long track without a map. What do we call such people? Lost. Somebody will eventually need to rescue them. In academic settings, the rescuers are often editors and reviewers.

Second, skipping a difficult section while writing a draft. Assuming that you prepared an outline, you have an end goal in mind and so the problem is just to execute. However, the only way to test whether the outline actually represents a coherent narrative for your story is to go through it from beginning to end, in order. The difficult bits that you skip over, assuming that they can be written later, might actually be the points where you eventually discover that you can’t get there from here. When writing an outline, it is fairly easy to convince yourself that, as seen from the mountaintop, there is a walkable trail from the park entrance to the creek. On the ground, however, the unbridgeable chasm that was concealed by the tall trees becomes readily apparent. It is much easier to do it right the first time than to walk back to the entrance and start over again.

Third, not recording the full reference for a book, article, or blog post when you first take notes on it. It is easy to convince yourself that you can always come back later and get the rest of the reference you need for the bibliography. Moreover, there’s a chance that you won’t actually use the material in your paper, and so why spend extra time writing down all that information when you’ll never need it. Indeed, why bother? The answer becomes painfully apparent when you discover the incomplete references on the morning you plan to submit the conference paper, to meet the announced deadline, and you find that the library server is down.

Fourth, not doing the descriptive statistics before beginning the multivariate analysis in a statistically based paper. I try to teach my students the relevance of this potential misstep when they bring me the first draft of their paper and I point out the implausibility of a coefficient or two. Could it really be true that people with college degrees earn less than those who dropped out of school? Quite likely, somewhere along the way, a coding error or data transformation mangled the true values. Carefully scanning means, standard deviations, skewness, and other basic properties of the data goes a long way toward reassuring me that you actually understand your data.

Fifth, sending out a paper for comments from your friends and colleagues before you have proofread and copyedited it. Nothing says “I don’t care about your time” more than sending a colleague a paper full of typos, misspellings, botched grammar, and other mistakes that could have easily been caught with an hour or so of careful reading. I suggest first running the paper through a standard spelling and grammar checker on your word processor, then printing the paper out and reading it line by line. To ensure perfection, you might try having a very patient and loyal friend read it aloud to you. Here’s your chance to discover the true meaning of a “strong tie.”

John Wooden was surely right: although each of these shortcuts will seem to save you time in the short run, the gains are purely illusory. You have merely embedded problems in your work that will come back to haunt you later, especially when they are discovered by others. Do yourself, your friends, and your reviewers a favor: take the time to do it right the first time.

Save

What To Do After the Reviews Arrive

Over the past decades, I have responded to more than 100 revise and resubmit requests from editors, served about 10 years as Associate Editor of the Administrative Science Quarterly, and reviewed hundreds of papers for dozens of journals. Closer to home, I’ve had the experience in the past year of responding to several tough R&R requests, and thus I decided to see whether I had learned enough to share some general tips with other authors. So, here are a few, with no claim to originality on any of them.

Ryōan-ji Temple

Ryōan-ji Temple, Kyoto, Japan. Only 14 of the 15 rocks are visible from all vantage points

First, just wait. After you finish reading the editor’s letter and the reviews, you might feel the need to “do something.” My advice? Don’t! Read the letters a couple of times to make sure you’ve noticed everything and haven’t spent more time on the negative then the positive reviews. Then, put them aside and just think about them for a day or two. Or longer.

Second, moderate your emotions. It is natural to feel annoyed, irritated, and even angry about a letter that points out the flaws in your work. However, letting your anger get the best of you will lead to unprofessional behavior which you will later regret. Given that the letter is a revise and resubmit request, there will be sentences in the reviews that say positive things about your work. Relish them and then move on. Similarly, there will be sentences in the reviews that lead you to question the intelligence and motives of the reviewers. Such thoughts are not helpful. They block rational thinking about the strategic course of action you need to take.

Third, thank the editor. The letter probably asks you to indicate to the Journal whether you plan to resubmit and if so, by what date. I never turn down such invitations. Even if the letter says something like “meeting the reviewers’ comments will require major changes in the manuscript,” the fact that the editor thinks you have a chance is reason for celebration. So, don’t complain in the message you send back to the Journal; just tell them that you’re grateful for the opportunity to revise the manuscript and you will be returning it within 30, 60, or 90 days, or whatever time frame it is that you’ve been given. Editing is mostly a thankless job — give the editor a break.

Fourth, do not send the manuscript unchanged to another journal! Occasionally I hear friends and colleagues tell me that the effort to meet the revision requests is not worth it and they’re just giving up and sending the paper to another journal. That’s a big mistake. Why? First, for most journals, far less than half of the authors submitting papers are given opportunities for revision. Somebody likes your work. Second, the odds of an acceptance skyrocket for most journals, once the paper has been given an R&R. For some journals, the odds approach 50% that a revision will be accepted. Whatever the number, if you cared enough about the Journal’s reputation to submit to it in the first place, you’ve now got a much better opportunity to publish in it and so you shouldn’t turn it down. Third, and perhaps most important, it is quite likely that at least one of the reviewers who told the editor that the paper needed work will be a reviewer for a subsequent journal to which you send the paper. Nothing angers a reviewer more than to learn that all the work put into a review has been ignored by an author who has chosen to send the unchanged paper elsewhere. Sending the paper elsewhere without revising it is likely to elicit a strong rejection or at least one really strong negative review!

Fifth, consider the possibility that the reviewers were right. Lots of research in cognitive neuroscience tells us that people consistently overestimate the value of something they have produced, as well as being wildly overconfident that their work is above average. Don’t be one of those people. Take the reviews to a colleague and, without biasing their response with a negative cue, ask them to tell you what they think of the reviews. You will be surprised at the response. Although your colleagues can put themselves in your shoes as a spurned author, they probably have had much more experience as a reviewer than as an author (judging by the average colleague’s CV), and so their sympathies are quite likely to lie with the reviewers. Listen to what they have to tell you.

Sixth, make a plan. Every revise and resubmit editor’s letter asks an author to not only revise the manuscript but to include a document that indicates how each of the comments made by the reviewers was dealt with. You can get a head start on that document by using the reviews to create a plan for revision. First, fit the plan to the comments. If the points in the reviews aren’t already numbered, number them. Keep track of which reviewer said what by giving each reviewer a letter, e.g. A, B, and C. Second, indicate whether you accept the criticism and can do something about it (adding literature, clarifying language, conducting new analyses, and so forth) or you can’t. If you can’t do something about it, indicate why. Third, clearly indicate which of the comments will require moving words around (theories, concepts, models, etc.) versus which will require new analyses. Externalizing your thoughts in this way will show you whether you truly understand what the editor is asking you to do. (You can write back and asked for clarification of murky points.) It also gives you a sense of the magnitude of the effort required to meet the comments. This document will be both a plan for revising the manuscript and the template for the letter you will write to the editor.

Seventh, don’t lengthen the text. Almost all papers that are submitted are close to or over the ideal word limit set by a journal. Although you may claim that you can’t deal with the reviewers’ comments without adding more words to the text, that is surely not true. Unless you had hired a professional editor before submitting the paper, there are many places in the text where you will find redundancies, unnecessary digressions, and so forth. Use the revision opportunity to shorten the text, or at least to ensure that it is no longer than the original submitted.

Eighth, be generous. Avoid gratuitous insults to the editor or reviewers in the letter you of explanation write. Although it is tempting to claim to an editor that the reviewers misunderstood your manuscript because they’re out of touch with the literature or have inferior reading skills, you’ll gain no profit by doing so. Avoid effusive praise, but do thank the reviewers when they genuinely pointed out something that you had missed. In simple declarative prose, explain to the editor and the reviewers how you understood each of their points, how you responded to it, or why you were not able to.

If you follow these tips, I cannot guarantee you that your resubmitted manuscript will be accepted. However, I can guarantee that your professional reputation will emerge intact and you’ll live to fight another day. You will also feel better about the process.

Save

Use the norm of reciprocity to get constructive feedback on your work

In popular fiction, authors are often portrayed as isolated and tortured souls, locked away in a garret apartment or in a cabin in the forest, producing their great works without benefit of human companionship. In reality, writing is an extremely social activity, highly dependent upon an individual’s network of family and friends. Peer networks play in a particularly important role in moving writing from solipsistic doodling to prose that others want to read. Let me suggest one way in which social relationships are critical: finding people willing to offer critically constructive feedback on the work.

Boston MFA

MFA Boston

When your draft is completed, how will you know what reception it will receive from the intended readers? When I talk to academic writers about this question, I point out that the most risky action an author can take is to submit to a journal a paper that no one else has yet read. Although it seems incredibly shortsighted, I often talk to people who’ve done exactly that – – they claim that they really couldn’t find anybody they thought would be a good reviewer. Thus, to get feedback on their work, they plunged ahead and sent it out for review.

Why is this risky? For most academic journals, the best outcome of initial review is an invitation to revise and resubmit the paper. Thus, the initial submission is not an attempt to get the paper published as is, but rather to convince the reviewers, and hence the editor, that the paper has enough merit to warrant the journal spending an additional round of reviewing on it. If you submit a paper that has not been evaluated by anyone yet, it is quite likely that it contains so many serious problems as to preclude the likelihood that reviewers and editors will see it as worthy of a revision. So, at the very least, it makes sense to seek feedback from others that will enable you to fix major problems with the paper before submitting it, thus increasing the odds that you will get a chance from the journal to revise the paper.

But who should you turn to for such feedback? I recommend thinking first of strong ties and then expanding beyond them in selecting people whom you will ask for feedback. First, you can turn to your local circle of colleagues and mentors. You might join a writing group with others who seek feedback on their work. With others writing about similar topics, you could organize a workshop at which you present your papers. Second, you can locate potential reviewers by attending conference sessions at which people in your field are presenting their own work. Third, if there are several authors whom you relied heavily upon in your literature review, they are likely candidates for providing feedback.

When I make such suggestions, especially to graduate students and junior faculty, they almost immediately object. Why would these people be interested in helping you out? Providing feedback is a lot of work and it seems unlikely that people you hardly know would voluntarily give up their precious time to spend it on your draft. Even close colleagues and mentors are mentioned as people evidently disinterested in providing timely feedback. (That could be a topic for another blog post!)

Anthropologists and sociologists have given us the answer to why we might expect these people to help: the norm of reciprocity. How does it work? In just about every culture where the phenomenon has been studied, we observe that when somebody does a favor for someone else, the person receiving the favor incurs an obligation to return it. You can use this norm to your advantage, but you must be strategic about it.

Locally, ask a fellow graduate student or junior faculty member if they are working on any projects on which they would like feedback. Do they have any working drafts they can share with you? At a conference session, stick around after the presentations and speak with the scholars from whom you’d like feedback. Again, don’t tell them that you are seeking feedback on your own work, but rather ask them if the paper they presented has been published, and if not, if they would like feedback on it? For scholars you know only through their journal publications, write and tell them that you find their work very interesting and that you are interested in keeping up with cutting edge findings in your field. Ask them if they can send you their working papers and if they’d be interested in receiving feedback.

If the targets of your solicitation respond positively, take the next step. Ask them to send you a digital copy of their work and ask what specific issues they are concerned about; for example, adequate recognition of prior work, arguments supporting proffered hypotheses, and clarity in interpretations of findings. Find out if they are working to any deadlines and in particular, if the paper has already been submitted for review. It’s important to ask about the status of a paper, as there is little point in sending people comments on a paper that is currently in the review process. Whatever you have to say will be overridden by the editor’s requests of the author.

Now we come to a critical component of my recommendation: in your comments, you need to show the person what kind of feedback you would like for your own work. In doing so, you prepare the groundwork for a working relationship with a constructive critic who can give you the help you need.

How do you teach someone else what kind of feedback you would like? Remembering the norm of reciprocity, you need to show them what a competent and constructive review looks like by doing one yourself. Provide higher order rather than lower order feedback. Critique their ideas and concepts, not their spelling and grammar. For example, point out problems with the flow of the argument or with lack of a connection between the literature cited and the inferences drawn.

Technology is our friend in this case. Microsoft Word makes available two excellent writing tools — Insert Comments and Track Changes — one of which you can use in providing feedback and one that you should avoid.

Do use insert comments by marking blocks of text and then writing explanations to the author for why you have marked them, as I noted above. Do not use track changes by inserting your own words into the author’s text. “Track changes” is for co-authors, not commentators. It is not your job to point out spelling mistakes or errors in grammar, unless someone specifically asks you to do so, and even then, you should resist. In short, leave the copy editing to the author or somebody who is paid to do such work. Your job, as a reviewer of their work, is to provide feedback on the substance of their argument. So, aim high.

The norm of reciprocity is so powerful that it will probably not be necessary for you to enter into an agreement, a priori, with the person whose work you are citing. Instead, by following the above steps, you will have prepared the other person for a subsequent request from you, asking them if they’d be so kind as to provide similar feedback for a draft paper of yours. Of course, there is always a risk that you will be turned down, which is why I recommend only commenting on the work of others whose work you genuinely find interesting and is in fields which you know enough to provide useful feedback.

Start small, with your local circle, and then work your way out to others you don’t know personally. You will be astonished at how grateful even well-known authors are to receive constructive feedback on their work. I have met a lot of good friends this way.

In my own work, I am delighted if I can find five or six people to provide the kind of feedback I’ve mentioned above – – feedback on the substance of my arguments. More than that and I’m ecstatic! This past month, I sent a paper to three friends overseas with whom I previously talked about the topic, and I sent it to three other scholars whose work I admired but whom I never met face-to-face. In their cases, I made clear to them why I was seeking their help and advice on whether I had used their work properly. I sent it to several people with a request that they only look at one particular facet of the paper, and ignore the rest, unless they had the time to do more. As a consequence of these solicitations, I believe the paper improved enormously from the early drafts to the latest drafts, and now I am ready to send it on to the formal reviewing process.

Journal reviewers will be anonymous and brutally honest! Better to learn all you can before the paper reaches their desks

Journal submissions: Playing up (or down) to the competition

Every fall I look forward to the opening of the college sports season: football, soccer, field hockey, volleyball, and so forth. In particular, I enjoy the discussions in the sports press about the choices athletic directors and coaches have made in setting up their schedule of games. Unlike the professional sports leagues, where scheduling is taken care of by the league office and considerations of equity and robust competition are explicitly taken into account, college schedules are under no such strictures.

Carolina soccer player

photo credit: IMG_9551 via photopin (license)

In scheduling games, college athletic directors face two stark choices. They can make their teams look good by padding their records with early games against lesser opponents, thus ensuring at least a winning record for the early part of the season. Or, they can put their teams through a trial-by-fire by scheduling tough opponents. Why wouldn’t athletic directors choose the easier path, giving teams an early-season break and setting them up with a winning record?

As sports commentators are quick to point out, the danger of taking the easy road early on is that playing against lesser competition lulls teams into a false sense of confidence. Running up the score against weaker teams covers up mistakes and gaps in preparation, such as athletes’ lack of endurance. Players who look like All-Americans against East Overshoe Tech become sloppy, take plays off, and are overwhelmed when they come up against teams in their own league. No matter how much the coaches tell them that the early-season games against weaker opponents are no gauge of their true strength, players may start believing their own press clippings.

The other danger of scheduling easy games early in the season comes back to haunt teams at the end of the season, in those sports where teams are invited to championship competition based on the strength of their schedule. Committees deciding on which teams to include in the playoffs, such as in women’s soccer or men’s basketball, can easily spot the flaws in a winning record based on playing weak competition. Florida State’s overwhelming early season win against the Texas State Bobcats will be heavily discounted!

Athletic directors with their eyes on the future are advised to schedule at least a few tough opponents early in the hope that difficult competition will pay off and prepare the team for its league schedule against stronger opponents. Such planning may also ease a team’s path into the postseason.

I’m reminded of this scheduling dilemma every time graduate students and junior faculty come to me, seeking advice on where to send their papers. As I see it, academics just beginning their careers face the same conundrum: do they aim for the top journals in the field or do they try for the easier route, in journals with weaker review boards and softer selection criteria? Why not try to pad your resume by aiming for the Journal of Lost Causes, which you know accepts almost half the papers it receives?

My answer to the authors seeking advice is the same that savvy athletic directors offer to desperate coaches: if you began by playing down to the weakest level of competition you can find, you set in motion a process that becomes hard to reverse, after a few iterations. Getting into a top journal is hard work, requiring sophisticated literature reviews, excellent research designs, and state-of-the-art analysis, regardless of the kind of data collected. If the paper is a theoretical/conceptual contribution, rigorous analytical thinking will be required, as well as lots of revisions before the paper is submitted.

I tell students that even if a paper is ultimately rejected at a top journal, the experience gained in the process is priceless. Most top academic journals use what they call a “developmental” review process, a much kinder and gentler process than authors faced three or four decades ago. (I hasten to add that I discourage “frivolous” submissions which just clog up the review process – – a senior colleague can quickly tell a junior scholar whether a paper is worth submitting to a top journal.) As a colleague reminded me, you must be prepared to invest anywhere from 2 to 6 months, on average, with every submission. That’s another reason to “start early.

Recruitment and promotion committees are seldom fooled by resumes padded with acceptances at weaker journals. So, I would turn to the weaker journals only after I had tried the top journals.

Although the odds of acceptance are low, gaining experience with the practices necessary to compete at the highest levels pays off in the long run.