We invited Design and Culture’s Principal Reviews Editor, Maggie Taft, to respond to questions about different aspects of journal publishing. This is the first of a series from Taft, an independent scholar and Director of Writing Space, a community-based writing center for artists and designers.

Question: How do design research journals evaluate the quality of submitted articles?

Answer:

I think of this question as a two-parter: how do journals go through the process of evaluating submissions and what criteria do they commonly use at different stages in that process?

After an author submits a journal article, it moves through many people’s hands before the author hears back with the journal’s decision. Typically, a Managing Editor, who’s in charge of the journal’s day-to-day coordination and administrative work, is the first to review. The Managing Editor confirms that the article abides by the submission standards and style guidelines evaluating, for instance, that the piece meets the journal’s length specifications and that the topic corresponds with the journal’s. (You might be surprised how often that’s not the case!) Next, the journal’s Editor-in-Chief—that’s the person in charge of the journal’s vision—reviews the piece to confirm that it aligns with the journal’s mission and that, broadly speaking, it follows academic publishing standards. For instance, does it include research with citations? If the Managing Editor and the Editor-in-Chief think it looks good, then they pass it on to one of the journal’s Associate Editors. Associate Editors, sometimes called Editorial Board Members, are specialists in various fields who collectively contribute to a journal’s scope in expertise, perspective, and geographic reach. They oversee the next phase of the review process, during which an article receives more thorough attention and feedback from peer reviewers. Typically, peer reviewers do not hold positions on the journal; you won’t find their names on the masthead. Rather, they’re specialists in the article’s subject area and thus are particularly well-positioned to evaluate its argument, methodology, and contribution to the field.

Different journals configure the peer review process differently, and most use either a double-blind or single-blind review process. Double-blind peer review, which is typically the most highly regarded within academia, means that the reviewers will not know who the author is and the author will not know who the peer reviewers are. Single-blind peer review means that the author does not know who the reviewer is but the reviewer does know who the author is.

When a journal sends out an article for peer review, reviewers are typically asked to evaluate a piece using a questionnaire. Every journal has its own version of this questionnaire—unfortunately, they don’t often make them public—but most ask reviewers to assess some or all of the following features—

Scope: How does the article fit within the journal’s purview?

Impact: What is the significance of this article for its field of study? How does it contribute to research on its topic? How and in what ways does it build on existing literature?

Audience: Who is this article’s primary audience? (For instance, practitioners? Design historians? Design studies researchers?)

Argumentation: What is the article’s central claim? Is it clearly presented and well-supported with adequate evidence? Does any part of the paper need to be expanded to make the argument convincing? Is the approach methodologically sound and appropriate for the subject at hand?

Accuracy: Are there any factual errors in the paper? Is anything misrepresented?

Form: Is the writing and its style clear? Is the paper sensibly organized? Are all citations provided and complete? Are images of high quality and adequately captioned?

As you prepare an article, you may find that answers to the above questions feel self-evident. But it’s important to confirm that they are also apparent on the page. To evaluate if they are, I recommend stepping out of our role as author and trying to approach your text as a reader. Reread your article and try to answer the above questions using only the information provided by the text. For instance, does the text name its intended audience? Can you highlight a sentence that states the article’s central claim? If so, is this in a place where a reader could easily find it, such as the introduction, or is it buried in the paper’s middle?

Approaching your text as though you are a reader can provide new insights into how effectively the text presents your article’s argument, methodology, and contribution to the field. These are the key things that your article’s first readers—the managing editor, editor-in-chief, and peer reviewers at the journal where you’re submitting—will be looking for.

Maggie Taft, PhD
Founding Director
Writing Space