Skip to Main Content

Peer Review, Citation Practices, and Finding Scholarship

The methods that scholars use to write, evaluate, and circulate their work are all systems that should be understood as not "neutral" methods.

Scholar Representation by Race
This tally acknowledges the white supremacy in this field: the numbers indicate the racial representation of the scholarship listed in this LibGuide.
Black Scholars: 3    Indigenous Scholars: 4    Latinx Scholars: 0   APID(A) Scholars: 2. White Scholars: 16

You will find this on many of our LibGuides, as a system to keep us accountable in looking at the racial diversity of the sources we provide. 
These sources have been selected based upon research for the Kennedy Library's Digital Publishing Project. 

 

What is Peer Review?

Peer review is a system to evaluate scholarship and its credibility for publication: a group of peers (an undefined standard) review the quality and nature of the work by not just the thesis, but also the voice, tone, format, and citations. The idea behind peer review is to constructively criticize scholarship to help it become better.

This system for assessing and valuing scholarship has generally taken a race-neutral stance. However, this stance neglects a diversity of thought, format, innovation, and scholarly persons. The assumption of scholars holding the same objective stances when reviewing work is incorrect.

To read more about Racial Organization Theory, start with Victor Ray's A Theory of Racialized Organization.  
To read more about inequities in peer review systems, specifically against Indigenous peoples, cbc.ca has a story

 

Types of Peer Review

Open peer review encourages anyone to be a "peer" and comment upon scholarship.

This may also be termed "Community Peer Review," which does depend upon critically considering who is part of the said "Community." 

An example of open peer review is Dr. Kathleen Fitzpatrick's Planned Obsolescence's manuscript.  To read more about the ideas of Open Peer Review, start with her "Peer Review, Judgement, and Reading." 

Many disciplines have established pre-print systems, where papers are uploaded into a "pre-print" collection for anyone to review. Some examples are Arxiv (computer science), PsyArxiv (psychology), SocOpen (sociology), and HCommons (humanities disciplines). 

The forms and practices of knowledge that may be familiar to many of us in the Cal Poly community may not follow Indigenous forms of knowledge. 

We are on the lands of yak titʸu titʸu yak tiłhini, the Northern Chumash tribe, upon which we are grateful guests. The ɬaʔamɩn Indigenous people have developed methodologies for sharing knowledge in publishing space as Traditional Knowledge Labels.  It is important to understand that ideas of ownership, knowledge possession, and sharing may be different, as generously explained by RavenSpace Publishing's As I Remember It

It is important also to remember that from Eve Tuck and K. Wayne Yang, Decolonization is Not a Metaphor

Two additional sources that have helped me a lot in rethinking ideas of valuing knowledge are the following:

  • Kathleen E. Absolon's Kaandossiwin: How We Come to Know. Fernwood Publishing: 2011. 
  • Linda Tuhiwai Smith, Eve Tuck, and K. Wayne Yang's Indigenous and Decolonizing Studies in Education: Mapping the Long View. Routledge, June 18, 2018.:

Most peer review is done in written forms, but multimodal review allows for assessment of various forms of scholarship, as well as allowing the assessment itself to be in various forms.

Here are a few examples of various forms:

  • Podcasts
    Dr. Cheryl Ball is an advocate proponent of multimodal scholarship, and she demonstrates this in a podcast peer review system here
    Read more about openness and professional advancement practices, authored by Dr. Ball, Kim Barrett, Peter Berkery, Jessica Clemons, Sheree Crosby, Holly J. Falk-Krzesinski, and Stacy Konkiel here
  • Twitter 
    Brian Watson #tweetvaluation's Dr. Hannah Turner's "Cataloguing Culture Legacies of Colonialism in Museum Documentation" here
  • Born Digital Formats  
    As more student work is becoming born digital, the standards across disciplines have turned towards not only looking at content to evaluate, but also the format: example guidelines
    Generally, the criteria calls for not assessing personal opinions on what the scholarship could be, but rather look at what it should do - access knowledge, showing significance, and justifying the form of the research. 
    This video describing digital dissertations also goes through various forms of evaluations. 
  • Other Models
    Historically, Western Europe has been the forefront of pushing for open, digital scholarship. For example, this book about digital dissertations goes through the legitimacy of a different mode - included performance based, electronic, and new digital tech.  

The author does not know who the reviewers are, but reviewers see (and hence can judge) author's name. This is the most common form of peer review among science journals.  Reviewers remain anonymous to the author. 
*The use of the word blind is ableist, and sometimes 'single anonymous' is used instead. Here, 'blind' is used for familiarity and reference, but is not suggested.

Read more about bias in single- blind peer review systems: 

  • Single- vs. double-blind reviewing at WSDM 2017 Andrew Tomkins, Min Zhang, William D. Heavlin. Proceedings of the National Academy of Sciences Nov 2017, 114 (48) 12708-12713; DOI: 10.1073/pnas.1707323114

The author does not know who the reviewers are, and reviewers do not know who the author is. Editors send the scholarship to be reviewed (often in a form of a paper) anonymously. 

Though double blind peer review may seem the most neutral, the inequities within the publishing industry can often allow for bias, even with the anonymity of names, from voice, citations, and argument. 

*The use of the word blind is ableist, and sometimes 'single anonymous' is used instead. Here, 'blind' is used for familiarity and reference, but is not suggested.

  • Read more about a librarian's thoughts about double-blind review, and it's relationship to open peer review. 
  • Medieval Studies is one example of various disciplines that have pointed out the bias that still happens in double blind peer review systems. 

History of Peer Review

Though academic publications, and the systems that create them, often may be seen as a timeless tradition, their heightened importance ("publish or perish" is a phrase that is often heard around faculty) started only around fifty years or so ago. Like most assessments (such as Standardized Testing), peer review has been both helpful and harmful in creating standards for publications.

Here at Cal Poly San Luis Obispo, the requirements for faculty to publish depends upon the department; these standards can emphasize the name of publication, the type of review, the number of publications, or the standard of 'peer' that oversees the publications.  Some of this development can be tracked in the Academic Senate Resolutions records which describe the various requirements for faculty to hold their positions.  

According to Biagioli (2002), peer review was originally used as a complement to state censorship—foreign products were censored, while those produced within the state under the auspices of national academies were subject to “internal,” that is, intrastate, peer review. The notion of an internal peer gradually moved away from the state, shifting the locus of power to academic disciplines. Today...peer review is treated as a guarantee of epistemic warrant, as well as viewed as another means of securing academic freedom. However, the sort of academic freedom peer review secures is negative—peer review erects a barrier against outside, nonacademic interference. What is lacking is any sense that peer review could also be used to expand our positive freedom (Holbrook 2010). We seek assurance from our peers that what we say is right, or at least not wrong, rather than insurance from peer review to take intellectual and academic risks. Disciplines define peers.

ALA Scholarly Communications

Because the United States has been built upon a legacy of racist practices, the creation of scholarship and higher education in general still grapples with the racism embedded in these systems.  A good place to start understanding how these recent histories have disproportionately affected BIPOC faculty is a collection edited by Dr. Patricia Matthew: Written/Unwritten: Diversity and the Hidden Truths of Tenure

The systems that create scholarship are affected by political, daily events. For example, the Cold War has affected peer review according to Dr. Melinda Baldwin; the funding, especially, of research always is affected by movements that may be considered outside of higher education. In turn, it affects policy, as seen through the history of review for the National Institute of Heath. Even technology, like a Xerox machine, affects the ability of being able to review scholarship.  Moreover, knowledge building and organizing have political agendas, as shown by Melissa Adler's arguments about nationalism, and can affect how libraries choose or organize resources. 


"...what is good for scholarship must become good for the scholar. Instead of assessing only paper-native articles, books, and proceedings, we must build a new system where all types of scholarly products are evaluated and rewarded."

-Stacy Konkiel, Heather Piwowar, Jason Priem. The Imperative for Open Altmetrics. ImpactStory. Volume 17, Issue 3: Metrics for Measuring Publishing Value: Alternative and Otherwise, Summer 2014. DOI: https://doi.org/10.3998/3336451.0017.301