Comments on Dion and Mitchell 2019 "How many citations to women is 'enough'? Estimates of gender representation in political science"

In the 2019 PS: Political Science & Politics article "How Many Citations to Women Is 'Enough'? Estimates of Gender Representation in Political Science", Michelle L. Dion and Sara McLaughlin Mitchell address a question about "the normative standard for the amount women should be cited" (p. 1).

The first proposed Dion and Mitchell 2019 measure is the proportion of female members of the American Political Science Association (APSA) by section and primary field, using data from 2018. According to Dion and Mitchell 2019: "When political scientists compose course syllabi, graduate reading lists, and research bibliographies, these membership data provide guidance about the minimum representation of scholarship by women that should be included to be representative by gender" (p. 3).

But is APSA section membership in 2018 a reasonable benchmark for gender representation in course syllabi that include readings from throughout history?

Hardt et al. 2019 reported on data for readings assigned in the training of political science graduate students. Below are percentages of graduate student readings in these data that had a female first author:

So the pattern is increasing representation of women over time. If this pattern reflects increasing representation of women over time in APSA section membership or increasing representation of women among the set of researchers whose research interests include the topic of a particular section, then APSA section membership data from 2018 will overstate the percentage of women needed to ensure fair gender representation on syllabi or research bibliographies. For illustrative purposes, if a section had 20% women across the 1990s, 30% women across the 2000s, and 40% women across the 2010s, a fair "section membership" benchmark for gender representation on syllabi would not be 40%; rather, a fair "section membership" benchmark for gender representation on syllabi would be something like 20% women for syllabi readings across the 1990s, 30% women for syllabi readings across the 2000s, and 40% women for syllabi readings across the 2010s.

---

Dion and Mitchell 2019 propose another measure that is biased in the same direction and for the same reason: gender distribution of authors by journal from 2007 to 2016 inclusive for available years.

About 68% of readings in the Hardt et al. 2019 graduate training readings data were published prior to 2007: 15% of these pre-2007 readings had a first female author, but 24% of the 2007-2016 readings in the data had a first female author.

Older readings are included on Hardt et al. 2019 readings with decent frequency: 42% of readings that had the gender of the first author coded were published before 2000. However, the Dion and Mitchell 2019 measure of journal representation from 2007 to 2016 ignores these older readings, which produces a biased measure favoring women if fair representation means representation that matches the representation in the relevant pool of syllabi-worthy journal articles.

---

In a sense, this bias in the Dion and Mitchell 2019 measures might not matter much if the measures are used in the biased manner that Dion and Mitchell 2019 proposed (p. 6):

We remedy this gap by explicitly providing conservative estimates of gender diversity based on organization membership and journal article authorship for evaluating gender representation. Instructors, researchers, and editors who want to ensure that references are representative can reference these as floors (rather than ceilings) for minimally representative citations.

The Dion and Mitchell 2019 suggestion above is that instructors, researchers, and editors who want to ensure that references are representative use a conservative estimate as a floor. Both the conservative nature of the estimate and its use as a floor would produce a bias favoring women, so I'm not sure how that is helpful for instructors, researchers, and editors who want to ensure that references are representative.

---

NOTE:

1. Stata code for the analysis of the Hardt et al. 2019 data:

tab female1 if year<1970

tab female1 if year>=1970 & year<1980

tab female1 if year>=1980 & year<1990

tab female1 if year>=1990 & year<2000

tab female1 if year>=2000 & year<2010

tab female1 if year>=2010 & year<2019

 

tab female1

tab female1 if year<2000

di 36791/87398