Friday, October 21, 2016

Simpson's Paradox

Elaine Schwartz  has an article on  Project Include which will be tracking employment data in the Silicon Valley. Their initial goal is to form a list of metrics, to gather the facts from 18 firms, to anonymize all and then share the results. As a baseline, those results can become a springboard for inspiring diversity. She raises the question of Simpson's Paradox.

Simpson’s Paradox
To illustrate Simpson’s paradox, I found more detailed data for a similar situation at the University of California in a 1973 gender discrimination case:
Simpson's Paradox and gender discrimination for graduate school admissions
From: Casper Albers Bog
Because 44% of all male applicants and 35% of all female applicants were accepted to their graduate schools in 1973, gender discrimination seemed rather obvious…until you took a closer look.
Questions about gender discrimination at the University of California
From: Brookings
As you can see, more women applied to more competitive departments. Meanwhile more men competed for spots in departments with high acceptance rates. Combined, these stats conveyed the impression that women experienced discrimination. However, if we consider the relationship between the proportion of female applications and admissions rates the data point us in a different direction.
Similarly, in the more recent Dutch case, analysis of the data by a Dutch psychologist indicated no discrimination. But, as he adds, although the data indicate no bias, that does not mean it did not exist.

No comments: