The developer's reply: equal accuracy for both groups
On 8 July 2016 the research department of Northpointe answered ProPublica's audit with its own analysis of the same data. The report by William Dieterich, Christina Mendoza and Tim Brennan gave the area under the ROC curve for the general recidivism scale on the outlet's main sample as 0.69 for black and 0.69 for white defendants, and found the share of those labelled higher risk who did not re-offend to be lower, not higher, for black defendants: 37 per cent against 41 per cent.
Why it matters
This is a rare case where both sides of an argument about algorithmic harm set out their numbers on the same dataset. After this reply the question stopped being which side had computed correctly and became which of two definitions of fairness has to hold, because both sides had computed different quantities correctly.
The exact values in the report: the AUC of the general recidivism scale on ProPublica's main sample is 0.69 (0.68-0.71) for black and 0.69 (0.67-0.71) for white defendants, and 0.70 (0.69-0.71) over the whole sample; for the violent recidivism scale, 0.67 for black and 0.66 for white defendants, 0.68 overall. The two-year base rate of recidivism in that sample is 0.51 for black against 0.39 for white defendants, and 0.14 against 0.09 for violent recidivism. The mean number of prior arrests is 4.4 against 2.6 and the mean age at assessment 32.7 against 37.7 years. The central objection: the outlet compared the complements of sensitivity and specificity, quantities computed separately on recidivists and non-recidivists, where a claim of bias needs the complements of the predictive values, which take the base rate into account. The record does not claim that the reply refuted the audit. Northpointe reached its conclusion that the error-rate gap follows naturally from different score distributions through its own simulation rather than a proof, and the report says so itself. The proof arrived separately and not from this company. The publisher's own copy now answers 404, so the record cites an archival copy of the same file.