Healthy Skepticism

In my last post I told you about some of my research on crime-scene fingerprint identification. Turns out that it’s actually a human, not a computer, who decides whether a crime-scene print matches a suspect or not. To make a decision, examiners place prints side-by-side, visually inspect them, and declare that the prints match or not, based on their training and experience.

The problem is that, even though fingerprints have been used in criminal courts for more than 100 years, no properly controlled experiments on fingerprint examiners' accuracy in identifying perpetrators had been conducted. Some experts have even claimed to be infallible, but mistakes made to date have resulted in innocent people being wrongly accused.

We wanted to find out whether these experts were any more accurate than the average person, and to get an idea of how many criminals are being wrongly set free and how many innocents are being wrongly convicted. Well, it took two years of planning and a visit to every major state police department in Australia, but we finally managed to conduct the critical experiment.

We gave 37 qualified fingerprint experts and 37 UQ students pairs of fingerprints to examine and decide whether a simulated crime-scene print matched a potential suspect or not. Some of the print pairs belonged to the "criminal" while others were highly similar but actually belonged to an "innocent" person.

The experts correctly matched just over 92 percent of the prints to the criminal. But, they mistakenly matched 0.68 percent of the prints to the innocent person. That they made so few errors means pretty impressive human performance, in my opinion.

We concluded that qualified court-practicing fingerprint experts are exceedingly accurate compared to novices, but are not infallible. Our experts tended to err on the side of caution by making errors that would free the guilty rather than convict the innocent. Even so, they made the kind of error that may lead to false convictions.

So, was my initial skepticism unwarranted?

Well, I think healthy skepticism is the cornerstone of science and rational enquiry. Fingerprint examiners make important decisions that can put lives and livelihoods at risk and the burden of proof is on the profession to demonstrate the scientific validity and reliability of its claims. We’ve worked closely with examiners, who were were eager to demonstrate their abilities, and we’ve shown that expertise with prints provides a real benefit.

Even though this one experiment consumed my existence for two years, it was totally worth it!

Matt
mbthompson.com

Tangen, J. M., Thompson, M. B., & McCarthy D. J. (2011). Identifying Fingerprint Expertise. Psychological Science. [PDF] [Press Release]