NAS Crime Lab Report

[L.J.Steele]

Science Found Wanting in Nation’s Crime Labs

Forensic evidence that has helped convict thousands of defendants for nearly a century is often the product of shoddy scientific practices that should be upgraded and standardized, according to accounts of a draft report by the nation’s pre-eminent scientific research group.

The report by the National Academy of Sciences is to be released this month. People who have seen it say it is a sweeping critique of many forensic methods that the police and prosecutors rely on, including fingerprinting, firearms identification and analysis of bite marks, blood spatter, hair and handwriting.

The report says such analyses are often handled by poorly trained technicians who then exaggerate the accuracy of their methods in court. It concludes that Congress should create a federal agency to guarantee the independence of the field, which has been dominated by law enforcement agencies, say forensic professionals, scholars and scientists who have seen review copies of the study. Early reviewers said the report was still subject to change.

* * *

One person who has reviewed the draft and who asked not to be identified because of promises to keep the contents confidential said: “I’m sure that every defense attorney in the country is waiting for this report to come out. There are going to be challenges to fingerprints and firearms evidence and the general lack of empirical grounding. It’s going to be big.”

http://www.nytimes.com/2009/02/05/us/05 ... wanted=all

[Gerald Clough]

The report says such analyses are often handled by poorly trained technicians who then exaggerate the accuracy of their methods in court.

I don't doubt there's some truth in that. But exaggerations of accuracy are at least as common among scientific experts working the defense side.
(I'm just a little sour right now, dealing with absurdly absolute forensic conclusions that make you want to slap someone.)

It concludes that Congress should create a federal agency to guarantee the independence of the field, which has been dominated by law enforcement agencies, say forensic professionals, scholars and scientists who have seen review copies of the study. Early reviewers said the report was still subject to change.

I kind of doubt that there is any constitutional federal authority to control forensic analysis outside the federal criminal justice system and - maybe - the federal civil (meaning criminal) appeals system, even if there was a willingness to pay the enormous cost, and the appeals courts don't retry the evidence, and state cases will still carry evidence processed to state standards along with it in the record.

One person who has reviewed the draft and who asked not to be identified because of promises to keep the contents confidential said: “I’m sure that every defense attorney in the country is waiting for this report to come out. There are going to be challenges to fingerprints and firearms evidence and the general lack of empirical grounding. It’s going to be big.”

They may have been waiting for the report, but I doubt they would be happy to be themselves limited to the same central forensic authority. Too much of their defense and appeals argument depends on equal or greater overstatement of certainty and often cleverly worded opinion citing deliberately selected evidence, not uncommonly by experts who are publicly espousing a personal agenda in ways that would earn an official analyst a sharp reprimand.

And some of the defense bar will, as so often happens, try to apply features of the report without any real understanding of the different perspectives of academic science and forensics. The rest know what they're doing and will recognize that nothing's changed and that the mode is still adversarial advocacy and will take the issues raised in the report as useful things to discuss with their own paid experts and may very properly use it to help them urge the court to provide alternate expert review of evidence. That, in my opinion, is the way to keep people more or less honest and reporting properly. Scientists are naturally looking for exactly reliable systems that always produce the truth. Well, justice ain't science.

[Charles Parker]

Gerald---can you run for Governor----I want to vote for you.

As usual your reasoning is sound and I appreciate you posting in the best manner that you do.

If I try posting on that subject I would have been seeing red.

And to close with "Justice ain't Science" well that is the nail on the head.

[Gerald Clough]

Gerald---can you run for Governor----I want to vote for you.

I don't think I have gubernatorial hair. The only office I was ever elected to was secretary of a volunteer fire department. I failed to attend the election of officers meeting and, for my sins, was voted in. I'm more likely to run for the border than for governor and would probably have to if elected. And, if drafted, I can readily produce a note from my doctor attesting that I'm too well to serve in that capacity.

I think I have to agree with the old Ranger (Joaquin Jackson, I think) that the only high office worth having is Mexican general.

[mdavis]

Ironic, isn't it, that the reported language of the report is guilty of the same sins of which it accuses crime labs? One would expect the actual NAS language to be different. I note the report itself is not quoted.
"Forensic evidence .... is often the product..."
"... shoddy scientific practices ...." (an oxymoron)
".... analyses are often handled ...." (I wonder what their definition of "often" might be)

"...Congress should create a federal agency ..." Don't the feds now require any lab applying for federal funds be "accredited?" That requirement alone has led to the closure of many excellent smaller independent labs and has created a large expense for remaining labs to increase personnel and increase redundant testing and paperwork with a resultant increase in backlogs. Yes, there is the suggestion of a conflict of interest in the "Local Police Crime Lab" testifying to evidence submitted by "The Local Police Department." The real question is whether forensic analysis should be under the direct control of submitting agencies or "disinterested third party" labs, and whether technicians are trained scientists or rollovers from law enforcement. Of the tens of thousands of reports issued daily by our forensic labs, what percentage of reports are inaccurate? How many innocent defendants are exonerated by accurate forensic analysis? Does one Mayfield or McKee case qualify the output as "often?"

[Daktari]

Oh dear, oh dear, oh dear.
Time for another Action Plan for Excellence methinks!
Or maybe not.

[Mark]

Apparently this report is coming out on Wednesday 2/18, after a public briefing.

Public release date: 13-Feb-2009
[ Print Article | E-mail Article | Close Window ]

Contact: Sara Frueh
news@nas.edu
202-334-2138
National Academy of Sciences

Feb. 18 public briefing on forensic science in the US
STRENGTHENING THE FORENSIC SCIENCE SYSTEM IN THE UNITED STATES: A PATH FORWARD, a new congressionally mandated report from the National Research Council, takes a broad look at the needs of the nation's crime labs and medical examiner system, discusses the scientific status of many forensic methods, and recommends steps policymakers and practitioners should take to improve the U.S. forensic science system. The report will be released at a one-hour public briefing on Wednesday, Feb. 18.

The National Research Council is the operating arm of the National Academy of Sciences and National Academy of Engineering.

BRIEFING DETAILS: 1 p.m. EST on Feb. 18 in the Lecture Room of the National Academy of Sciences building, 2100 C St., N.W., Washington, D.C. Those who cannot attend may listen to a live audio webcast and submit questions using an e-mail form at http://national-academies.org.


###

PARTICIPATING FROM THE COMMITTEE THAT WROTE THE REPORT:

-- Harry T. Edwards (co-chair), senior circuit judge and chief judge emeritus, U.S. Court of Appeals for the District of Columbia Circuit, Washington, D.C.

-- Constantine Gatsonis (co-chair), professor of biostatistics and director, Center for Statistical Sciences, Brown University, Providence, R.I.

REPORTERS: TO REQUEST A COPY OF THE REPORT OR REGISTER TO ATTEND THE BRIEFING, contact the Office of News and Public Information at tel. 202-334-2138 or e-mail news@nas.edu.

[mdavis]

"Crime laboratories around the country are grossly underfunded, lack a scientific foundation and are compromised by critical delays in analyzing physical evidence..."

Actually, I think the report is rather fair. But, like everything else these days, there isn't enough money to go around, so we choose our priorities. The recent federal requirements for labs to be "accredited" is basically a good idea, but it has in turn caused a backlash of funding issues. The accreditation burden is sapping the budgets and manpower out of the lab system. Staggering, unwieldy paperwork requirements associated with the accreditation bookwork significantly reduces time on-task for technicians, adds a layer of administration and directly impacts the "24% increase in backlogs since 2002." Given persons whose sole responsibility is to maintain and advance the accreditation profile, the entire process has become a self-fulfilling prophecy and is consuming itself in time and money at the expense of timely analysis. (When you have a lawn mower, everything looks like grass.)

Additionally, the reference to scientific basis has some merit. How many latent print examiners (for example) were educated with degrees in the hard sciences? That is not to say that an English or history or criminal justice major cannot become a very competent latent print examiner. But it can speak toward the potential level of conservatism and awareness of potential bias in writing conclusions. It is amazing how few people know what "the scientific method" is or how to explain it to a jury, for example.

In the past, latent print examiners had two levels of protection. First, the discipline was "self policing." Today, that has seemingly gotten out of hand in some agencies, either through inadequate training and/or peers rubber stamping results without scrutiny. Second, we are now blessed and cursed by a plethora of candidates generated by increasingly sophisticated and accurate search algorithms in our AFIS systems. Where we used to have a few known suspects from which to choose, now we have dozens of very likely possibilities from national databases and an exponentially greater chance of a bad ident (excuse me, "individualization").

The report is a good idea. Let's just hope the feds don't jump off the deep end by adding additional requirements and restraints instead of weeding out the people who really shouldn't be in our lab systems. Let's not throw the baby out with the bathwater over a small handful of cherry-picked cases that are an infinitesimally small percentage of the daily output from all of our labs.

[Kasey Wertheim]

International Association for Identification
Robert J. Garrett, President
[address, phone, and e-mail removed]

February 19, 2009

Yesterday the National Academy of Sciences (NAS) released the long-awaited report of the Committee titled Identifying the Needs of the Forensic Science Community. That report contains numerous items of interest to members of the International Association for Identification (IAI). There has not been adequate time to fully evaluate the entire 254 page report but based on preliminary reviews, below are some points that might be considered if members are queried regarding the report:

• The Consortium of Forensic Science Organizations (CFSO) (website: www.thecfso.org) of which the IAI is a member was largely responsible for convincing Congress of the need for this committee. We thank Congress and the National Academies for their hard work over the past several years to produce this report and call attention to the needs of the nation’s forensic community.

• The IAI has over the years supported the need for this kind of study. During the course of the NAS hearings the IAI was invited to present its positions concerning these issues. With the release of this report, the IAI stands ready to support many of the committee’s recommendations and work with the necessary parties to achieve those goals.

• The IAI endorses continuing research in pattern evidence to include fingerprint evidence. In fact that was recognized in the final paragraph of the IAI’s Standardization Committee’s report in 1973 that called for additional research to be conducted to provide continuing scientific support to fingerprint identification

• Over the years a number of research projects have been conducted. None of those projects refuted the scientific principle that fingerprints are unique and permanent.

• There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.

• The IAI endorses accreditation of forensic service providers as well as certification of examiners in their respective disciplines. To that end the IAI put in place its first certification program 32 years ago and to date has added certification programs in six additional disciplines.

• Members who may have to testify about friction ridge identifications are reminded that the admissibility of their testimony rests with the presiding judge. Challenges to the underlying science and practice are handled in Daubert/Frye type hearings and should not affect direct testimony in the trial proper.

• It is suggested that members not assert 100% infallibility (zero error rate) when addressing the reliability of fingerprint comparisons.
Visit us on the Web at TheIAI.org

• Although the IAI does not, at this time, endorse the use of probabilistic models when stating conclusions of identification, members are advised to avoid stating their conclusions in absolute terms when dealing with population issues.

• The IAI will be asking its science and practice committees to review the NAS report over the next few weeks and will provide the membership with additional guidance in the future.

• The Executive Summary of the report is available at: www.nas.edu at no cost. The entire report is available from the same website at a cost of $33.00.

Robert Garrett
IAI President

[L.J.Steele]

There was an interesting discussion of the report on WBUR today. You can listen below.

http://www.onpointradio.org/shows/2009/ ... l-science/