Update on STRmix™ research in response to PCAST
The President’s Council of Advisors on Science and Technology (PCAST) is a council, chartered (or re-chartered) in each US President’s administration with a broad mandate to advise the President on science and technology.
In 2016, PCAST published recommended “actions to strengthen forensic science and promote its more rigorous use in the courtroom”. This report discussed a number of forensic disciplines. The report [1] and subsequently an addendum [2] included recommendations for the interpretation of complex DNA mixtures.
PCAST defined a complex mixture as any profile with three or more donors. PCAST is rightly strongly positive about probabilistic genotyping and see it as a large improvement over previous methods. The report noted perceived limits to the proof of validity of the use of probabilistic genotyping in some situations. In particular they highlighted gaps regarding high ratio and high contributor number mixtures. PCAST considered validity proven for mixtures containing “three contributors where the person of interest comprises at least 20% of the sample.” [2]. They noted that the “few studies that have explored 4- or 5-person mixtures often involve mixtures that are derived from only a few sets of people (in some cases, only one).” [2]. They call for the expansion of empirical studies, testing the validity and reliability of PG methods across a broader relevant range of profile types.
PCAST also perceived there was a gap in “the need for clarity about the scientific standards for the validity and reliability of forensic methods.” [1]. The Scientific Working Group on DNA Analysis Methods (SWGDAM) [3] and the International Society for Forensic Genetics (ISFG) [4] have both published comprehensive guidelines that inform how to test a probabilistic genotyping system to ensure reliability and validity of results. The developmental validation of STRmix™ following the SWGDAM guidelines has previously been published [5].
At the time of the PCAST report there was a considerable number of empirical studies already undertaken by various laboratories who had implemented, or were in the process of implementing, STRmix™. These followed the SWGDAM guidelines for internal validation [6, 7]. They were not published in the peer reviewed literature largely because it is the policy of many journals not to publish such material. Some of these studies are already in the public domain on websites (see for example [8, 9]). These unpublished internal validations are better than something generic but published. This is because they are specific to the exact methods employed at the laboratory.
Since the appearance of the PCAST report, the Federal Bureau of Investigation Laboratory, Quantico, has published its STRmix™ internal validation in the peer reviewed literature [10], also in accordance with the SWGDAM guidelines. This publication reports 277 mixtures with two to five donors and a range of mixture ratios and templates.
We add to that published work, by compiling the STRmix™ internal validation material from 31 laboratories, which allows a novel look at data spanning laboratory technology and process. 2825 mixtures compiled from 31 laboratories (including multi laboratory systems) who are using STRmix™ in casework (28/31) or currently validating STRmix™ for future use in casework (3/31) has been submitted for consideration for publication. Mixtures of three, four, five, and six contributors were specifically targeted in order to address the four key areas that they felt additional validation would be merited:
(1) How well does the method perform as a function of the number of contributors to the mixture? How well does it perform when the number of contributors to the mixture is unknown?
(2) How does the method perform as a function of the number of alleles shared among individuals in the mixture? Relatedly, how does it perform when the mixtures include related individuals?
(3) How well does the method perform—and how does accuracy degrade—as a function of the absolute and relative amounts of DNA from the various contributors?
(4) Under what circumstances—and why—does the method produce results (random inclusion probabilities) that differ substantially from those produced by other methods?
We address point 1 by analysing all 2825 mixtures assuming the apparent number of contributors. The apparent number of contributors (N) was determined blind by the submitting laboratory following their own standard operating procedures. Additionally, we have assumed N+1 for a subset of the data. Point 2 we address by interrogating the data with respect to the amount of allelic sharing. Point 3 we address by conducting Hp and Hd true tests on the 2825 mixtures. With over 10,000 Hp true tests and 28,250,000 Hd true tests we demonstrate the foundational validity of STRmix™ for complex, mixed DNA profiles to levels well beyond the complexity and contribution levels suggested by PCAST.
References
1. President’s Council of Advisors on Science and Technology. PCAST Releases Report on Forensic Science in Criminal Courts. 2016 20 July 2017]; Available from: https://obamawhitehouse.archives.gov/blog/2016/09/20/pcast-releases-report-forensic-science-criminal-courts(external link).
2. President’s Council of Advisors on Science and Technology. An addendum to the PCAST report on forensic science in criminal courts. 2016 20 July 2017]; Available from: https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensics_addendum_finalv2.pdf(external link)
3. Scientific Working Group on DNA Analysis Methods (SWGDAM). Guidelines for the Validation of Probabilistic Genotyping Systems. 2015 3 October 2016]; Available from: http://media.wix.com/ugd/4344b0_22776006b67c4a32a5ffc04fe3b56515.pdf(external link)
4. Coble, M.D., et al., DNA Commission of the International Society for Forensic Genetics: Recommendations on the validation of software programs performing biostatistical calculations for forensic genetics applications. Forensic Science International: Genetics, 2016. 25: p. 191-197.
5. Bright, J.-A., et al., Developmental validation of STRmix™, expert software for the interpretation of forensic DNA profiles. Forensic Science International: Genetics, 2016. 23: p. 226-239.
6. Taylor, D., J.-A. Bright, and J. Buckleton, The interpretation of single source and mixed DNA profiles. Forensic Science International: Genetics, 2013. 7(5): p. 516-528.
7. Bright, J.-A., et al., Developing allelic and stutter peak height models for a continuous method of DNA interpretation. Forensic Science International: Genetics, 2013. 7(2): p. 296-304.
8. The New York City Office of Chief Medical Examiner. Internal Validation of STRmix™ V2.4 for Fusion NYC OCME. 2016 [cited 2017 22 April 2017]; Available from: http://www1.nyc.gov/assets/ocme/downloads/pdf/STRmix-V2-4-Fusion-5C-Validation%20Summary.pdf(external link)
9. District of Columbia Department of Forensic Science Forensic Science Laboratory Forensic Biology Unit. Internal validation of STRmix™ V2.3. 2015 [cited 2017 22 April 2017]; Available from: https://dfs.dc.gov/sites/default/files/dc/sites/dfs/page_content/attachments/STRmix%20Validation.pdf(external link)
10. Moretti, T.R., et al., Internal validation of STRmix for the interpretation of single source and mixed DNA profiles. Forensic Science International: Genetics, 2017. 29: p. 126-144.