Automatic de-identification of textual documents in the electronic health record: a review of recent research

By Stéphane M. Meystre, J. Friedlin, Brett R. South, Shuying Shen, Matthew H. Samore. BMC Medical Research Methodology, 2010

370 citations recorded by the source corpus.

BackgroundIn the United States, the Health Insurance Portability and Accountability Act (HIPAA) protects the confidentiality of patient data and requires the informed consent of the patient and approval of the Internal Review Board to use data for research purposes, but these requirements can be waived if data is de-identified. For clinical data to be considered de-identified, the HIPAA "Safe Harbor" technique requires 18 data elements (called PHI: Protected Health Information) to be removed. The de-identification of narrative text documents is often realized manually, and requires significant resources. Well aware of these issues, several authors have investigated automated de-identification of narrative text documents from the electronic health record, and a review of recent research in this domain is presented here.MethodsThis review focuses on recently published research (after 1995), and includes relevant publications from bibliographic queries in PubMed, conference proceedings, the ACM Digital Library, and interesting publications referenced in already included papers.ResultsThe literature search returned more than 200 publications. The majority focused only on structured data de-identification instead of narrative text, on image de-identification, or described manual de-identification, and were therefore excluded. Finally, 18 publications describing automated text de-identification were selected for detailed analysis of the architecture and methods used, the types of PHI detected and removed, the external resources used, and the types of clinical documents targeted. All text de-identification systems aimed to identify and remove person names, and many included other types of PHI. Most systems used only one or two specific clinical document types, and were mostly based on two different groups of methodologies: pattern matching and machine learning. Many systems combined both approaches for different types of PHI, but the majority relied only on pattern matching, rules, and dictionaries.ConclusionsIn general, methods based on dictionaries performed better with PHI that is rarely mentioned in clinical text, but are more difficult to generalize. Methods based on machine learning tend to perform better, especially with PHI that is not mentioned in the dictionaries used. Finally, the issues of anonymization, sufficient performance, and "over-scrubbing" are discussed in this publication.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/81a53e17462b4e067f69315768c09f79f39eec71

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Molecular Biology

Citations in the corpus

De-identification of patient notes with recurrent neural networks (2016, methodology): “For a thorough review of existing systems, see (Meystre et al., 2010; Stubbs et al., 2015).”
Recent Advances in Clinical Natural Language Processing in Support of Semantic Analysis (2015, neutral): “Since the thorough review of state-of-the-art in automated de-identification methods from 2010 by Meystre et al. [21], research in this area has continued to be very active.”
Extracting information from the text of electronic medical records to improve case detection: a systematic review (2016, background): “28,79 If these algorithms perform well enough, they could be run at source—for example, within the clinical institution where identifiers are not a problem—and anonymize the text before EMRs are extracted for secondary purposes.”
Mining electronic health records: towards better research applications and clinical care (2012, background): “De-identification normally allows researchers to circumvent costly consent regimes 94 , but the lack of identifiers makes certain types of population-wide research impossible, as other information cannot be linked to data subjects.”