2017 proved to be a banner year for patient identification and matching experimentation and discovery. While not all initiatives end in success, 2017 certainly proved the classic saying: nothing ventured, nothing gained.
Office of the National Coordinator (ONC) Patient Matching Algorithm Challenge (PMAC).
The Challenge ran from April 2017 through mid-October, attracting considerable interest from vendors, researchers, and private organizations. 600 mostly anonymous teams registered, generating 7,509 total data submissions. ONC provided a synthetic dataset and challenged participants to produce the highest matching accuracy while transparently demonstrating their algorithm processing and performance metrics. Participants could make as many submittals as desired to achieve the highest matching accuracy (Precision, Recall, F-Score).
My observations from ONC communications (including their December webcast) and conversations with several of the participants:
- Matching and accuracy might be construed to be “easy” since results showed that over 20 vendors scored within 1% point of each other. This is a dangerous assumption.
- Synthetic datasets have limitations. During the December 11 webcast, the four PMAC winners each recommended that organizations use real world data (with appropriate confidentiality agreements) to assess a potential patient matching vendor’s accuracy, rather than relying on synthetic datasets.
- Allowing unlimited submissions was counterproductive. I heard from several participants that “gaming the system” was really quite easy given the PMAC construct, and ONC noted this concern on the webcast.
- Patient algorithm matching transparency was a goal of the initiative, but I’m not clear how that has been achieved. Perhaps that will be forthcoming in ONC’s final analysis, to be released early this year.
- Some level of confirmatory (human) review was done by all finalists.
- Three of the four finalists used the Fellegi Sunter mathematical theory for identifying matched records. The top vendor used machine learning based upon eight predictors, but no details were provided. I’m anxious to hear more details, as I believe machine learning will have a key role in a solution approach to patient matching.
CHIME National Patient ID Challenge.
The CHIME Challenge began approximately two years ago, with four finalists and two semi-finalists collaborators drawn from 40 countries and 370 groups. The Challenge established a lofty goal: “privately, accurately, and safely confirm a patient’s identity 100 percent of the time.”
CHIME disbanded the challenge in late 2017 without elaborating on the specific reasons for the decision. I believe it was because they focused too narrowly on identifying technology to meet the stated goals, rather than taking a broader solution approach. On the positive side, CHIME announced that they will form a Patient Identification Task Force to further champion this critical patient safety and interoperability need.
Based on the ONC and CHIME projects, it’s increasingly clear that we must address the patient identification and patient matching challenge from a solution perspective that reflects the real-world data and challenges. That means going back to the basics of people, process and technology!
2018 could see some traction on this focus, with initial activity from the AHRQ awards to the Regenstrief Institute. A five year, $2.5M grant to study automated methods to patient matching includes a very promising focus on real-world, evidence-based methodology. The participation of the Indiana Network for Patient Care, the largest inter-organization clinical data repository in the US, will obviously be key to applying innovation in a real-world scenario.
The August 2017 PEW Trust letter to VA encouraged the VA to advance patient matching as part of the VA EHR deployment over the next decade. It will be interesting to see how the DOD and VA begin to address patient matching for this integrated EHR, particularly with their increased focus on interoperability.
2018 promises to be a busy year for the patient matching conversation and activities. As all sectors of the healthcare ecosystem participate in the conversation, the realization is growing that patient matching is critical to value-based payments, analytics, consumer engagement, precision medicine – and we must be willing to venture into new areas in order to make gains.