Meredith Nahm | Duke University (original) (raw)

Papers by Meredith Nahm

Increasing amounts of clinical research data are collected by manual data entry into electronic s... more Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

Studies in health technology and informatics, 2011

Governments and providers are investing in health information technologies with little evidence a... more Governments and providers are investing in health information technologies with little evidence as to their ultimate value. We present a conceptual framework that can be used by hospitals, clinics, and health care systems to evaluate their health information technologies. The framework contains three dimensions that collectively define generic evaluation types. When these types are combined with contextual considerations, they define specific evaluation problems. The first dimension, domain, determines whether the evaluation will address the information intervention or its outcomes. The second dimension, mechanism, identifies the specific components of the new information technology and/or its health care system that will be the subject of the evaluation study. And, the third dimension, timing, determines whether the evaluation occurs before or after the health information technology is implemented. Answers to these questions define a set of evaluation types each with generic sets o...

AMIA Joint Summits on Translational Science proceedings AMIA Summit on Translational Science, 2010

Medical record abstraction, a primary mode of data collection in secondary data use, is associate... more Medical record abstraction, a primary mode of data collection in secondary data use, is associated with high error rates. Cognitive factors have not been studied as a possible explanation for medical record abstraction errors. We employed the theory of distributed representation and representational analysis to systematically evaluate cognitive demands in medical record abstraction and the extent of external cognitive support employed in a sample of clinical research data collection forms.We show that the cognitive load required for abstraction in 61% of the sampled data elements was high, exceedingly so in 9%. Further, the data collection forms did not support external cognition for the most complex data elements. High working memory demands are a possible explanation for the association of data errors with data elements requiring abstractor interpretation, comparison, mapping or calculation. The representational analysis used here can be used to identify data elements with high co...

Background: Industry standards provide rigorous descriptions of required data presentation, with ... more Background: Industry standards provide rigorous descriptions of required data presentation, with the aim of ensuring compatibility across different clinical studies. However despite their crucial importance, these standards are often not used as expected in the development of clinical research. The reasons for this lack of compliance could be related to the high cost and time-intensive nature of the process of data standards implementation. The bjective of this study was to evaluate the value of the extra time and cost required for different levels of data standardisation and the likelihood of researchers to comply with these levels. Since we believe that the cost and time necessary for the implementation of data standards can change over time, System Dynamics (SD) analysis was used to investigate how these variables interact and influence the adoption of data standards by clinical researchers. Methods: Three levels of data standards implementation were defined through focus group discussion involving four clinical research investigators. Ten Brazilian and eighteen American investigators responded to an online questionnaire which presented possible standards implementation scenarios, with respondents asked to choose one of two options available in each scenario. A random effects ordered probit model was used to estimate the effect of cost and time on investigators' willingness to adhere to data standards. The SD model was used to demonstrate the relationship between degrees of data standardisation and subsequent variation in cost and time required to start the associated study. Results: A preference for low cost and rapid implementation times was observed, with investigators more likely to incur costs than to accept a time delay in project start-up. SD analysis indicated that although initially extra time and cost are necessary for clinical study standardisation, there is a decrease in both over time. Conclusions: Future studies should explore ways of creating mechanisms which decrease the time and cost associated with standardisation processes. In addition, the fact that the costs and time necessary for data standards implementation decrease with time should be made known to the wider research community. Policy makers should attempt to match their data standardisation policies better with the expectations of researchers.

Clinical Trials, 2011

Background Case report forms (CRFs) are used to collect data in clinical research. Case report fo... more Background Case report forms (CRFs) are used to collect data in clinical research. Case report form development represents a significant part of the clinical trial process and can affect study success. Libraries of CRFs can preserve the organizational knowledge and expertise invested in CRF development and expedite the sharing of such knowledge. Although CRF libraries have been advocated, there have

Objective: Inefficiencies in clinical trial data collection cause delays, increase costs, and may... more Objective: Inefficiencies in clinical trial data collection cause delays, increase costs, and may reduce clinician participation in medical research. In this proof-of-concept study, we examine the feasibility of using point-of-care data capture for both the medical record and clinical research in the setting of a working clinical trial. We hypothesized that by doing so, we could increase reuse of patient

AMIA Joint Summits on Translational Science proceedings AMIA Summit on Translational Science, 2013

The panel addresses the urgent need to ensure that comparative effectiveness research (CER) findi... more The panel addresses the urgent need to ensure that comparative effectiveness research (CER) findings derived from diverse and distributed data sources are based on credible, high-quality data; and that the methods used to assess and report data quality are consistent, comprehensive, and available to data consumers. The panel consists of representatives from four teams leveraging electronic clinical data for CER, patient centered outcomes research (PCOR), and quality improvement (QI) and seeks to change the current paradigm where data quality assessment (DQA) is performed "behind the scenes" using one-off project specific methods. The panelists will present their process of harmonizing existing models for describing and measuring clinical data quality and will describe a comprehensive integrated framework for assessing and reporting DQA findings. The collaborative project is supported by the Electronic Data Methods (EDM) Forum, a three-year grant from the Agency for Healthc...

AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium, 2009

A systematic classification of study designs would be useful for researchers, systematic reviewer... more A systematic classification of study designs would be useful for researchers, systematic reviewers, readers, and research administrators, among others. As part of the Human Studies Database Project, we developed the Study Design Typology to standardize the classification of study designs in human research. We then performed a multiple observer masked evaluation of active research protocols in four institutions according to a standardized protocol. Thirty-five protocols were classified by three reviewers each into one of nine high-level study designs for interventional and observational research (e.g., N-of-1, Parallel Group, Case Crossover). Rater classification agreement was moderately high for the 35 protocols (Fleiss' kappa = 0.442) and higher still for the 23 quantitative studies (Fleiss' kappa = 0.463). We conclude that our typology shows initial promise for reliably distinguishing study design types for quantitative human research.

Drug Information Journal - DRUG INF J, 2007

Interactions between the health care and clinical research communities are currently inefficient.... more Interactions between the health care and clinical research communities are currently inefficient. The present environment forces unnecessary redundancy, from the capture of patient data in the clinician-patient encounter to multiple uses of thai data. Clinical research operations must become more integrated with health care processes to improve efficiencies in both patient care and research. Achieving a single instance of data capture to serve the combined needs of both environments should facilitate translation of knowledge from research into better patient care. A critical first step in achieving true interoperability is to develop formal data standards that are then adopted by the larger health care and research communities. The rewards of interoperability include streamlined subject screening and enrollment procedures, improved reporting, merging and subsequent analysis of clinical data sets, and expansion of knowledge made possible by leveraging research data and results from o...

Journal of the American Medical Informatics Association, 2007

Inefficiencies in clinical trial data collection cause delays, increase costs, and may reduce cli... more Inefficiencies in clinical trial data collection cause delays, increase costs, and may reduce clinician participation in medical research. In this proof-of-concept study, we examine the feasibility of using point-of-care data capture for both the medical record and clinical research in the setting of a working clinical trial. We hypothesized that by doing so, we could increase reuse of patient data, eliminate redundant data entry, and minimize disruption to clinic workflow. We developed and used a point-of-care electronic data capture system to record data during patient visits. The standards-based system was used for clinical research and to generate the clinic note for the medical record. The system worked in parallel with data collection procedures already in place for an ongoing multicenter clinical trial. Our system was iteratively designed after analyzing case report forms and clinic notes, and observing clinic workflow patterns and business procedures. Existing data standards from CDISC and HL7 were used for database insertion and clinical document exchange. Our system was successfully integrated into the clinic environment and used in two live test cases without disrupting existing workflow. Analyses performed during system design yielded detailed information on practical issues affecting implementation of systems that automatically extract, store, and reuse healthcare data. Although subject to the limitations of a small feasibility study, our study demonstrates that electronic patient data can be reused for prospective multicenter clinical research and patient care, and demonstrates a need for further development of therapeutic area standards that can facilitate researcher use of healthcare data.

Journal of Medical Systems, 2011

New technologies may be required to integrate the National Institutes of Health&amp;amp;a... more New technologies may be required to integrate the National Institutes of Health&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s Patient Reported Outcome Management Information System (PROMIS) into multi-center clinical trials. To better understand this need, we identified likely PROMIS reporting formats, developed a multi-center clinical trial process model, and identified gaps between current capabilities and those necessary for PROMIS. These results were evaluated by key trial constituencies. Issues reported by principal investigators fell into two categories: acceptance by key regulators and the scientific community, and usability for researchers and clinicians. Issues reported by the coordinating center, participating sites, and study subjects were those faced when integrating new technologies into existing clinical trial systems. We then defined elements of a PROMIS Tool Kit required for integrating PROMIS into a multi-center clinical trial environment. The requirements identified in this study serve as a framework for future investigators in the design, development, implementation, and operation of PROMIS Tool Kit technologies.

Journal of Biomedical Informatics, 2009

Data management software applications specifically designed for the clinical research environment... more Data management software applications specifically designed for the clinical research environment are increasingly available from commercial vendors and open-source communities, however, general-purpose spreadsheets remain widely employed in clinical research data management (CRDM). Spreadsheet suitability for this use is controversial, and no formal comparative usability evaluations have been performed. We report on an application of the UFuRT (User, Function, Representation, and Task (analyses)) methodology to create a domain-specific process for usability evaluation. We demonstrate this process in an evaluation of differences in usability between a spreadsheet program (Microsoft® Excel) and a commercially available clinical research data management system (CDMS) (Phase Forward Clintrial ™ ). Through this domain-specific operationalization of UFuRT methodology, we successfully identified usability differences and quantified task and cost differences, while delineating these from socio-technical aspects. UFuRT can similarly be generalized to other domains.

International Journal of Functional Informatics and Personalised Medicine, 2010

We report the development and implementation of a methodology for standardising clinical data ele... more We report the development and implementation of a methodology for standardising clinical data elements. The methodology, piloted using Tuberculosis (TB) and Acute Coronary Syndromes (ACS) domains, relies on clinicians for natural language definitions and on ...

Clinical Trials, 2009

Background-Clinical trial networks were created to provide a sustaining infrastructure for the co... more Background-Clinical trial networks were created to provide a sustaining infrastructure for the conduct of multisite clinical trials. As such, they must withstand changes in membership. Centralization of infrastructure including knowledge management, portfolio management, information management, process automation, work policies, and procedures in clinical research networks facilitates consistency and ultimately research.

AMIA Summits on Translational Science Proceedings, 2010

Academic Medicine, 2009

Clinical and translational research increasingly requires computation. Projects may involve multi... more Clinical and translational research increasingly requires computation. Projects may involve multiple computationally oriented groups including information technology (IT) professionals, computer scientists, and biomedical informaticians. However, many biomedical researchers are not aware of the distinctions among these complementary groups, leading to confusion, delays, and suboptimal results. Although written from the perspective of Clinical and Translational Science Award (CTSA) programs within academic medical centers,

AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium, 2010