Sonish Sivarajkumar - Academia.edu (original) (raw)
Papers by Sonish Sivarajkumar
arXiv (Cornell University), Jun 5, 2023
Objective: To pre-train fair and unbiased patient representations from Electronic Health Records ... more Objective: To pre-train fair and unbiased patient representations from Electronic Health Records (EHRs) using a novel weighted loss function that reduces bias and improves fairness in deep representation learning models. Methods: We defined a new loss function, called weighted loss function, in the deep representation learning model to balance the importance of different groups of patients and features. We applied the proposed model, called Fair Patient Model (FPM), to a sample of 34,739 patients from the MIMIC-III dataset and learned patient representations for four clinical outcome prediction tasks. Results: FPM outperformed the baseline models in terms of three fairness metrics: demographic parity, equality of opportunity difference, and equalized odds ratio. FPM also achieved comparable predictive performance with the baselines, with an average accuracy of 0.7912. Feature analysis revealed that FPM captured more information from clinical features than the baselines. Conclusion: FPM is a novel method to pre-train fair and unbiased patient representations from EHR data using a weighted loss function. The learned representations can be used for various downstream tasks in healthcare and can be extended to other domains where bias and fairness are important.
Cells, Jun 2, 2022
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
BACKGROUND Natural language processing (NLP) has become an emerging technology in health care tha... more BACKGROUND Natural language processing (NLP) has become an emerging technology in health care that leverages a large amount of free-text data in electronic health records to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models often requires large, annotated data sets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated data sets is typical in clinical NLP; therefore, ensuring that deep learning models perform well is crucial for real-world clinical NLP applications. A widely adopted approach is fine-tuning existing pretrained language models, but these attempts fall short when the training data set contains only a few annotated samples. Few-shot learning (FSL) has recently been investigated to tackle this problem. Siamese neural network (SNN) has been widely used as an FSL approach in computer vision but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. OBJECTIVE The aim of our study is to propose and evaluate SNN-based approaches for few-shot clinical NLP tasks. METHODS We propose 2 SNN-based FSL approaches, including pretrained SNN and SNN with second-order embeddings. We evaluate the proposed approaches on the clinical sentence classification task. We experiment with 3 few-shot settings, including 4-shot, 8-shot, and 16-shot learning. The clinical NLP task is benchmarked using the following 4 pretrained language models: bidirectional encoder representations from transformers (BERT), BERT for biomedical text mining (BioBERT), BioBERT trained on clinical notes (BioClinicalBERT), and generative pretrained transformer 2 (GPT-2). We also present a performance comparison between SNN-based approaches and the prompt-based GPT-2 approach. RESULTS In 4-shot sentence classification tasks, GPT-2 had the highest precision (0.63), but its recall (0.38) and F score (0.42) were lower than those of BioBERT-based pretrained SNN (0.45 and 0.46, respectively). In both 8-shot and 16-shot settings, SNN-based approaches outperformed GPT-2 in all 3 metrics of precision, recall, and F score. CONCLUSIONS The experimental results verified the effectiveness of the proposed SNN approaches for few-shot clinical NLP tasks.
Research Square (Research Square), Apr 3, 2023
Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facili... more Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating e cient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this eld. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this eld. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases,
arXiv (Cornell University), Mar 8, 2022
Alzheimer's Disease (AD) is the most common form of dementia in the United States. Sleep is one o... more Alzheimer's Disease (AD) is the most common form of dementia in the United States. Sleep is one of the lifestylerelated factors that has been shown critical for optimal cognitive function in old age.. However, there is a lack of research studying the association between sleep and AD incidence. A major bottleneck for conducting such research is that the traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients' subjective experience. In this study, we developed a rule-based NLP algorithm and machine learning models to automate the extraction of sleep-related concepts, including snoring, napping, sleep problem, bad sleep quality, daytime sleepiness, night wakings, and sleep duration, from the clinical notes of patients diagnosed with AD. We trained and validated the proposed models on the clinical notes retrieved from the University of Pittsburgh of Medical Center (UPMC). The results show that the rule-based NLP algorithm consistently achieved the best performance for all sleep concepts.
JMIR AI
Background Natural language processing (NLP) has become an emerging technology in health care tha... more Background Natural language processing (NLP) has become an emerging technology in health care that leverages a large amount of free-text data in electronic health records to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models often requires large, annotated data sets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated data sets is typical in clinical NLP; therefore, ensuring that deep learning models perform well is crucial for real-world clinical NLP applications. A widely adopted approach is fine-tuning existing pretrained language models, but these attempts fall short when the training data set contains only a few annotated samples. Few-shot learning (FSL) has recently been investigated to tackle this problem. Siame...
arXiv (Cornell University), Aug 31, 2022
Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that l... more Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that leverages a large amount of free-text data in electronic health records (EHRs) to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models usually requires large annotated datasets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated datasets is typical in clinical NLP and therefore, ensuring that deep learning models perform well is crucial for the models to be used in real-world applications. A widely adopted approach is fine-tuning existing Pre-trained Language Models (PLMs), but these attempts fall short when the training dataset contains only a few annotated samples. Few-Shot Learning (FSL) has recently been investigated to tackle this problem. Siamese Neural Network (SNN) has been widely utilized as an FSL approach in computer vision, but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. In this paper, we propose two SNN-based FSL approaches for clinical NLP, including Pre-Trained SNN (PT-SNN) and SNN with Second-Order Embeddings (SOE-SNN). We evaluated the proposed approaches on two clinical tasks, namely clinical text classification and clinical named entity recognition. We tested three few-shot settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP tasks were benchmarked using three PLMs, including Bidirectional Encoder Representations from Transformers (BERT), Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT), and Bio + Clinical BERT (BioClinicalBERT). The experimental results verified the effectiveness of the proposed SNN-based FSL approaches in both NLP tasks.
Cells
Organ-on-a-chip (OOAC) is an emerging technology based on microfluid platforms and in vitro cell ... more Organ-on-a-chip (OOAC) is an emerging technology based on microfluid platforms and in vitro cell culture that has a promising future in the healthcare industry. The numerous advantages of OOAC over conventional systems make it highly popular. The chip is an innovative combination of novel technologies, including lab-on-a-chip, microfluidics, biomaterials, and tissue engineering. This paper begins by analyzing the need for the development of OOAC followed by a brief introduction to the technology. Later sections discuss and review the various types of OOACs and the fabrication materials used. The implementation of artificial intelligence in the system makes it more advanced, thereby helping to provide a more accurate diagnosis as well as convenient data management. We introduce selected OOAC projects, including applications to organ/disease modelling, pharmacology, personalized medicine, and dentistry. Finally, we point out certain challenges that need to be surmounted in order to fu...
Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facili... more Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this field. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases, including Ovid Embase, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Databa...
Deep learning algorithms are dependent on the availability of large-scale annotated clinical text... more Deep learning algorithms are dependent on the availability of large-scale annotated clinical text datasets. The lack of such publicly available datasets is the biggest bottleneck for the development of clinical Natural Language Processing(NLP) systems. Zero-Shot Learning(ZSL) refers to the use of deep learning models to classify instances from new classes of which no training data have been seen before. Prompt-based learning is an emerging ZSL technique where we define task-based templates for NLP tasks. We developed a novel prompt-based clinical NLP framework called HealthPrompt and applied the paradigm of prompt-based learning on clinical texts. In this technique, rather than fine-tuning a Pre-trained Language Model(PLM), the task definitions are tuned by defining a prompt template. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
Alzheimer’s Disease (AD) is the most common form of dementia in the United States. Sleep is one o... more Alzheimer’s Disease (AD) is the most common form of dementia in the United States. Sleep is one of the lifestyle-related factors that has been shown critical for optimal cognitive function in old age.. However, there is a lack of research studying the association between sleep and AD incidence. A major bottleneck for conducting such research is that the traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients’ subjective experience. In this study, we developed a rule-based NLP algorithm and machine learning models to automate the extraction of sleep-related concepts, including snoring, napping, sleep problem, bad sleep quality, daytime sleepiness, night wakings, and sleep duration, from the clinical notes of patients diagnosed with AD. We trained and validated the proposed models on the clinical notes retrieved from the University of Pittsburgh Medical Center (UPMC). The results show that the rule-based NLP algorithm consiste...
arXiv (Cornell University), Jun 5, 2023
Objective: To pre-train fair and unbiased patient representations from Electronic Health Records ... more Objective: To pre-train fair and unbiased patient representations from Electronic Health Records (EHRs) using a novel weighted loss function that reduces bias and improves fairness in deep representation learning models. Methods: We defined a new loss function, called weighted loss function, in the deep representation learning model to balance the importance of different groups of patients and features. We applied the proposed model, called Fair Patient Model (FPM), to a sample of 34,739 patients from the MIMIC-III dataset and learned patient representations for four clinical outcome prediction tasks. Results: FPM outperformed the baseline models in terms of three fairness metrics: demographic parity, equality of opportunity difference, and equalized odds ratio. FPM also achieved comparable predictive performance with the baselines, with an average accuracy of 0.7912. Feature analysis revealed that FPM captured more information from clinical features than the baselines. Conclusion: FPM is a novel method to pre-train fair and unbiased patient representations from EHR data using a weighted loss function. The learned representations can be used for various downstream tasks in healthcare and can be extended to other domains where bias and fairness are important.
Cells, Jun 2, 2022
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
BACKGROUND Natural language processing (NLP) has become an emerging technology in health care tha... more BACKGROUND Natural language processing (NLP) has become an emerging technology in health care that leverages a large amount of free-text data in electronic health records to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models often requires large, annotated data sets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated data sets is typical in clinical NLP; therefore, ensuring that deep learning models perform well is crucial for real-world clinical NLP applications. A widely adopted approach is fine-tuning existing pretrained language models, but these attempts fall short when the training data set contains only a few annotated samples. Few-shot learning (FSL) has recently been investigated to tackle this problem. Siamese neural network (SNN) has been widely used as an FSL approach in computer vision but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. OBJECTIVE The aim of our study is to propose and evaluate SNN-based approaches for few-shot clinical NLP tasks. METHODS We propose 2 SNN-based FSL approaches, including pretrained SNN and SNN with second-order embeddings. We evaluate the proposed approaches on the clinical sentence classification task. We experiment with 3 few-shot settings, including 4-shot, 8-shot, and 16-shot learning. The clinical NLP task is benchmarked using the following 4 pretrained language models: bidirectional encoder representations from transformers (BERT), BERT for biomedical text mining (BioBERT), BioBERT trained on clinical notes (BioClinicalBERT), and generative pretrained transformer 2 (GPT-2). We also present a performance comparison between SNN-based approaches and the prompt-based GPT-2 approach. RESULTS In 4-shot sentence classification tasks, GPT-2 had the highest precision (0.63), but its recall (0.38) and F score (0.42) were lower than those of BioBERT-based pretrained SNN (0.45 and 0.46, respectively). In both 8-shot and 16-shot settings, SNN-based approaches outperformed GPT-2 in all 3 metrics of precision, recall, and F score. CONCLUSIONS The experimental results verified the effectiveness of the proposed SNN approaches for few-shot clinical NLP tasks.
Research Square (Research Square), Apr 3, 2023
Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facili... more Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating e cient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this eld. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this eld. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases,
arXiv (Cornell University), Mar 8, 2022
Alzheimer's Disease (AD) is the most common form of dementia in the United States. Sleep is one o... more Alzheimer's Disease (AD) is the most common form of dementia in the United States. Sleep is one of the lifestylerelated factors that has been shown critical for optimal cognitive function in old age.. However, there is a lack of research studying the association between sleep and AD incidence. A major bottleneck for conducting such research is that the traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients' subjective experience. In this study, we developed a rule-based NLP algorithm and machine learning models to automate the extraction of sleep-related concepts, including snoring, napping, sleep problem, bad sleep quality, daytime sleepiness, night wakings, and sleep duration, from the clinical notes of patients diagnosed with AD. We trained and validated the proposed models on the clinical notes retrieved from the University of Pittsburgh of Medical Center (UPMC). The results show that the rule-based NLP algorithm consistently achieved the best performance for all sleep concepts.
JMIR AI
Background Natural language processing (NLP) has become an emerging technology in health care tha... more Background Natural language processing (NLP) has become an emerging technology in health care that leverages a large amount of free-text data in electronic health records to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models often requires large, annotated data sets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated data sets is typical in clinical NLP; therefore, ensuring that deep learning models perform well is crucial for real-world clinical NLP applications. A widely adopted approach is fine-tuning existing pretrained language models, but these attempts fall short when the training data set contains only a few annotated samples. Few-shot learning (FSL) has recently been investigated to tackle this problem. Siame...
arXiv (Cornell University), Aug 31, 2022
Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that l... more Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that leverages a large amount of free-text data in electronic health records (EHRs) to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models usually requires large annotated datasets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated datasets is typical in clinical NLP and therefore, ensuring that deep learning models perform well is crucial for the models to be used in real-world applications. A widely adopted approach is fine-tuning existing Pre-trained Language Models (PLMs), but these attempts fall short when the training dataset contains only a few annotated samples. Few-Shot Learning (FSL) has recently been investigated to tackle this problem. Siamese Neural Network (SNN) has been widely utilized as an FSL approach in computer vision, but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. In this paper, we propose two SNN-based FSL approaches for clinical NLP, including Pre-Trained SNN (PT-SNN) and SNN with Second-Order Embeddings (SOE-SNN). We evaluated the proposed approaches on two clinical tasks, namely clinical text classification and clinical named entity recognition. We tested three few-shot settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP tasks were benchmarked using three PLMs, including Bidirectional Encoder Representations from Transformers (BERT), Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT), and Bio + Clinical BERT (BioClinicalBERT). The experimental results verified the effectiveness of the proposed SNN-based FSL approaches in both NLP tasks.
Cells
Organ-on-a-chip (OOAC) is an emerging technology based on microfluid platforms and in vitro cell ... more Organ-on-a-chip (OOAC) is an emerging technology based on microfluid platforms and in vitro cell culture that has a promising future in the healthcare industry. The numerous advantages of OOAC over conventional systems make it highly popular. The chip is an innovative combination of novel technologies, including lab-on-a-chip, microfluidics, biomaterials, and tissue engineering. This paper begins by analyzing the need for the development of OOAC followed by a brief introduction to the technology. Later sections discuss and review the various types of OOACs and the fabrication materials used. The implementation of artificial intelligence in the system makes it more advanced, thereby helping to provide a more accurate diagnosis as well as convenient data management. We introduce selected OOAC projects, including applications to organ/disease modelling, pharmacology, personalized medicine, and dentistry. Finally, we point out certain challenges that need to be surmounted in order to fu...
Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facili... more Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this field. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases, including Ovid Embase, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Databa...
Deep learning algorithms are dependent on the availability of large-scale annotated clinical text... more Deep learning algorithms are dependent on the availability of large-scale annotated clinical text datasets. The lack of such publicly available datasets is the biggest bottleneck for the development of clinical Natural Language Processing(NLP) systems. Zero-Shot Learning(ZSL) refers to the use of deep learning models to classify instances from new classes of which no training data have been seen before. Prompt-based learning is an emerging ZSL technique where we define task-based templates for NLP tasks. We developed a novel prompt-based clinical NLP framework called HealthPrompt and applied the paradigm of prompt-based learning on clinical texts. In this technique, rather than fine-tuning a Pre-trained Language Model(PLM), the task definitions are tuned by defining a prompt template. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
Alzheimer’s Disease (AD) is the most common form of dementia in the United States. Sleep is one o... more Alzheimer’s Disease (AD) is the most common form of dementia in the United States. Sleep is one of the lifestyle-related factors that has been shown critical for optimal cognitive function in old age.. However, there is a lack of research studying the association between sleep and AD incidence. A major bottleneck for conducting such research is that the traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients’ subjective experience. In this study, we developed a rule-based NLP algorithm and machine learning models to automate the extraction of sleep-related concepts, including snoring, napping, sleep problem, bad sleep quality, daytime sleepiness, night wakings, and sleep duration, from the clinical notes of patients diagnosed with AD. We trained and validated the proposed models on the clinical notes retrieved from the University of Pittsburgh Medical Center (UPMC). The results show that the rule-based NLP algorithm consiste...