Abstract
Aboriginal pregnant women and new mothers face an increased risk of mental health issues, often stemming from historical trauma, including violence and discrimination. These challenges could contribute to complex trauma and adverse perinatal outcomes, highlighting the need for culturally sensitive care. However, non-Aboriginal clinicians often face barriers due to limited cultural knowledge, exacerbated by other factors such as time constraints for training and reliance on one-time training. Large Language Models (LLMs)-based chatbots offer the potential to support self-directed learning and enhance clinicians' self-efficacy through interactive question and answer. However, LLMs also pose challenges, including hallucinatory responses, outdated knowledge, fictitious information, unverifiable references, and difficulty handling domain-specific queries. In this study, we aim to mitigate these challenges by developing a specialized chatbot for improving Aboriginal perinatal mental health question-answering. The chatbot integrates Retrieval-Augmented Generation (RAG) with a semantic search engine, enabling it to retrieve verified external knowledge and provide more accurate, contextually relevant responses without frequent retraining. We evaluate its performance against a baseline GPT-3.5-turbo model and compare LLMs integrated with different RAG techniques to assess improvements in accuracy and reliability. Clinical Relevance- This study shows the potential of the specialized RAG LLM-based chatbot to improve domain-specific, clinically relevant, and on-demand question-answering support for clinicians. By providing accurate, verified information through interactive responses, it may help bridge knowledge gaps, support self-directed learning, and complement existing training.
