Harnessing the Power of Large Language Models in Orthodontics: AI-Generated Insights on Class II and Class III Orthopedic Appliances: A Cross-Sectional Study

Authors: Laiba Amin, Rashna H. Sukhia, Mubassar Fida

Abstract:

Introduction: This study evaluates the accuracy of responses from ChatGPT, Google Bard, and Microsoft Copilot regarding dentofacial orthopedic appliances. As artificial intelligence (AI) increasingly enhances various fields, including healthcare, understanding its reliability in specialized domains like orthodontics becomes crucial. By comparing the accuracy of different AI models, this study aims to shed light on their effectiveness and potential limitations in providing technical insights. Materials and Methods: A total of 110 questions focused on dentofacial orthopedic appliances were posed to each AI model. The responses were then evaluated by five experienced orthodontists using a modified 5-point Likert scale to ensure a thorough assessment of accuracy. This structured approach allowed for consistent and objective rating, facilitating a meaningful comparison between the AI systems. Results: The results revealed that Google Bard demonstrated the highest accuracy at 74%, followed by Microsoft Copilot, with an accuracy of 72.2%. In contrast, ChatGPT was found to be the least accurate, achieving only 52.2%. These results highlight significant differences in the performance of the AI models when addressing orthodontic queries. Conclusions: Our study highlights the need for caution in relying on AI for orthodontic insights. The overall accuracy of the three chatbots was 66%, with Google Bard performing best for removable Class II appliances. Microsoft Copilot was more accurate than ChatGPT, which, despite its popularity, was the least accurate. This variability emphasizes the importance of human expertise in interpreting AI-generated information. Further research is necessary to improve the reliability of AI models in specialized healthcare settings.

Keywords: artificial intelligence, large language models, orthodontics, dentofacial orthopaedic appliances, accuracy assessment.