Search results for: FKGL

Authors: Oscar Covarrubias, Diane Ghanem, Christopher Murdock, Babar Shafiq

Abstract:

Introduction: ChatGPT is an advanced language AI tool designed to understand and generate human-like text. The aim of this study is to assess the ability of ChatGPT-4 to re-write orthopaedic trauma patient education materials at the recommended 6th-grade level. Methods: Two independent reviewers accessed ChatGPT-4 (chat.openai.com) and gave identical instructions to simplify the readability of provided text to a 6th-grade level. All trauma-related articles by the Orthopaedic Trauma Association (OTA) and American Academy of Orthopaedic Surgeons (AAOS) were sequentially provided. The academic grade level was determined using the Flesh-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE). Paired t-tests and Wilcox-rank sum tests were used to compare the FKGL and FRE between the ChatGPT-4 revised and original text. Inter-rater correlation coefficient (ICC) was used to assess variability in ChatGPT-4 generated text between the two reviewers. Results: ChatGPT-4 significantly reduced FKGL and increased FRE scores in the OTA (FKGL: 5.7±0.5 compared to the original 8.2±1.1, FRE: 76.4±5.7 compared to the original 65.5±6.6, p < 0.001) and AAOS articles (FKGL: 5.8±0.8 compared to the original 8.9±0.8, FRE: 76±5.5 compared to the original 56.7±5.9, p < 0.001). On average, 14.6% of OTA and 28.6% of AAOS articles required at least two revisions by ChatGPT-4 to achieve a 6th-grade reading level. ICC demonstrated poor reliability for FKGL (OTA 0.24, AAOS 0.45) and moderate reliability for FRE (OTA 0.61, AAOS 0.73). Conclusion: This study provides a novel, simple and efficient method using language AI to optimize the readability of patient education content which may only require the surgeon’s final proofreading. This method would likely be as effective for other medical specialties.

Keywords: artificial intelligence, AI, chatGPT, patient education, readability, trauma education

Procedia PDF Downloads 41

Authors: Diane Ghanem, Oscar Covarrubias, Ridge Maxson, Samir Sabharwal, Babar Shafiq

Abstract:

Introduction: Web-based resources serve as a fundamental educational platform for orthopaedic trauma patients; however, they are notoriously written at a high grade reading level and are often too complicated for patients to benefit from them. The aim of this study is to perform an updated assessment of the readability of the AAOS trauma-related educational articles and compare their readability with that of injury-specific patient education materials developed by the OTA. Methods: All forty-six trauma-related articles on the AAOS patient education website were analyzed for readability. Two independent reviewers used the (1) Flesch-Kincaid Grade Level (FKGL) and the (2) Flesch Reading Ease (FRE) algorithms to calculate the readability level. Mean readability scores were compared across body part categories. One-sample t-test was done to compare mean FKGL with the recommended 6th-grade readability level and the average American adult reading level. Two-sample t-test was used to compare the readability scores of the AAOS trauma-related articles to those of the OTA. Results: The average FKGL and FRE for the AAOS articles were 8.9±0.74 and 57.2±5.8, respectively. All articles were written above the 6th-grade reading level. The average readability of the AAOS articles was significantly greater than the recommended 6th-grade and average American adult reading level. The average FKGL (8.9±0.74 vs 8.1±1.14) and FRE (57.2±5.8 vs 65.6±6.6) for all AAOS articles was significantly greater compared to that of OTA articles. Excellent agreement was observed between raters for the FKGL 0.956 (95%CI 0.922 - 0.975) and FRE 0.993 (95%CI 0.987 – 0.996). Discussion: Our findings suggest that, after almost a decade, the readability of the AAOS trauma-related articles remains unchanged. The AAOS and OTA trauma patient education materials have high readability levels and may be too difficult for patient comprehension. A need remains to improve the readability of these commonly used trauma education materials.

Keywords: american ocademy of orthopaedic surgeons, FKGL, FRE, orthopaedic trauma association, patient education, readability

Procedia PDF Downloads 35