Research conducted by UT Southwestern Medical Center has revealed the remarkable capabilities of ChatGPT, an AI chatbot initially created for language-related tasks, in clinical research. Their discoveries represent a significant advancement in utilizing artificial intelligence to extract meaningful information from doctors' clinical notes, which has the potential to completely transform the field of biomedical research and the process of making healthcare decisions.

Although containing valuable information, clinical notes have historically presented a difficulty because of their unstructured format, requiring labour-intensive manual annotation by skilled experts. The research team aimed to overcome this obstacle by investigating if ChatGPT could simplify evaluating clinical notes to obtain organized data for research objectives.

Natural language processing (NLP) tools can turn free-text clinical notes into structured data, but they often need problem-specific comments and model training. This study aims to determine how well and quickly ChatGPT can pull information from free-text medical notes. We created a large language model (LLM)--based workflow using the spiral "prompt engineering" process and systems engineering methods. 

The researchers also used OpenAI's API for batch searching ChatGPT. They tested how well this method worked by comparing the results of ChatGPT-3.5 (gpt-3.5-turbo-16k) with structured data that experts had carefully chosen. The datasets they used were more than 1000 pathology reports for lung cancer and 191 pathology reports for pediatric osteosarcoma. ChatGPT-3.5 could extract pathological classifications in a lung cancer dataset with an overall accuracy of 89%, better than two standard NLP methods. The performance is affected by how the educational prompt is made.

Their research entailed examining more than 700 collections of pathology notes for patients with lung cancer, utilizing ChatGPT to discern crucial attributes such as tumour morphology, lymph node engagement, and the stage and subtype of the disease. ChatGPT exhibited an impressive average accuracy of 89% in making these assessments, outperforming conventional natural language processing techniques and substantially decreasing the time needed for data extraction.

Building upon these encouraging findings, the researchers expanded their inquiry to encompass other medical conditions, particularly osteosarcoma, which is the prevailing form of bone cancer among children and teenagers. ChatGPT demonstrated outstanding precision, achieving an accuracy rate of approximately 99% in classifying cancer grades and a perfect accuracy rate of 100% in evaluating margin status based on 191 clinical notes.

The consequences of this discovery are significant. Researchers can leverage ChatGPT's capabilities to quickly extract valuable insights from clinical notes, thereby expediting the progress of biological research and facilitating the creation of more efficacious healthcare therapies. Moreover, ChatGPT's precision and effectiveness are a hopeful avenue for enhancing automated clinical decision-making tools, benefiting patient outcomes and healthcare delivery.

The study conducted by UT Southwestern Medical Center demonstrates the profound capacity of AI, specifically ChatGPT, to reveal the concealed worth in clinical notes and stimulate advancements in biomedical research and healthcare.

Image source: Unsplash

Source: https://www.nature.com/articles/s41746-024-01079-8