The urology department routinely handles a high volume of inpatient elective procedures, contributing to a demanding clinical environment. The application of Artificial Intelligence (AI) has the potential to streamline repetitive tasks, particularly in clinical documentation. This study aims to evaluate the performance and accuracy of generative AI and large language models (GAI/LLMs) in producing admission summaries based on outpatient clinic notes.

Patients undergoing inpatient elective procedures, arranged through outpatient clinic visits between January and April 2024, were included in this study. AI models, GPT-4 (Model 1) and GPT-4o (Model 2), were prompted to generate admission summaries based on a single outpatient clinic note per patient. The quality of the generated summaries was assessed using the QNOTE scoring system, a non-disease-specific, 12-category, 44-individual-item rubric that evaluates the quality of clinical documentation across various domains.

A total of 14 patients were included in the evaluation. Both AI models produced high-quality admission summaries, with Model 1 (GPT-4) achieving an average QNOTE score of 87.92, and Model 2 (GPT-4o) scoring 91.48 (out of 100). Both models achieved perfect scores in several categories. However, Model 2 consistently outperformed Model 1 in both subjective assessments and across multiple QNOTE domains. The distribution of QNOTE scores for the outpatient clinic notes, comparing Model 1 and Model 2, is shown in Figure 1.

GAI/LLMs demonstrate the capability to generate high-quality admission summaries for inpatient elective urology procedures based on a single outpatient clinic note. GPT-4o outperformed GPT-4 in both objective and subjective evaluations. While these AI models show strong potential, it remains essential for clinicians to review the generated summaries for accuracy and consistency. Further research with larger sample sizes and continued development of AI models are necessary to validate these findings and refine their clinical application.