Recent advances in large language models (LLMs) have opened new possibilities for automating clinical documentation. The use of generative AI to produce admission summaries offers a potential solution to the increasing documentation burden faced by clinicians. The standardized workflow of elective surgeries in the urology department offers a valuable setting to explore the integration of AI for automating documentation processes. This study aims to explore the feasibility and performance of LLM-based AI tools in generating admission notes, and to compare their quality and completeness with clinician-written summaries using a standardized assessment framework.

Patients scheduled for elective inpatient procedures through outpatient clinics between January and July 2024 were enrolled. Admission summaries were generated by GPT-4o based on unstructured outpatient clinic notes and compared with physician-written admission notes for the corresponding procedures. Documentation quality was assessed using the QNOTE scoring system, a validated tool comprising 12 categories and 44 items for evaluating clinical documentation across multiple domains. Comparisons among the three groups—outpatient clinic notes, AI-generated admission summaries (GPT-4o), and physician-written admission notes—were conducted using non-parametric statistical methods. A p-value < 0.05 was considered statistically significant.

A total of 46 patients were included in the analysis. Comparative results are summarized in Table 1. Both GPT-4o–generated and physician-written admission summaries demonstrated superior documentation quality, with median total QNOTE scores of 78.00 and 80.50, respectively (out of 84), compared to 43.00 for the original outpatient clinic notes. While physician-authored summaries scored significantly higher in multiple categories, this difference may reflect the AI’s limited input compared to physicians' access to comprehensive clinical data. Notably, in sections emphasizing structured information (e.g. review of systems, physical findings), AI performance was comparable to or exceeded that of physicians. This suggests inconsistencies in physician documentation, with some outpatient findings omitted from structured admission notes.

Generative AI models (GAI/LLMs) demonstrated the capacity to generate high-quality admission summaries for elective urology procedures based on outpatient clinic notes. While physician-written notes achieved higher overall QNOTE scores, the AI-generated summaries performed comparably in several structured domains, and in some instances outperformed physicians—particularly in documenting symptoms and physical findings. Although clinician oversight remains essential, AI-generated notes offer a promising supplement to clinical workflows. Further studies are warranted to evaluate their integration into routine practice, including assessments of efficiency and reliability.