gen-ai-capstone
Wise Verse AI
1. Introduction
According to a 2023 AMIA survey reported by the American Academy of Family Physicians (AAFP), over 70% of clinicians cited clinical documentation burden as a major contributor to burnout. The same report revealed that physicians spend more time on charting and paperwork than direct patient care, with many expressing frustration over redundant and inefficient workflows in Electronic Health Record (EHR) systems.
This growing dissatisfaction underscores the need for a smarter, more efficient solution—one that reduces administrative overhead without compromising the quality of care.
This project addresses that gap by building a generative AI-based pipeline for automating medical documentation. The system processes audio-based doctor-patient interactions and outputs structured, FHIR-compliant resources, including QuestionnaireResponse objects and SOAP-style summaries. The goal is to minimize manual documentation work in healthcare settings by generating AI-powered clinical records that are accurate, standardized, and immediately usable.
2. Problem Statement
Manual documentation in healthcare is time-consuming and often detracts from patient care. Doctors spend a significant amount of time recording patient information, which can lead to inefficiencies and burnout. There is a need for an AI-powered system that can:
- Reduce documentation workload
- Deliver faster and accurate clinical records
- Generate structured outputs compatible with EHR systems
3. Project Objectives
- Transcribe and diarize clinical conversations
- Match relevant FHIR Questionnaires dynamically
- Generate QuestionnaireResponse based on context
- Summarize the interaction using the SOAP format
- Validate and store data in a FHIR-compliant manner
- Rate the generated output and determine its legitimacy
4. Dataset
The dataset used for this project is a collection of simulated patient-physician interviews, available at: https://springernature.figshare.com/collections/A_dataset_of_simulated_patient-physician_medical_interviews_with_a_focus_on_respiratory_cases/5545842/1
5. System Workflow
The pipeline consists of several stages, from input processing to response generation and storage.
Step-by-Step Process:
- Input Prompt: The user provides an audio file along with specific instructions.
- Questionnaire Discovery: The system queries the HAPI FHIR server to find a matching Questionnaire. If not found, it exits with an error.
- Audio Analyzation: The audio file is analyzed using a custom pipeline for intent extraction and segment understanding.
- Contextual Input Integration: Additional medical records can be added.
- Response Generation: A valid QuestionnaireResponse is created.
- Summarization: The conversation is summarized using the SOAP format (Subjective, Objective, Assessment, Plan).
- Binary file for soap note: The SOAP note is saved as a FHIR Binary resource. A linked DocumentReference ensures it’s easily accessible and EHR-compatible.
- Validation & Rating: The output is checked for validity and rated.
- Data Saving: The structured response is uploaded to the FHIR server.
- User Feedback: Confirmation is sent to the user.
6. Tech Stack
- Backend Logic: Python 3.11
- Development Environment: Kaggle Notebook
- Generative AI: Google GenAI (google.genai, google.genai.types)
- API Handling: Google API Core and Retry
- AI Experimentation Platform: Google AI Studio
7. Output
CASE 1. Valid FHIR Output
If a matching questionnaire is found and all conditions are met, the system produces:
- A valid QuestionnaireResponse resource
- A SOAP-format summary encoded as a FHIR Binary
- A DocumentReference linking the SOAP note
- Legitimacy check of the generated content
- Rating system to evaluate the quality of the output
- All resources are validated and uploaded to the HAPI FHIR server
- The user receives a success confirmation
CASE 2. Questionnaire Not Found
If the questionnaire is not available in the database, but the prompt is still relevant:
- A SOAP-format summary is still generated and encoded as a Binary
- A corresponding DocumentReference is created
- Legitimacy check and rating system are applied
- No QuestionnaireResponse is generated
- The user is notified that the questionnaire was not found, but the summary was processed successfully
CASE 3. Unsupported Instruction / Invalid Form Request
If the input prompt refers to a form or instruction for which no associated questionnaire exists in the system:
- No documentation is generated
-
The system returns the following error:
Error: Cannot pretty print AgentState fields.
The questionnaire associated with the requested form is currently unavailable in the database. - The user is prompted to revise the input or try a different form
8. Challenges Faced
- Data collection and preprocessing
- Designing models tailored for specific clinical documentation
- Team collaboration and task delegation
- Ensuring accurate transcription from varied audio inputs
- Handling incomplete or inconsistent questionnaire data
- Validating complex nested FHIR resources
9. Limitations
- Data: Acquiring diverse, real-world clinical data (privacy/ethics).
- Accuracy: Achieving high transcription/NLU accuracy in noisy, varied real settings.
- Validation: Ensuring clinical validity and safety, not just technical correctness.
- Integration: Seamlessly embedding into diverse EHR workflows.
- Trust & Adoption: Gaining clinician acceptance; designing efficient review processes.
- Bias: Detecting and mitigating potential AI biases (demographics, accents).
- Cost & Scalability: Infrastructure and maintenance considerations.
10. Future Enhancements
- Adaptive Questionnaire Generation: AI could dynamically generate questionnaires based on the conversation, even when none are pre-defined.
- Real-Time Doctor Assistant: Integration with live transcription and summarization could allow doctors to receive suggestions in real time during consultations.
- Full EHR Integration: By embedding the system within EHR platforms, doctors could review, edit, and approve AI-generated content directly in their clinical workflow.
- Personalized AI Co-pilots for Healthcare: With fine-tuning and user feedback, the system could evolve into an AI co-pilot that understands clinician preferences and adapts outputs accordingly.
- App or Web-Based GUI: Develop an intuitive, accessible interface to bring the system into real-world clinical environments.
11. Conclusion
This project highlights how generative AI can be leveraged to reduce documentation burdens in healthcare. By combining transcription, contextual integration, FHIR compliance, and summarization, the system delivers a scalable solution for modern medical documentation workflows.
12. References
- Howard, A., Flynn, B., O’Neill, M., lu, N., & ln, P. (2025). Gen AI Intensive Course Capstone 2025Q1. @Kaggle. https://www.kaggle.com/competitions/gen-ai-intensive-course-capstone-2025q1#submission-instructions
- FPM Editors. (2024, September 10). Documentation tasks still a significant burden for physicians. Brand; American Academy of Family Physicians. https://www.aafp.org/pubs/fpm/blogs/inpractice/entry/amia-survey.html
- Smith Christopher William., Faiha Fareez, Parikh, T., Wavell, C., Shahab, S., Chevalier, M., Good, S., Isabella De Blasi, Rafik Rhouma, McMahon, C., Lam, J.-P., Lo, T., & Smith, C. W. (2022). A dataset of simulated patient-physician medical interviews with a focus on respiratory cases. Scientific Data, 9(1). https://doi.org/10.1038/s41597-022-01423-1
- Creates FHIR Binary and DocumentReference resources to represent the generated SOAP note in a standard, interoperable format suitable for EHR integration.
Video:
- having an explanation using some presentation which includes the problem statement and the use case of the project
- explaining the workflow of the project using the flowchart
- Explaining using the code:
- explaining the major functions in the code
- learnings which we have learned during the 5 day workshop
- showing the output
Adaptive Questionnaire Generation: AI could dynamically generate questionnaires based on the conversation, even when none are pre-defined.
Real-Time Doctor Assistant: Integration with live transcription and summarization could allow doctors to receive suggestions in real time during consultations.
Full EHR Integration: By embedding the system within EHR platforms, doctors could review, edit, and approve AI-generated content directly in their clinical workflow.
Personalized AI Co-pilots for Healthcare: With fine-tuning and user feedback, the system could evolve into an AI co-pilot that understands clinician preferences and adapts outputs accordingly.
App or Web-Based GUI: Develop an intuitive, accessible interface to bring the system into real-world clinical environments.
Youtube overview
