2025 Real World Testing Plan for Zoobook EHR
Click the following to view the Real World Test Plans from previous years
Transitions of Care
Report Id: 1
Standards and Certification Criteria
- Standard: United States Core Data for Interoperability (USCDI)
-
Certification Criteria:
- § 170.315(b)(1): Transitions of Care
- Method for Standards Update: Standards Version Advancement Process (SVAP)
Testing Objectives
Objective 1: Transition of Care Functionality
Certification Criterion: § 170.315(b)(1)
-
Measurement/Metric:
- Percentage of successful C-CDA (Continuity of Care Document) creations, sends, and receives, validated by system logs and data exchange records.
- Frequency of transmission errors (measured over a testing period to ensure ongoing interoperability).
-
Methodology:
-
System Performance Logging:Collect data on successful vs. failed transmission attempts for C-CDA documents. Log time taken for each successful transmission, ensuring secure exchange via
TLS.
- Automated Monitoring: Continuously monitor for transmission errors and analyze log patterns to ensure system performance aligns with certification criteria.
- Real Patient Data Verification: In outpatient and residential behavioral health settings, conduct real-time validation of C-CDA creation and exchange with recipient systems.
-
Expected Outcome:
- System Functionality: ≥ 95% successful document exchange rate for C-CDA across all tested settings, with less than 2% error rate over a continuous period.
- Interoperability Validation: All successful transmissions adhere to secure exchange protocols, validating TLS functionality.
Objective 2: Compatibility with C-CDA and XDM Attachments
Certification Criterion: § 170.315(b)(1)
-
Measurement/Metric:
- Percentage of XDM package and C-CDA attachment processing success rates, monitored over a set time frame.
- Accuracy in MIME type recognition and handling during transmission and receipt, verified through automated processing logs.
-
Methodology:
- Automated Testing: Run system checks for attachment recognition, including MIME type, and verify accurate rendering of attachments in receiving system.
- Integration Tests: Use different attachment types in real-world cases within each care setting to confirm handling consistency.
-
Expected Outcome:
- User and System Utility: ≥ 90% successful processing and readability for all XDM and C-CDA attachments.
- Error-Free Attachment Handling: Less than 5% failure rate in attachment recognition and rendering during the entire testing period.
Objective 3: Rendering and Formatting of C-CDA Documents
Certification Criterion: § 170.315(b)(1)
-
Measurement/Metric:
- Rate of successful, clear rendering for C-CDA documents with CSS styling over a defined user sample.
- Percentage of C-CDA documents that meet readability standards without display issues.
-
Methodology:
- System Validation Testing: Conduct readability testing with real C-CDA documents in both behavioral health settings to ensure CSS compatibility.
- Automated Checks: Run checks to validate CSS rendering and identify any cases where document formatting fails to meet readability criteria.
-
Expected Outcome:
- Readability Verification: 100% of tested documents are rendered clearly with CSS, maintaining readability standards.
- Error Rate: Document readability issues affect less than 1% of tested C-CDA exchanges, with failures documented and addressed.
Care Settings Addressed
- Outpatient Behavioral Health: Capability to generate C-CDA with details like reason for referral and referring provider’s information.
- Residential Behavioral Health: Support for C-CDA discharge summaries with referral reasons, provider information, and discharge instructions.
Electronic Health Information (EHI) Export
Report Id: 2
Standards and Certification Criteria
- Standard: United States Core Data for Interoperability (USCDI)
-
Certification Criteria:
- § 170.315(b)(10): Electronic Health Information Export
- Method for Standards Update: Standards Version Advancement Process (SVAP)
Testing Objectives
Objective 1: EHI Export Functionality Accuracy
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- Success rate of error-free EHI exports (target ≥ 98%).
- Real-time monitoring of export failures and associated error codes.
-
Methodology:
- System Log Analysis: Regular logging of data export attempts to monitor error rates and track any anomalies.
- Data Integrity Testing: Random sampling of exported files for validation of format, structure, and completeness.
-
Expected Outcome:
- Interoperability Validation: ≥ 98% of exports are completed without data corruption or format errors, ensuring accurate data transfer.
Objective 2: Export Usability Efficiency
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- Average completion time for export processes (target ≤ 2 minutes per export).
- Percentage of exports completed without user intervention.
-
Methodology:
- Automated Timing Logs: Measure time taken for exports, from initiation to completion, ensuring efficient workflow.
- Usability Tracking: Monitor user interactions to identify any instances where additional steps or troubleshooting are required.
-
Expected Outcome:
- System Efficiency: Exports complete within the defined time frame for 90% of attempts, minimizing user burden.
Objective 3: Role-Based Access Control
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- 100% access success for authorized users; 0% access for unauthorized users, recorded in access logs.
-
Methodology:
- Role-Based Testing: Verify access control by simulating various user roles, ensuring only authorized personnel can initiate exports.
- Continuous Monitoring: Log each export attempt, tagging authorized and unauthorized users to maintain security compliance.
-
Expected Outcome:
- Data Security: Full compliance with access restrictions, safeguarding patient data by restricting export initiation to authorized users only.
Objective 4: Date Range Configuration Accuracy
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- Accuracy rate in applying date ranges in exports, targeting 98% correct application.
-
Methodology:
- Internal Configuration Testing: Regularly test export configurations to ensure accurate adherence to selected date ranges.
- Sample Export Validation: Cross-check exported data against the specified date range to confirm configuration accuracy
-
Expected Outcome:
- Export Precision: ≥ 98% of date-range-configured exports are executed correctly, ensuring reliable data selection.
Objective 5: Batch Export Efficiency
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- Average processing time for batch exports to be ≤ 1.5 times the time required for individual exports.
-
Methodology:
- Performance Logging: Monitor processing times for batch exports relative to individual exports.
- System Load Testing: Simulate various batch sizes to validate consistency in processing speed and efficiency.
-
Expected Outcome:
- Batch Processing Efficiency: Batch exports achieve targeted processing times in at least 90% of cases, maintaining system efficiency during high-volume exports.
Objective 6: Transmission Reliability and Storage Options
Certification Criterion: § 170.315(b)(10)
-
Measurement/Metric:
- Transmission success rate of ≥ 99% across all tested storage endpoints
-
Methodology:
- Automated Transmission Tracking: Continuously monitor and log transmission outcomes, noting any failures or delays.
- Endpoint Validation: Test multiple storage locations to ensure compatibility and consistent data transfer.
-
Expected Outcome:
- System Reliability: ≥ 99% of exports reach the designated storage endpoint without transmission errors, ensuring data integrity and adaptability for various clinical workflows.
Care Settings Addressed
- Outpatient Behavioral Health: Essential for EHI export needs, particularly for transitions between facilities.
- Residential Behavioral Health: Facilitates continuity of care for inpatient and post-discharge follow-up, emphasizing role-based access.
Clinical Quality Measures (CQM)
Report Id: 3
Standards and Certification Criteria
- Standard: United States Core Data for Interoperability (USCDI)
-
Certification Criteria:
- § 170.315(c)(1): Clinical Quality Measures – Record and Export
- § 170.315(c)(2): Clinical Quality Measures – Import and Calculate
- § 170.315(c)(3): Clinical Quality Measures – Report
- Method for Standards Update: Standards Version Advancement Process (SVAP)
Testing Objectives
Objective 1: Record and Export Clinical Quality Measures
Certification Criterion: § 170.315(c)(1)
-
Measurement/Metric:
- Accuracy rate of recorded CQM data, targeting ≥ 99%.
- Success rate of data exports in the required QRDA format, with less than 1% failure rate.
-
Methodology:
- Automated Data Integrity Checks: Validate recorded CQM data accuracy and consistency across a sample of patient records.
- Export Success Rate Monitoring: Track and log successful exports and capture any formatting errors in compliance with §170.205(h)(2).
-
Expected Outcome:
- Data Integrity and Export Success: ≥ 99% accuracy in data recording and export for CQMs, ensuring full interoperability with standards.
Objective 2: Import and Calculate Clinical Quality Measures
Certification Criterion: § 170.315(c)(2)
-
Measurement/Metric:
- 100% compliance of imported files with §170.205(h)(2) standards.
- Accuracy rate of CQM calculations, with all calculations validated against expected outcomes.
-
Methodology:
- Automated File Import Validation: Track compliance of imported files with required standards, ensuring no data loss or misinterpretation.
- Calculation Consistency Check: Perform validation of CQM calculations for sample cases to confirm accurate computation without developer intervention
-
Expected Outcome:
- System Accuracy: Accurate and consistent import and calculation for all CQMs, allowing independent functionality as required for certification.
Objective 3: Report Clinical Quality Measures
Certification Criterion: § 170.315(c)(3)
-
Measurement/Metric:
- Percentage of reports generated in QRDA-compliant formats, with 100% adherence to CMS guidelines.
- Success rate of reporting for each use case (category I for inpatient, category III for ambulatory), targeting 100%.
-
Methodology:
- Automated Reporting Validation: Use CMS guidelines to validate QRDA format compliance for all generated reports.
- Use Case Testing: Conduct thorough testing to ensure all reports meet the specific requirements of inpatient and ambulatory care.
-
Expected Outcome:
- Standards Compliance: 100% of reports meet QRDA standards, providing full compatibility for quality reporting.
Care Settings Addressed
- Outpatient Behavioral Health: Emphasis on QRDA standards tailored to outpatient clinical quality measurement reporting.
- Residential Behavioral Health: Adapts QRDA documentation for inpatient settings, reflecting unique data requirements.
Application Access
Report Id: 4
Standards and Certification Criteria
- Standard: United States Core Data for Interoperability (USCDI)
-
Certification Criteria:
- § 170.315(g)(7): Application Access – Patient Selection
- § 170.315(g)(9): Application Access – All Data Request
- § 170.315(g)(10): Standardized API for Patient and Population Services
- Method for Standards Update: Standards Version Advancement Process (SVAP)
- Relied Upon Software: EMR Direct – phiQuery HL7 FHIR Services
Testing Objectives
Objective 1: Patient Selection Functionality
Certification Criterion: § 170.315(g)(7)
-
Measurement/Metric:
- Success rate of generating unique patient identifiers, targeting ≥ 99%.
- Response time for retrieving patient data, with a target of ≤ 3 seconds per query.
-
Methodology:
- Automated Performance Monitoring: Track all patient selection requests, capturing success rates and response times.
- System Logging and Error Tracking: Monitor any instances where patient identifiers are not generated or retrieved accurately.
-
Expected Outcome:
- System Reliability: ≥ 99% success rate for unique patient ID generation and retrieval.
- Performance: Average response time for patient data retrieval is ≤ 3 seconds, ensuring quick access.
Objective 2: All Data Request Functionality
Certification Criterion: § 170.315(g)(9)
-
Measurement/Metric:
- Accuracy and completeness rate of data retrieval for patient summaries in CCD format, targeting 100%.
- Success rate in retrieving data across all USCDI data categories, with less than 1% failure rate.
-
Methodology:
- Automated Data Verification: Run consistency checks to verify that all requested USCDI data elements are included in retrieved summaries.
- System Performance Monitoring: Track data retrieval requests, monitoring for accuracy and compliance with USCDI standards.
-
Expected Outcome:
- Data Completeness: 100% of data requests return CCD-formatted summaries meeting USCDI standards.
- System Accuracy: Less than 1% error rate in data retrieval, ensuring comprehensive patient data access.
Objective 3: Standardized API for Patient and Population Services
Certification Criterion: § 170.315(g)(10)
-
Measurement/Metric:
- Success rate for handling multiple patient data requests in parallel, with a target of ≥ 98%.
- Response times for both single and multiple patient data requests, with ≤ 5 seconds for bulk requests.
-
Methodology:
- API Load Testing: Test API with simultaneous patient data requests to ensure consistent performance under load.
- System Log Analysis: Record and analyze request success rates and response times to measure efficiency and reliability.
-
Expected Outcome:
- Performance Efficiency: ≥ 98% success rate for handling both single and multiple data requests.
- Speed of Response: Response time for multiple patient data requests is ≤ 5 seconds, meeting industry standards for API performance.
Care Settings Addressed
- Outpatient Behavioral Health: Unique documentation requirements based on outpatient workflows.
- Residential Behavioral Health: Testing in residential settings to account for varying documentation and data exchange needs.
Direct Project
Report Id: 5
Standards and Certification Criteria
- Standard: United States Core Data for Interoperability (USCDI)
-
Certification Criteria:
- § 170.315(h)(1): Direct Project
- Method for Standards Update: Standards Version Advancement Process (SVAP)
- Relied Upon Software: EMR Direct – phiMail Direct Messaging
Testing Objectives
Objective 1: Application Access for Direct Messaging
Certification Criterion: § 170.315(h)(1)
-
Measurement/Metric:
- Success rate of sending and receiving health information via Direct Messaging, targeting ≥ 98% success across all attempts.
- Average time taken to send and receive Direct messages, with a target of ≤ 3 seconds per transaction.
-
Methodology:
- Automated System Monitoring: Track the success rate of message transmission and receipt to ensure seamless interoperability.
- Performance Logging: Log and measure response times for each transaction, ensuring efficiency within the specified timeframe.
- Error Analysis: Capture and analyze any messaging failures, tracking error rates and root causes.
-
Expected Outcome:
- System Reliability: ≥ 98% of Direct messages are successfully sent and received, with response times of ≤ 3 seconds per transaction.
- Compliance with Standards: All transmitted messages adhere to the Applicability Statement for Secure Health Transport, Version 1.2.
Objective 2: Compliance with Delivery Notification Standards
Certification Criterion: § 170.315(h)(1)
-
Measurement/Metric:
- Accuracy rate for delivery notification transmission, targeting 100% compliance with Version 1.0 of the Implementation Guide (IG) for Delivery Notification.
- Rate of confirmed receipt notifications, targeting ≥ 99% reliability
-
Methodology:
- Automated Compliance Verification: Continuously track and validate that all delivery notifications are sent in compliance with Direct Messaging standards.
- Notification Success Tracking: Log each notification event, confirming that messages are reliably transmitted and received by third parties.
- System Error Monitoring: Identify and document any notification failures to maintain high reliability.
-
Expected Outcome:
- Notification Reliability: 100% of delivery notifications meet Direct Messaging standards, ensuring consistent and dependable communication.
- High Performance: Delivery notifications achieve ≥ 99% confirmed receipt rate, reflecting the system's dependability in real-world settings.
Care Settings Addressed
- Outpatient Behavioral Health: Supports workflows where messaging and information exchange follow outpatient standards.
- Residential Behavioral Health: Adapts for inpatient care requirements, reflecting operational differences in data sharing processes.