• Standard: United States Core Data for Interoperability (USCDI)
• Certification Criteria:
  • § 170.315(b)(1): Transitions of Care
• Method for Standards Update: Standards Version Advancement Process (SVAP)

Testing Objectives

• Measurement/Metric:
  • Percentage of successful C-CDA (Continuity of Care Document) creations, sends, and receives, validated by system logs and data exchange records.
  • Frequency of transmission errors (measured over a testing period to ensure ongoing interoperability).
• Methodology:
  • System Performance Logging:Collect data on successful vs. failed transmission attempts for C-CDA documents. Log time taken for each successful transmission, ensuring secure exchange via TLS.
  • Automated Monitoring: Continuously monitor for transmission errors and analyze log patterns to ensure system performance aligns with certification criteria.
  • Real Patient Data Verification: In outpatient and residential behavioral health settings, conduct real-time validation of C-CDA creation and exchange with recipient systems.
• Expected Outcome:
  • System Functionality: ≥ 95% successful document exchange rate for C-CDA across all tested settings, with less than 2% error rate over a continuous period.
  • Interoperability Validation: All successful transmissions adhere to secure exchange protocols, validating TLS functionality.

• Measurement/Metric:
  • Percentage of XDM package and C-CDA attachment processing success rates, monitored over a set time frame.
  • Accuracy in MIME type recognition and handling during transmission and receipt, verified through automated processing logs.
• Methodology:
  • Automated Testing: Run system checks for attachment recognition, including MIME type, and verify accurate rendering of attachments in receiving system.
  • Integration Tests: Use different attachment types in real-world cases within each care setting to confirm handling consistency.
• Expected Outcome:
  • User and System Utility: ≥ 90% successful processing and readability for all XDM and C-CDA attachments.
  • Error-Free Attachment Handling: Less than 5% failure rate in attachment recognition and rendering during the entire testing period.

• Measurement/Metric:
  • Rate of successful, clear rendering for C-CDA documents with CSS styling over a defined user sample.
  • Percentage of C-CDA documents that meet readability standards without display issues.
• Methodology:
  • System Validation Testing: Conduct readability testing with real C-CDA documents in both behavioral health settings to ensure CSS compatibility.
  • Automated Checks: Run checks to validate CSS rendering and identify any cases where document formatting fails to meet readability criteria.
• Expected Outcome:
  • Readability Verification: 100% of tested documents are rendered clearly with CSS, maintaining readability standards.
  • Error Rate: Document readability issues affect less than 1% of tested C-CDA exchanges, with failures documented and addressed.

Care Settings Addressed

1. Outpatient Behavioral Health: Capability to generate C-CDA with details like reason for referral and referring provider’s information.
2. Residential Behavioral Health: Support for C-CDA discharge summaries with referral reasons, provider information, and discharge instructions.

• Standard: United States Core Data for Interoperability (USCDI)
• Certification Criteria:
  • § 170.315(b)(10): Electronic Health Information Export
• Method for Standards Update: Standards Version Advancement Process (SVAP)

Testing Objectives

• Measurement/Metric:
  • Success rate of error-free EHI exports (target ≥ 98%).
  • Real-time monitoring of export failures and associated error codes.
• Methodology:
  • System Log Analysis: Regular logging of data export attempts to monitor error rates and track any anomalies.
  • Data Integrity Testing: Random sampling of exported files for validation of format, structure, and completeness.
• Expected Outcome:
  • Interoperability Validation: ≥ 98% of exports are completed without data corruption or format errors, ensuring accurate data transfer.

• Measurement/Metric:
  • Average completion time for export processes (target ≤ 2 minutes per export).
  • Percentage of exports completed without user intervention.
• Methodology:
  • Automated Timing Logs: Measure time taken for exports, from initiation to completion, ensuring efficient workflow.
  • Usability Tracking: Monitor user interactions to identify any instances where additional steps or troubleshooting are required.
• Expected Outcome:
  • System Efficiency: Exports complete within the defined time frame for 90% of attempts, minimizing user burden.

• Measurement/Metric:
  • 100% access success for authorized users; 0% access for unauthorized users, recorded in access logs.
• Methodology:
  • Role-Based Testing: Verify access control by simulating various user roles, ensuring only authorized personnel can initiate exports.
  • Continuous Monitoring: Log each export attempt, tagging authorized and unauthorized users to maintain security compliance.
• Expected Outcome:
  • Data Security: Full compliance with access restrictions, safeguarding patient data by restricting export initiation to authorized users only.

• Measurement/Metric:
  • Accuracy rate in applying date ranges in exports, targeting 98% correct application.
• Methodology:
  • Internal Configuration Testing: Regularly test export configurations to ensure accurate adherence to selected date ranges.
  • Sample Export Validation: Cross-check exported data against the specified date range to confirm configuration accuracy
• Expected Outcome:
  • Export Precision: ≥ 98% of date-range-configured exports are executed correctly, ensuring reliable data selection.

• Measurement/Metric:
  • Average processing time for batch exports to be ≤ 1.5 times the time required for individual exports.
• Methodology:
  • Performance Logging: Monitor processing times for batch exports relative to individual exports.
  • System Load Testing: Simulate various batch sizes to validate consistency in processing speed and efficiency.
• Expected Outcome:
  • Batch Processing Efficiency: Batch exports achieve targeted processing times in at least 90% of cases, maintaining system efficiency during high-volume exports.

• Measurement/Metric:
  • Transmission success rate of ≥ 99% across all tested storage endpoints
• Methodology:
  • Automated Transmission Tracking: Continuously monitor and log transmission outcomes, noting any failures or delays.
  • Endpoint Validation: Test multiple storage locations to ensure compatibility and consistent data transfer.
• Expected Outcome:
  • System Reliability: ≥ 99% of exports reach the designated storage endpoint without transmission errors, ensuring data integrity and adaptability for various clinical workflows.

Care Settings Addressed

1. Outpatient Behavioral Health: Essential for EHI export needs, particularly for transitions between facilities.
2. Residential Behavioral Health: Facilitates continuity of care for inpatient and post-discharge follow-up, emphasizing role-based access.

• Standard: United States Core Data for Interoperability (USCDI)
• Certification Criteria:
  • § 170.315(c)(1): Clinical Quality Measures – Record and Export
  • § 170.315(c)(2): Clinical Quality Measures – Import and Calculate
  • § 170.315(c)(3): Clinical Quality Measures – Report
• Method for Standards Update: Standards Version Advancement Process (SVAP)

Testing Objectives

• Measurement/Metric:
  • Accuracy rate of recorded CQM data, targeting ≥ 99%.
  • Success rate of data exports in the required QRDA format, with less than 1% failure rate.
• Methodology:
  • Automated Data Integrity Checks: Validate recorded CQM data accuracy and consistency across a sample of patient records.
  • Export Success Rate Monitoring: Track and log successful exports and capture any formatting errors in compliance with §170.205(h)(2).
• Expected Outcome:
  • Data Integrity and Export Success: ≥ 99% accuracy in data recording and export for CQMs, ensuring full interoperability with standards.

• Measurement/Metric:
  • 100% compliance of imported files with §170.205(h)(2) standards.
  • Accuracy rate of CQM calculations, with all calculations validated against expected outcomes.
• Methodology:
  • Automated File Import Validation: Track compliance of imported files with required standards, ensuring no data loss or misinterpretation.
  • Calculation Consistency Check: Perform validation of CQM calculations for sample cases to confirm accurate computation without developer intervention
• Expected Outcome:
  • System Accuracy: Accurate and consistent import and calculation for all CQMs, allowing independent functionality as required for certification.

• Measurement/Metric:
  • Percentage of reports generated in QRDA-compliant formats, with 100% adherence to CMS guidelines.
  • Success rate of reporting for each use case (category I for inpatient, category III for ambulatory), targeting 100%.
• Methodology:
  • Automated Reporting Validation: Use CMS guidelines to validate QRDA format compliance for all generated reports.
  • Use Case Testing: Conduct thorough testing to ensure all reports meet the specific requirements of inpatient and ambulatory care.
• Expected Outcome:
  • Standards Compliance: 100% of reports meet QRDA standards, providing full compatibility for quality reporting.

Care Settings Addressed

1. Outpatient Behavioral Health: Emphasis on QRDA standards tailored to outpatient clinical quality measurement reporting.
2. Residential Behavioral Health: Adapts QRDA documentation for inpatient settings, reflecting unique data requirements.

• Standard: United States Core Data for Interoperability (USCDI)
• Certification Criteria:
  • § 170.315(g)(7): Application Access – Patient Selection
  • § 170.315(g)(9): Application Access – All Data Request
  • § 170.315(g)(10): Standardized API for Patient and Population Services
• Method for Standards Update: Standards Version Advancement Process (SVAP)
• Relied Upon Software: EMR Direct – phiQuery HL7 FHIR Services

Testing Objectives

• Measurement/Metric:
  • Success rate of generating unique patient identifiers, targeting ≥ 99%.
  • Response time for retrieving patient data, with a target of ≤ 3 seconds per query.
• Methodology:
  • Automated Performance Monitoring: Track all patient selection requests, capturing success rates and response times.
  • System Logging and Error Tracking: Monitor any instances where patient identifiers are not generated or retrieved accurately.
• Expected Outcome:
  • System Reliability: ≥ 99% success rate for unique patient ID generation and retrieval.
  • Performance: Average response time for patient data retrieval is ≤ 3 seconds, ensuring quick access.

• Measurement/Metric:
  • Accuracy and completeness rate of data retrieval for patient summaries in CCD format, targeting 100%.
  • Success rate in retrieving data across all USCDI data categories, with less than 1% failure rate.
• Methodology:
  • Automated Data Verification: Run consistency checks to verify that all requested USCDI data elements are included in retrieved summaries.
  • System Performance Monitoring: Track data retrieval requests, monitoring for accuracy and compliance with USCDI standards.
• Expected Outcome:
  • Data Completeness: 100% of data requests return CCD-formatted summaries meeting USCDI standards.
  • System Accuracy: Less than 1% error rate in data retrieval, ensuring comprehensive patient data access.

• Measurement/Metric:
  • Success rate for handling multiple patient data requests in parallel, with a target of ≥ 98%.
  • Response times for both single and multiple patient data requests, with ≤ 5 seconds for bulk requests.
• Methodology:
  • API Load Testing: Test API with simultaneous patient data requests to ensure consistent performance under load.
  • System Log Analysis: Record and analyze request success rates and response times to measure efficiency and reliability.
• Expected Outcome:
  • Performance Efficiency: ≥ 98% success rate for handling both single and multiple data requests.
  • Speed of Response: Response time for multiple patient data requests is ≤ 5 seconds, meeting industry standards for API performance.

Care Settings Addressed

1. Outpatient Behavioral Health: Unique documentation requirements based on outpatient workflows.
2. Residential Behavioral Health: Testing in residential settings to account for varying documentation and data exchange needs.

• Standard: United States Core Data for Interoperability (USCDI)
• Certification Criteria:
  • § 170.315(h)(1): Direct Project
• Method for Standards Update: Standards Version Advancement Process (SVAP)
• Relied Upon Software: EMR Direct – phiMail Direct Messaging

Testing Objectives

• Measurement/Metric:
  • Success rate of sending and receiving health information via Direct Messaging, targeting ≥ 98% success across all attempts.
  • Average time taken to send and receive Direct messages, with a target of ≤ 3 seconds per transaction.
• Methodology:
  • Automated System Monitoring: Track the success rate of message transmission and receipt to ensure seamless interoperability.
  • Performance Logging: Log and measure response times for each transaction, ensuring efficiency within the specified timeframe.
  • Error Analysis: Capture and analyze any messaging failures, tracking error rates and root causes.
• Expected Outcome:
  • System Reliability: ≥ 98% of Direct messages are successfully sent and received, with response times of ≤ 3 seconds per transaction.
  • Compliance with Standards: All transmitted messages adhere to the Applicability Statement for Secure Health Transport, Version 1.2.

• Measurement/Metric:
  • Accuracy rate for delivery notification transmission, targeting 100% compliance with Version 1.0 of the Implementation Guide (IG) for Delivery Notification.
  • Rate of confirmed receipt notifications, targeting ≥ 99% reliability
• Methodology:
  • Automated Compliance Verification: Continuously track and validate that all delivery notifications are sent in compliance with Direct Messaging standards.
  • Notification Success Tracking: Log each notification event, confirming that messages are reliably transmitted and received by third parties.
  • System Error Monitoring: Identify and document any notification failures to maintain high reliability.
• Expected Outcome:
  • Notification Reliability: 100% of delivery notifications meet Direct Messaging standards, ensuring consistent and dependable communication.
  • High Performance: Delivery notifications achieve ≥ 99% confirmed receipt rate, reflecting the system's dependability in real-world settings.

Care Settings Addressed

1. Outpatient Behavioral Health: Supports workflows where messaging and information exchange follow outpatient standards.
2. Residential Behavioral Health: Adapts for inpatient care requirements, reflecting operational differences in data sharing processes.

---

1. Executive Summary

Zoobook Systems LLC conducted Real World Testing throughout 2025 to validate that our Zoobook EHR meets interoperability and functionality requirements in production settings. This report presents our 2025 Real World Testing results, covering:

• Transitions of Care
• Electronic Health Information (EHI) Export
• Clinical Quality Measures (CQMs)
• Application Access (API)
• Direct Project

Data was collected from a live behavioral health care treatment environment. The results indicate strong performance across all measured criteria, with minimal corrective actions required.

---

2. Testing Methodology

2.1 Approach/Methods

• Data Collection: We aggregated usage logs, transaction counts, and error data from live environments.
• Analysis & Thresholds: Each measure had predefined success thresholds from the Real World Test Plan.
• Site Involved: A live behavioral health care treatment environment.

2.2 Timeline

• Testing Period: January – December 2025.
• Quarterly Reviews: Data was validated and analyzed at the end of each quarter (Q1, Q2, Q3, Q4).
• Final Compilation: Results finalized in January 2026.

---

3. Key Measures & Associated Certification Criteria

The table below maps each Real World Test category to relevant certification criteria and metrics.

Category	Certification Criteria	Metrics Measured	Thresholds
1. Transitions of Care	170.315(b)(1)	- % of successful C-CDA (Continuity of Care Document) creations, sends, and receives - % of successful XDM package processing - % of readable C-CDA documents with CSS rendering - Frequency of transmission errors	≥95% success rate ≥90% success for XDM 100% readability with CSS
2. EHI Export	170.315(b)(10)	- % of error-free EHI exports - Average export completion time - Role-based access control compliance - Accuracy of date range configurations - Batch export efficiency - Transmission success across endpoints	≥98% success rate ≤2 min/export ≥99% transmission success
3. Clinical Quality Measures	170.315(c)(1)/(c)(2)/(c)(3)	- % of accurate CQM data recording - % of QRDA-compliant exports - Compliance of imported files with standards - Accuracy of CQM calculations - Success rate for ambulatory and inpatient reporting	≥99% data accuracy ≤1% export error state 100% compliance with CMS guidelines
4. Application Access (API)	170.315(g)(7)/(g)(9)/(g)(10)	- % of successful patient data retrieval (single and bulk) - Average response time for data requests - Accuracy of FHIR-based data retrieval	≥99% success for individual requests ≤5 seconds for bulk requests ≤1% error rate
5. Direct Project	170.315(h)(1)	- % of successful Direct messages sent and received - % of accurate delivery notifications - Average time for message transmission	≥98% messaging success rate 100% delivery notification compliance ≤3 seconds/message

---

4. Care Settings

Real World Testing was completed in a live behavioral health care treatment environment, comprised of 144 clients representing a variety of mental health and substance abuse conditions and severities. Treatment protocols involve a wide range of clinical workflows, ensuring Real World Testing reflects varied clinical scenarios.

---

5. Results & Analysis

5.1 Transitions of Care

• Certification Criteria: § 170.315(b)(1)
• Metrics Measured:
  • Percentage of successful C-CDA (Continuity of Care Document) creations, sends, and receives.
  • Frequency of transmission errors over the testing period.
  • Accuracy in handling XDM package and C-CDA attachments (e.g., MIME type recognition).
  • Readability and CSS rendering of C-CDA documents.
• Methodology:
  • System performance logging to monitor successful vs. failed transmission attempts.
  • Automated testing for XDM attachments and MIME type accuracy.
  • Validation of CSS rendering and readability across sampled C-CDA documents.
• Expected Outcomes:
  • ≥ 95% success rate for document exchange.
  • ≥ 90% success rate in processing XDM and C-CDA attachments.
  • 100% readability with CSS rendering, with issues affecting < 1% of documents.

---

5.2 Electronic Health Information (EHI) Export

• Certification Criteria: § 170.315(b)(10)
• Metrics Measured:
  • Success rate of error-free EHI exports (target ≥ 98%).
  • Average completion time for exports (target ≤ 2 minutes per export).
  • Role-based access control success for authorized vs. unauthorized users.
  • Accuracy in applying date ranges for exports (target ≥ 98%).
  • Batch export efficiency (target ≤ 1.5x the time for individual exports).
  • Transmission success rate across storage endpoints (target ≥ 99%).
• Methodology:
  • System log analysis for tracking successful vs. failed exports.
  • Automated timing and access control testing.
  • Validation of exported data integrity and batch processing efficiency.
• Expected Outcomes:
  • ≥ 98% success rate for error-free exports.
  • ≤ 2 minutes per individual export; ≤ 1.5x for batch exports.
  • ≥ 99% transmission success across storage endpoints.

---

5.3 Clinical Quality Measures (CQMs)

• Certification Criteria: § 170.315(c)(1), (c)(2), (c)(3)
• Metrics Measured:
  • Accuracy rate of recorded CQM data (target ≥ 99%).
  • Success rate of data exports in QRDA format (≤ 1% error rate).
  • Compliance of imported CQM files with §170.205(h)(2) standards.
  • Accuracy of CQM calculations validated against expected outcomes.
  • Reporting success rate for inpatient (Category 1) and ambulatory (Category III) reports (target 100%).
• Methodology:
  • Automated data integrity checks and file import validation.
  • Calculation consistency checks for sample cases.
  • CMS guideline compliance for QRDA reporting.
• Expected Outcomes:
  • ≥ 99% data accuracy.
  • 100% compliance with CMS guidelines for reporting formats.

---

5.4 Application Access (API)

• Certification Criteria: § 170.315(g)(7), (g)(9), (g)(10)
• Metrics Measured:
  • Success rate for patient selection and retrieval (target ≥ 99%).
  • Average response time for individual data requests (≤ 3 seconds).
  • Accuracy of FHIR-based requests for all USCDI data categories (≤ 1% error rate).
  • Success rate for bulk data requests (≥ 98%).
  • Response time for bulk requests (≤ 5 seconds).
• Methodology:
  • Performance monitoring for patient data selection and retrieval.
  • API load testing for single and bulk requests.
  • Consistency checks for FHIR-based data completeness and USCDI compliance.
• Expected Outcomes:
  • ≥ 99% success rate for individual data requests.
  • ≤ 5 seconds for bulk requests.

---

5.5 Direct Project

• Certification Criteria: § 170.315(h)(1)
• Relied Upon Software: EMR Direct – phiMail Direct Messaging
• Metrics Measured:
  • Success rate for Direct messages sent and received (target ≥ 98%).
  • Average time for message transmission (≤ 3 seconds per transaction).
  • Accuracy rate for delivery notifications (target 100%).
  • Rate of confirmed receipt notifications (target ≥ 99%).
• Methodology:
  • Automated monitoring of message transmission success and failures.
  • Performance logging for transaction timing.
  • Validation of delivery notifications against standards.
• Expected Outcomes:
  • ≥ 98% success rate for Direct messaging.
  • Delivery notifications achieve ≥ 99% confirmed receipt rate.

---

6. Standards Updates

• Standards & Versions:
  • FHIR R4 for Application Access (API)
  • CDA R2 (for C-CDA document exchange in Transitions of Care)
  • DirectTrust/Direct Project standards and certificates

No significant changes to these standards were introduced during 2025 that impacted real-world usage. Zoobook EHR remained aligned with the existing versions and updated measure specs as needed.

---

7. Key Achievements & Challenges

Achievements

1. High Success Rates Across All Categories: Each tested criterion met or exceeded thresholds.
2. Reduced Error Rates for CQMs: Streamlined measure library updates significantly decreased mismatch errors.
3. Improved User Training & Documentation: Proactive user education continues to reduce configuration errors.

---

8. Future Steps & Continuous Improvement

1. Further Optimization: Although the module performed well overall, continued optimization of database queries may enhance performance for large data sets.
2. Monitor Performance: Ongoing monitoring is recommended for API response times in production environments to ensure continued compliance.