Real World Test Plans 2022

  • The measurements used for our Real World Testing Plan are listed below.
    • Associated Certification Criteria
    • Measurement/Metric
    • Description
    • Justification
    • Expected Outcome
    • Test Method*
  • *Test Method
    Each Measurement/Metric uses a testing methodology. Here are the testing methodologies that will
    be used for this Test Plan:

    • Internal Testing – Members of the development team or organization will be users insofar
      as they have the ability and motivation to voice and communicate problems.
    • Survey – This method evaluates the compliance of the Health IT Module to the following
      certification criteria(s) by gathering end user feedbacks.
Associated Certification Criteria Measurement/Metric Description Test Method Justification Expected Outcome
§ 170.315 (b)(1): Transitions of Care Transition of Care Functionality Provider identified users are able to create transitions of care, validate and
display C-CDA, send and receive summaries from the Health IT module.		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module.
Internal testing will also be done to ensure expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module must be capable of receiving and sending message via SMTP and
transmission should be encrypted by TLS
Must have the capability to receive one or more attachments such as C-CDA documents
and XDM packages. Our EHR must be able to process and recognize XDM mime types
Must be capable of sending/receiving XDR message with Full (XDS) metadata (IHE XDR
Profile for Limited Metadata )
Health IT Module must be able to validate the format correctness of C-CDA documents
Health IT Module must be able to render C-CDA Documents and XDM packages with
support for styling via css stylesheets for human readability
Health IT Module must have the capability to create and send valid C-CDA documents
via SMTP Edge Protocol under a TLS connection
Must ensures that the required Data Elements for different settings are included in
generated C-CDA’s
§ 170.315 (b)(6): Data Export C-CDA Data Export Functionality Provider identified users are able to successfully export user-identified client
C-CDA files for user-defined date/time ranges from the Health IT module.		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module.
Internal testing will also be done to ensure expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
Provider identified users can create export summaries in real-time; based on a
relative date and time or a specific date and time.
Provider identified users can set configuration options for data elements, date and
time ranges, and locations when generating a C-CDA
Provider identified users can generate a set of C-CDAs for between one client and
all clients, without generating each C-CDA individually
C-CDA generation is limited to those users who have the proper role or system
credentials, as determined by the provider
Provider identified users not having proper system credentials cannot generate
C-CDAs. Applies to:

  • Document creation configuration
  • Timeframe configuration
  • Location Configuration

Only authorized users can set the limits of users who can do the above.
C-CDAs may be exported to local or network storage, in addition to integrated
transmission methods
§ 170.315 (c)(1): Clinical Quality Measures – Record and Export Clinical quality measures (CQM) functionality Provider identified users are able to record and export all of the data that would
be necessary to calculate each CQM		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module must be able to record all of the data that would be necessary
to calculate each CQM.
Must be able to export a data file at any time the user chooses and without
subsequent developer assistance to operate:

  • Formatted in accordance with the standard specified in §170.205(h)(2);
  • Ranging from one to multiple patients; and
  • That includes all of the data captured for each and every CQM to which
    technology was certifiaed under paragraph (c)(1)(i) of this section.
§ 170.315 (c)(2): Clinical quality measures import and calculate Clinical quality measures (CQM) functionality Provider identified users are able to import a data file and use it to calculate all
of the data that would be necessary to each CQM		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The user must be able to import a data file in accordance with the standard
specified in §170.205(h)(2) for one or multiple patients and use such data to
perform the capability specified in paragraph (c)(2)(ii) of this section. A user
must be able to execute this capability at any time the user chooses and without
subsequent developer assistance to operate
The Health IT Module can calculate each and every clinical quality measure for which
it is presented for certification.
§ 170.315(c)(3) Clinical quality measures report Clinical quality measures (CQM) functionality Provider identified users are able to generate CQM data files for transmission based
on appropriate QRDA guidelines.		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The user must be able to electronically create a data file for transmission of
clinical quality measurement data:

(i) In accordance with the applicable implementation specifications specified by the
CMS implementation guides for Quality Reporting Document Architecture (QRDA),
category I, for inpatient measures in § 170.205(h)(3) and CMS implementation guide
for QRDA, category III for ambulatory measures in § 170.205 (k)(3); or

(ii) In accordance with the standards specified in § 170.205(h)(2) and §
170.205(k)(1) and (2) for the period until December 31, 2022.
§ 170.315 (g)(7) Application access – patient selection Application access functionality An API must be able to receive a request with sufficient data to uniquely identify a
patient and return an ID or other token that can be used by an application to
subsequently execute requests for that patient’s data		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module will uniquely identify a patient and return an ID or other
token that can be used by the application to subsequently execute requests for that
patient’s data.
§ 170.315 (g)(8) Application access – data category request Application access functionality Upon receipt of an ID/Token, the API will return the requested data specified in the
CCDS in full and in a computable format, and respond in regard to specific date
requests as well as ranged requests.		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module must be able to respond to requests for patient data (using an
ID or other token) for each of the individual categories listed in the Common
Clinical Data Set (CCDS) and return the full set of data for that category,
according to the required data standards in a computable format.
The Health IT Module must be able to respond to requests for patient data associated
with a specific date as well as requests for patient data with a specific date
range.
§ 170.315 (g)(9) Application access – all data request Application access functionality Upon receipt of an ID/Token, the API will return the requested data in a summary
format following CCD documentation template guidelines, or provide the CCDS, and
address assessment and plan of treatment, goals, health concerns, and unique device
identifiers for any implantable devices.		 Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module must be able to respond to requests for patient data (using an
ID or other token) for all of the data categories specified in the United States
Core Data for Interoperability Standard (USCDI) at one time in a summary record
formatted according to the Consolidated CDA Release 2.1 Continuity of Care Document
(CCD) template.
The Health IT Module must be able to respond to requests for patient data associated
with a specific date as well as with a specific date range.
§ 170.315 (h)(1) Direct Project Direct Project functionality Provider identified users are able to send and receive health information Survey, Internal Testing This measure will survey end users to determine real world interoperability and
usability of the Health IT module. Internal testing will also be done to ensure
expected outcomes are met.		 Positive end user feedback will provide a benchmark to determine real-world
interoperability and usability of the Health IT Module:
The Health IT Module can electronically transmit (send and receive) health
information to a 3rd party which must be formatted only as a “wrapped” message using
the Applicability Statement for Secure Health Transport, Version 1.2.
The Health IT Module can electronically transmit (send and receive) health
information to a 3rd party using Direct in accordance with the Implementation Guide
(IG) for Delivery Notification in Direct, Version 1.0.
Associated Certification Criteria Care Setting Justification
§ 170.315 (b)(1): Transitions of Care Outpatient behavioral health Outpatient (ambulatory) behavioral health providers is able to create a C-CDA that
also includes the reason for referral, and the referring or transitioning provider’s
name and office contact information.
Residential behavioral health Residential behavioral health providers is able to create a C-CDA Discharge Summary
Document that includes reason for referral; and referring or transitioning
provider’s name and office contact information; and discharge instructions.
§ 170.315 (b)(6): Data Export Outpatient behavioral health Outpatient (ambulatory) behavioral health providers frequently provide C-CDA
documentation to outside entities. Outpatient care C-CDA documentation requires
reason for referral; and referring or transitioning provider’s name and office
contact information.
Residential behavioral health Residential behavioral health providers frequently provide C-CDA documentation to
outside entities. Residential CCDA documentation requires reason for referral; and
referring or transitioning provider’s name and office contact information; and
discharge instructions.
§ 170.315 (c)(1): Clinical Quality Measures – Record and Export
§ 170.315 (c)(2): Clinical Quality Measures – Import and Calculate
§ 170.315 (c)(3): Clinical Quality Measures – Report		 Outpatient behavioral health QRDA documentation is specific to outpatient behavioral health settings.
Residential behavioral health QRDA documentation is specific to inpatient behavioral health settings.
§ 170.315 (g)(7) Application access – patient selection
§ 170.315 (g)(8) Application access – data category request
§ 170.315 (g)(9) Application access – all data request		 Outpatient behavioral health Documentation requirements at the time of discharge are different from Residential
Behavioral Health setting requirements.
Residential behavioral health Documentation requirements at the time of discharge are different from Outpatient
Behavioral Health setting requirements.
§ 170.315 (h)(1) Direct Project Outpatient behavioral health While technically similar, operational processes differ from residential behavioral
health programs.
Residential behavioral health While technically similar, operational processes differ from outpatient behavioral
health programs.

Real World Test Plan Result

Measurement/Metric Associated Criterion Relied Upon Software
(if applicable)		 Outcomes Challenges Encountered
(if applicable)
Application access functionality § 170.315 (g)(7) Application access – patient selection
§ 170.315 (g)(8) Application access – data category request
§ 170.315 (g)(9) Application access – all data request		 phiQuery HL7 FHIR services API is be able to receive a request with sufficient data to uniquely identify a
patient and return an ID or other token that can be used by an application to
subsequently execute requests for that patient’s data

Practices Queried: 10
Practices Reporting
Results/Utilizing
Certification
Functionality: 6
Average Result: 8
Median Result: 8

 During our testing, we did not discover any errors or criteria non-conformities.
Our results reveal our EHR Module functionality is working as expected.
Clinical quality measures (CQM) functionality § 170.315 (c)(1): Clinical Quality Measures – Record and Export
§ 170.315 (c)(2): Clinical Quality Measures – Import and Calculate
§ 170.315 (c)(3): Clinical Quality Measures – Report		 Users are able to record and export and import all of the data that would be
necessary to calculate each CQM

Practices Queried: 10
Practices Reporting
Results/Utilizing
Certification Functionality: 5
Average Result: 12.5
Median Result: 7.5

 During our testing, we did not discover any errors or criteria non-conformities.
Our results reveal our EHR Module functionality is working as expected.
C-CDA Data Export Functionality § 170.315 (b)(6): Data Export Users are able to successfully export user-identified client C-CDA files for user-
defined date/time ranges from the Health IT module.

Practices Queried: 10
Practices Reporting
Results/Utilizing
Certification Functionality: 3
Average Result: 6.5
Median Result: 6.5

 During our testing, we did not discover any errors or criteria non-conformities.
Our results reveal our EHR Module functionality is working as expected.
Direct Project Functionality § 170.315 (h)(1) Direct Project phiMail Direct Messaging (HISP) service Users are able to send and receive health information

Practices Queried: 10
Practices Reporting
Results/Utilizing
Certification Functionality: 8
Average Result: 9
Median Result: 9

 During our testing, we did not discover any errors or criteria non-conformities.
Our results reveal our EHR Module functionality is working as expected.
Transition of Care Functionality § 170.315 (b)(1): Transitions of Care Users are able to create transitions of care, validate and display C-CDA, send and
receive summaries from the Health IT module.

Practices Queried: 10
Practices Reporting
Results/Utilizing
Certification Functionality: 7
Average Result: 8.5
Median Result: 8.5

 During our testing, we did not discover any errors or criteria non-conformities.
Our results reveal our EHR Module functionality is working as expected.