iCare.com
REAL WORLD TESTING PLAN 2022
GENERAL INFORMATION
Plan Report ID Number: [For ONC-Authorized Certification Body use only]
Developer Name: iCare.com LLC
Product Name(s): iCare EHR
Version Number(s): V 2.0
Certified Health IT Product List (CHPL) ID(s): 15.04.04.2617.iCar.02.00.1.200220
Developer Real World Testing Page URL: https://icare.com/realworldtesting
JUSTIFICATION FOR REAL WORLD TESTING APPROACH
The iCare.com EHR is a pure cloud product, therefore our Real World Testing plan includes considerations for our delivery methodology which differs significantly from on-premise (or simply hosted) software. The iCare cloud environment permits us to monitor and test most criteria of our certified technology without the requirement to involve our customers with significant portions of the testing methodologies.
The annual Real World Testing will be performed by iCare.com using metrics from our production environment which serves actual customers using our software to manage the healthcare of real patients. Customers will participate in testing those criteria which require their involvement.
During Real World Testing activities we anticipate that there will be instances where we may find a need to modify our testing methodologies, approaches, or expected outcomes that were originally laid out in our plans to address unexpected changes during the testing period. In such instances, we will adjust our Real World Testing approaches and our results report will include a description of these types of changes, the reasons for them, and how intended outcomes were more efficiently met as a result. Please note that the Real World Testing Plan referenced in the CMS CHPL may not reflect the currently implemented plan.
The following list is a summary of the Real World Testing Approach.
| The live production environment will be used for Real World Testing. Customers will use the system as usual and customer involvement will be limited to those areas where it is necessary. Data will be recorded quarterly. We can also use the EHR Measure Report (g1 g2) to generate data.The received CCDA will be validated, incorporated and reconciled and the measure report will be reviewed. The Data Export functionality will be tested periodically as required.Logs will be analyzed to measure successful messages sent for Syndromic Surveillance.ELR – Check for data sent from the production environment.e-Prescribe – the production system connection to Surescripts will be monitored and the e-prescribe measure report will be generated.CQM – CMS 72, CMS 105, CMS 108 and CMS 111 will be tested in the live production environment. |
STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS (SVAP) AND UNITED STATES CORE DATA FOR INTEROPERABILITY (USCDI))
Not Applicable
MEASURES USED IN OVERALL APPROACH
This Real World Testing Plan includes at least one measurement/metric that addresses each applicable certification criterion in the iCare.com EHR’s scope of certification. The method for measuring how the approach(es) chosen meet the intent and purpose of Real World Testing are considered for each item.
For each measurement/metric, the following elements are described below:
✔ Description of the measurement/metric
✔ Associated certification criteria
✔ Justification for selected measurement/metric
✔ Care setting(s) that is addressed
✔ Expected outcomes
Description of Measurement/Metric
The following measures will be used to support the overall approach to Real World Testing.
| Measurement/Metric | Description |
| 170.315(b)(1) | b2 is dependent on this measure- therefore it is considered as done when b1 is doneValidate the Received CCDA on Secure Messaging from third parties.Send a file to an external health information system and mark acknowledged on Edit/Discharge Screeng1/g2 report will be reviewed. |
| 170.315(b)(2) | Clinical Information Reconciliation and Incorporate- Number of successful reconciliation for Medication List, Problem List and the Allergy list in the Production environment will be evaluated quarterly |
| 170.315(b)(6) | Data Export – Will analyze archived data transmissions and logs in the production environment. |
| 170.315(c)(1)-(3) | Record and Export – CQM data will be entered and the file export analyzed, Report – % achieved |
| 170.315(e)(1) | View Download Transmit to third party-CCDA from Patient Portal- Number of Patients with web access and who viewed, downloaded or transmitted the CCDA will be evaluated quarterly. |
| 170.315(f)(2) | Transmission to Public Agencies – Syndromic Surveillance- Number of successful ADT messages sent to the Public agency will be evaluated. |
| 170.315(f)(3) | Transmission to Public Agencies – Reportable LaboratoryTests and Values/results |
| 170.315(g)(7) | Application Access – Patient Selection- Patient with webaccess an activated – will evaluate same as e(1) |
| 170.315(g)(8) | Application Access- Data Category- Patient with webaccess an activated – will evaluate same as e(1) |
| 170.315(g)(9) | Application Access – All data request-Patient with webaccess an activated – will evaluate same as e(1) |
| 170.315(h)(1) | Direct Project- Number of CCDA sent via direct messaging to another care setting will be evaluated. |
Associated Certification Criteria
The following certification criteria are associated with the measures.
| Measurement/Metric | Associated Certification Criteria |
| 170.315(b)(2) Clinical informationreconciliation and incorporation | 170.315(b)(1) Transitions of care, 170.315(b)(7) Security tags – summary of care – send, 170.315(b)(8) Security tags – summary of care – receive170.315(h)(1) Direct Project |
| 170.315(c)(3)—View Report | 170.315(c)(1)—record and export170.315(c)(2)—import and calculate |
| 170.315(g)(9) Application access—all data reque32t | 170.315(g)(7) Application access—patient selection170.315(g)(8) Application access—data category request |
Justification for Selected Measurement/Metric
The following section presents explanations for each measurement/metric selected to conduct Real World Testing.
| Measurement/Metric | Justification |
| 170.315(b)(1) | The CCDA documents will be received in and sent from the application, this process can be monitored and evaluated. |
| 170.315(b)(2) Clinical informationreconciliation and incorporation | This scenario allows us to see the real time performance of the system for receiving, incorporating and reconciliation of data in the application. To perform Reconciliation, the CCDA document will be received and incorporated. To send that CCDA to another environment the same product can be used, so all the related measures can be tested with this measure. |
| 170.315(b)(3) | We will monitor transactions in the actual production environment which will allow us to determine the success rates of both sending and receiving parties. |
| 170.315(b)(6) | Export of the Patient data in CCDA format is done, which can be evaluated periodically. |
| 170.315(c)(1) | The recorded data for each CQM is exported to CQM processor when the date range is applied, this measure helps to demonstrate the adherence to the CQM being followed by the users and the capability of the application. |
| 170.315(c)(2) | The import of the processed data files and calculation of the CQM measures are done on a simple UI which is user friendly and accurate. |
| 170.315(c)(3)—View report | The report is viewed as per decided period and this helps to demonstrate the ease in getting the CQM reports. Once the export is done the user receives a message for the completion of the process to go back and look into the report. |
| 170.315(e)(1) | This measure helps to see the accessibility to the portal for the Users. The documents and the clinical information shared with consumers can be evaluated. |
| 170.315(f)(2) | The capability of the application to transmit ADT data to the public agency can be successfully demonstrated. |
| 170.315(f)(3) | The capability of the application to transmit Laboratory results to public agencies is successfully demonstrated with this measure. |
| 170.315(g)(7) | The consumer can access the application and get the data, this is the consumer oriented facility, g7 specifically allows the access by name after personalised identification methods. So security of the system can also be demonstrated with this measure. |
| 170.315(g)(8) | The user can access data by category, which specified the need rather than bringing entire data. Testing this measure helps in demonstrating secured access. |
| 170.315(g)(9) Application access—all data request | All data requested through API, brings the CCDA documents, the count of the documents will demonstrate the success of the measure in Real World Testing. |
| 170.315(h)(1) | This measure helps understand the inbound and outbound messages /documents transaction success rate. |
Care Setting(s)
Our Real World Testing plan will address each type of clinical setting in which our certified health IT is marketed.
| Care Setting | Justification |
| AcuteAmbulatoryEmergency Department | Unless otherwise noted, the care settings for testing of each measurement will include one or more of these care settings. |
Expected Outcomes
We will detail how the approaches chosen will successfully demonstrate that the certified health IT:
- is compliant with the certification criteria, including the required technical standards and vocabulary codes sets; –
- is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and/or,
- EHI is received by and used in the certified health IT. (from 85 FR 25766)
Not all of the expected outcomes listed above will be applicable to every certified Health IT Module
| Measurement/Metric | Expected Outcomes |
| 170.315(b)(1)- Transition of Care | % achieved (g1/g2)-CCDA received and used in the iCare EHR for encounters will be monitored through a report, using g1/g2 measure generation. |
| 170.315(b)(2)-Clinical Information Reconciliation and Incorporate | % achieved (g1/g2) – CCDA received and used in the iCare EHR for encounters will be monitored through a report, using g1/g2 measure generation. |
| 170.315(b)(3)- e-Prescribe | % achieved (g1/g2) – The ePrescriptions sent from and RxRenewal/RxChange requests received in the system will be monitored. Report(s) will be generated using the g1/g2 measure and a high percentage of successful transmissions should be observed. |
| 170.315(b)(6)- Data Export | Export data will be assessed and measured for a high rate of successful delivery of the data files through the SFTP servers. |
| 170.315(c)(1)- (3) – Record and Export, Import & Calculate and Report | % achieved CQM – The qualified eCQM records will be sent to the CQM processor and the calculated outcome will be available in percentage. This will be monitored and a report will be analyzed for a very high rate of accuracy and success. |
| 170.315(e)(1)- View, Download and Transmit to third party | % achieved (g1/g2) – EHI is accessed by patient users in the iCare Patient Portal and sent to other electronic health systems. This process will be monitored. The g1/g2 module in the application generates reports periodically, This report will be generated and analyzed during the testing period.. |
| 170.315(f)(2)-Transmission to Public – Syndromic Surveillance | The ADT messages for encounters which are transmitted to public agencies will be evaluated during the testing period and a high rate of success is expected. Failed messages, if any, will also be captured and analyzed. |
| 170.315(f)(3)-Transmission to Public- Reportable Laboratory Test Values/Results | The Laboratory Results in HL7 messages transmitted to a public agency, will be monitored and analyzed during the testing period. A high rate of successful transmissions is expected.. |
| 170.315(g)(7)- Application Access- Patient Selection | % achieved (g1/g2)- EHI is accessed by the user using the API and sent to other electronic health systems..The process will be monitored during the testing period. The g1/g2 module in the application generates reports periodically, A high rate of successful transmissions is expected. |
| 170.315(g)(8)- Application Access- Data Category | % achieved (g1/g2)- EHI is accessed by the user using the API and sent to other electronic health systems..The process will be monitored during the testing period. The g1/g2 module in the application generates reports periodically, A high rate of successful transmissions is expected. |
| 170.315(g)(9)- Application Access- All Data Request | % achieved (g1/g2)- EHI is accessed by the user using the API and sent to other electronic health systems..The process will be monitored during the testing period. The g1/g2 module in the application generates reports periodically, A high rate of successful transmissions is expected. |
| 170.315(h)(1)- Direct Project | A high success rate for transmissions of all inbound and outbound messages will be expected for this measure. |
SCHEDULE OF KEY MILESTONES
| Key Milestone | Implementation – How | Care Setting | Date/Time Frame |
| Have the RWT plan ready and upload to certifying body | Create the Plan and submit to ACB | 15th November 2021 | |
| Test Plan review | Response from ACB expected | December 2021 | |
| Start Real World Testing for those measures on Production Environment | Start capturing data | Acute, ambulatory, and emergency department | From Jul – Sept 2022 |
| Start Real World Testing for those measure requiring customer involvement | Production environment | Acute, ambulatory, and emergency department | From Jul – Sept 2022 |
| Data Collection, Data Recording for measurement period | Production Environment | Acute, ambulatory, and emergency department | 1st Week of Oct 2022 |
| Start Real World Testing for those measures in production environment | Analyze data from production logs and internal reports | Acute, ambulatory, and emergency department | From Oct – Dec 2022 |
| Start Real World Testing for those measure requiring customer involvement | Coordinate with customers | Acute, ambulatory, and emergency department | From Oct – Dec 2022 |
| Data Collection, Data Recording for Second quarter | From production environment | 1st Week of Jan,2023 | |
| Report upload | Feb 2023 |
ATTESTATION
This Real World Testing plan is complete with all required elements, including measures that address all
certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements.
Authorized Representative Name: James Riley
Authorized Representative Email: jim@icare.com
Authorized Representative Phone: 800-784-8045
Authorized Representative Signature: //James Riley//
Date: 12/06/2021
i Certified health IT continues to be compliant with the certification criteria, including the required technical standards and vocabulary codes sets; certified health IT is exchanging EHI in the care and practice settings for which it is marketed for use; and EHI is received by and used in the certified health IT. (85 FR 25766)