iCare.com

REAL WORLD TESTING RESULTS 2022

GENERAL INFORMATION

Plan Report ID Number: [For ONC-Authorized Certification Body use only]

Developer Name:  iCare.com LLC

Product Name(s):  iCare EHR

Version Number(s):  V 2.0

Certified Health IT Product List (CHPL) ID(s): 15.04.04.2617.iCar.02.00.1.200220

Developer Real World Testing Page URL:  https://icare.com/realworldtesting

SUMMARY OF TESTING METHODS AND KEY FINDINGS

The iCare.com EHR is a pure cloud product, therefore our Real World Testing plan included considerations for our delivery methodology which differs significantly from on-premise (or simply hosted) software.  The iCare cloud environment permitted us to monitor and test most criteria of our certified technology without the requirement to involve our customers with significant portions of the testing methodologies.

The annual Real World Testing for 2022 was performed by iCare.com using metrics from our production environment which serves actual customers using our software to manage the healthcare of real patients.  Customers participated in testing those criteria which require their involvement.

During Real World Testing activities, we anticipated that there could be instances where we may find a need to modify our testing methodologies, approaches, or expected outcomes that were originally laid out in our plans to address unexpected changes during the testing period.  In such instances, we would adjust our Real World Testing approaches and our results report will include a description of these types of changes, the reasons for them, and how intended outcomes were more efficiently met as a result.  No modifications to the original testing plan were required.

Please note that the Real World Testing Plan referenced in the CMS CHPL may not reflect the currently implemented plan.

The following list is a summary of the Real World Testing Approach.

The live production environment was used for Real World Testing.  Both hospital and clinic settings were included in the testing.  Customers used the iCare system as usual and customer involvement was limited to those areas where it was necessary.  Data was recorded quarterly. We also used the EHR Measure Report (g1 g2) to generate data.The received CCDA reports were validated, incorporated, and reconciled and the measure reports were reviewed. The Data Export functionality was tested periodically as required.Logs were analyzed to measure successful messages sent for Syndromic Surveillance.ELR – Checked for data sent from the production environment.e-Prescribe – the production system connection to Surescripts was monitored and the e-prescribe measure report was generated.CQM – CMS 72, CMS 105, CMS 108 and CMS 111 was tested in the live production environment.

WORLD TESTING PLAN TEMPLATE

STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS (SVAP) AND UNITED STATES CORE DATA FOR INTEROPERABILITY (USCDI))

Both required and voluntary standards updates were addressed in the Real World Testing plan. The Real World Testing plan included all certified health IT updated to newer versions of standards prior to August 31 of the year in which the updates were made.

The following indicates as to whether optional standards, via SVAP and/or USCDI, are leveraged as part of the certification of our health IT product(s):

[X] Yes, I have products certified with voluntary SVAP or USCDI standards. (If yes, please complete the table below.

[  ] No, none of my products include these voluntary standards.

Standard (and version) USCDI V.1
Updated certification criteria and associated product Cures Update, iCare EHR Version 2.0
CHPL Product Number 15.04.04.2617.iCar.02.00.1.200220
Conformance measure  g10, g9, b1, b2

Care Setting(s) 

The expectation is that our Real World Testing was conducted within each type of clinical setting in which our certified health IT is marketed. While health IT developers are not required to test their certified health IT in every setting in which it is marketed for use, we tested in various settings.

The following care settings were included in testing.

HospitalAmbulatory & Emergency CareClinicPhysician Practice

Relied Upon Software

The following third party software was relied upon to perform portions of the Real World Testing for several measures:

  1. popHealth

170.315(c)(1) – record and export

170.315(c)(2) – import and calculate

170.315(c)(3) – View Report

  • phiMail Server

170.315(h)(1) – Direct Project

Metrics and Outcomes 

We have detailed outcomes from our testing that successfully demonstrate that our certified health IT:

  • is compliant with the certification criteria, including the required technical standards and vocabulary code sets;
  • is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and/or,
  • EHI is received by and used in the certified health IT.

This section describes how the specific data collected from their Real World Testing measures demonstrate our results.

Measurement/Metric  Associated Criterion(a)  Relied Upon Software (if applicable)  Outcomes  Challenges Encountered(if applicable) 
170.315(b)(2)-Clinical Information Reconciliation and Incorporate 170.315(b)(1) Transitions of care,170.315(b)(7) Security tags – summary of care – send, 170.315(b)(8) Security tags – summary of care – receive170.315(h)(1) Direct Project Generated and analyzed the G1/G2 reports for 3rd and 4th quarter in the production environment. There were no errors or issues reported for invalid formats. Summary of care, Transition of care sent (b.1, b.7) and received (b.8, h.1) did not produce sufficient data for statistically significant results. The functionality was tested in the production environment and found to produce no errors for clinical reconciliation (b.2).
170.315(b)(3)- e-Prescribe Sure-scripts e-Prescriptions in Ambulatory and Hospital sites were analyzed.In the 3rd Quarter, e-Prescriptions were submitted and received electronic acknowledgements for all with a 100% success rate.In the 4th quarter e-Prescriptions submissions and acknowledgements were analyzed with a 100% success rate.
170.315(b)(6)- Data Export For the 3rd and 4th quarter, logs were analyzed showing bulk patient data export had a 100% success rate with no errors shown.
170.315- c(3) – View Report 170.315(c)(1)—record and export170.315(c)(2)—import and calculate For CQM measures 105, 108, 72 and 111 no data was generated by customers during the 3rdand 4th quarters, but this functionality was tested in the production environment with no errors noted.
170.315(e)(1)- View, Download and Transmit to third party The G1/G2 report for the 3rd and 4th quarters was generated and analyzed. Access to the patient portal was provided to 93.2% of eligible patients. Data and functionality for view, download and transfer was analyzed and tested. The error rate was found to be 0%.
170.315(f)(2)-Transmission to Public – Syndromic Surveillance During the 3rd and 4th quarter, Messages for Syndromic Surveillance were checked and calculated. 99.7% messages were sent successfully.
170.315(f)(3)-Transmission to Public- Reportable Laboratory Test Values/Results During the testing periods, the capability of the iCare system to transmit laboratory results to public agencies was tested in the production environment with no errors noted.
170.315(g)(9)- Application Access- All Data Request 170.315(g)(7) Application access—patient selection170.315(g)(8) Application access—data category request Generated and analyzed the G1/G2 reports for the 3rd and 4th quarters. No data generated for this measure as there was no data for access of the application through API during this period. We performed testing of this functionality in the production environment with no errors noted.

KEY MILESTONES

The following is a list of key milestones that were met during the Real World Testing process.  Details are included on how and when we implemented measures and collected data.

Key Milestone  Care Setting  Date/Timeframe 
During the 3rd Quarter of 2022, the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients. Ambulatory and Hospital 1st July – 31st Oct 2022
We started report generation during the first week of October 2022 and evaluation was performed as applicable for the 3rd quarter. Ambulatory and Hospital First week of October 2022
During the 4th Quarter of 2022 the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients. Ambulatory and Hospital 1st Oct – 31st Dec 2022
We started final report generation during the first week of Jan 2023 and performed evaluation and analysis as applicable for the 4th quarter. Ambulatory and Hospital First week of January 2023
Upload of final report Ambulatory and Hospital First week of February 2023