iCare.com

REAL WORLD TESTING RESULTS 2024

GENERAL INFORMATION

Plan Report ID Number: [For ONC-Authorized Certification Body use only] 

Developer Name:  iCare.com LLC

Product Name(s):  iCare EHR

Version Number(s):  V 2.0

Certified Health IT Product List (CHPL) ID(s): 15.04.04.2617.iCar.02.00.1.200220 

Developer Real World Testing Page URL:  https://icare.com/realworldtesting

SUMMARY OF TESTING METHODS AND KEY FINDINGS

The iCare.com EHR is a pure cloud product, therefore our Real World Testing plan included considerations for our delivery methodology which differs significantly from on-premise (or simply hosted) software.  The iCare cloud environment permitted us to monitor and test most criteria of our certified technology without the requirement to involve our customers with significant portions of the testing methodologies.

The annual Real World Testing for 2024 was performed by iCare.com using metrics from our production environment which serves actual customers using our software to manage the healthcare of real patients.  Customers participated in testing those criteria which require their involvement.

During Real World Testing activities, we anticipated that there could be instances where we may find a need to modify our testing methodologies, approaches, or expected outcomes that were originally laid out in our plans to address unexpected changes during the testing period.  In such instances, we would adjust our Real World Testing approaches and our results report will include a description of these types of changes, the reasons for them, and how intended outcomes were more efficiently met as a result.  No modifications to the original testing plan were required.

Please note that the Real World Testing Plan referenced in the CMS CHPL may not reflect the currently implemented plan.

The following list is a summary of the Real World Testing Approach.

  1. The live production environment was used for Real World Testing.  Both hospital and clinic settings were included in the testing.
  2. Customers used the iCare system as usual and customer involvement was limited to those areas where it was necessary.
  3. Data was recorded quarterly. We also used the EHR Measure Report (g1 g2) to generate data.
  4. The received CCDA reports were validated, incorporated, and reconciled and the measure reports were reviewed.
  5. The Data Export functionality was tested periodically as required.
  6. Logs were analyzed to measure successful messages sent for Syndromic Surveillance.
  7. ELR – Checked for data sent from the production environment.
  8. e-Prescribe – the production system connection to Surescripts was monitored and the e-prescribe measure report was generated.
  9. CQM – Select CQM measures were tested in the live production environment.
  10. When sufficient live data was not available, synthetic data was generated and tested in a shadow (test instance) environment.

Care Setting(s)

Our Real World Testing was conducted within clinical settings in which our certified health IT is marketed. While health IT developers are not required to test their certified health IT in every setting in which it is marketed for use, we tested in various settings. 

The following care settings were included in testing.

  1. Hospital
  2. Ambulatory & Emergency Care
  3. Clinic
  4. Physician Practice

 

Relied Upon Software

The following third party software was relied upon to perform portions of the Real World Testing for several measures:

  1. popHealth

170.315(c)(1) – record and export

170.315(c)(2) – import and calculate

170.315(c)(3) – View Report

  1. Interoperability Engine

170.315(h)(1) – Direct Project

Metrics and Outcomes

We have detailed outcomes from our testing that successfully demonstrate that our certified health IT:

  1. is compliant with the certification criteria, including the required technical standards and vocabulary code sets;
  2. is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and/or,
  3. EHI is received by and used in the certified health IT.

This section describes how the specific data collected from their Real World Testing measures demonstrate our results.

This section describes how the specific data collected from their Real World Testing measures demonstrate our results.

Measurement

/Metric

Associated

Criterion(a)

Relied Upon Software

(if applicable)

Outcomes

Challenges Encountered

(if applicable)

170.315(b)(2)-Clinical Information Reconciliation and Incorporate

170.315(b)(1) Transitions of care,

170.315(h)(1) Direct Project

 

Generated and analyzed the G1/G2 reports for 3rd and 4th quarter in the production environment. There were no errors or issues reported for invalid formats.

Summary of care, Transition of care sent (b.1) and received (h.1) did not produce sufficient data for statistically significant results. The functionality was tested in the production environment and found to produce no errors.  In addition, synthetic data consisting of no less than ten simulated patient encounters using real-world styled scenarios was generated and tested in a shadow (test instance) environment on a quarterly basis with no errors noted.

 

170.315(b)(3)- e-Prescribe

 

Sure-scripts

e-Prescriptions in Ambulatory and Hospital sites were analyzed.

In the 3rd Quarter, e-Prescriptions were submitted and received electronic acknowledgements for all with a 100% success rate.

In the 4th quarter e-Prescriptions submissions and acknowledgements were analyzed with a 100% success rate.

 

170.315(b)(6)- Data Export

 

 

For the 3rd and 4th quarter, logs were analyzed showing bulk patient data export had no errors.

 

170.315- c(3) – View Report

170.315(c)(1)—record and export

170.315(c)(2)—import and calculate

 

For CQM measures calculated during the 3rdand 4th quarters, functionality was tested in the production environment with no errors noted.

 

170.315(e)(1)- View, Download and Transmit to third party

 

 

The G1/G2 report for the 3rd and 4th quarters was generated and analyzed. Data and functionality for view, download and transfer was analyzed and tested. The error rate was found to be 0%.

 

170.315(f)(2)-Transmission to Public – Syndromic Surveillance

 

 

During the 3rd and 4th quarter, Messages for Syndromic Surveillance were analyzed including participation by customers and reporting agencies. Messages were sent successfully.

 

170.315(f)(3)-Transmission to Public- Reportable Laboratory Test Values/Results

 

 

During the testing periods, the capability of the iCare system to transmit laboratory results to public agencies was tested in the production environment with no errors noted.  In addition, synthetic data consisting of no less than ten simulated patient encounters using real-world styled scenarios was generated and tested in a shadow (test instance) environment on a quarterly basis by generating lab test results and processing messages through the iCare integration engine with no errors noted.

 

170.315(g)(9)- Application Access- All Data Request

170.315(g)(7) Application access—patient selection

170.315(g)(10) Standardized API with requests for

single patients and populations

phi

Generated and analyzed the G1/G2 reports for the 3rd and 4th quarters. We performed testing of this functionality in the production environment with no errors noted.  In addition, synthetic data was generated and tested in a shadow (test instance) environment and no errors were noted however a connection issue with a certified partner caused some failure of g10 requests.  The issue was resolved.

 

 

 KEY MILESTONES

The following is a list of key milestones that were met during the Real World Testing process.  Details are included on how and when we implemented measures and collected data.

Key Milestone Care Setting Date/Timeframe
During the 3rd Quarter of 2024, the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients. Ambulatory and Hospital 1st July – 31st Oct 2024
We started report generation during the first week of October 2024 and evaluation was performed as applicable for the 3rd quarter. Ambulatory and Hospital First week of October 2024
During the 4th Quarter of 2024 the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients. Ambulatory and Hospital 1st Oct – 31st Dec 2024
We started final report generation during the first week of Jan 2025 and performed evaluation and analysis as applicable for the 4th quarter. Ambulatory and Hospital First week of January 2025
Upload of final report Ambulatory and Hospital First week of February 2025
Powered by iCare