iCare.com

REAL WORLD TESTING RESULTS 2023

GENERAL INFORMATION

Plan Report ID Number: [For ONC-Authorized Certification Body use only] 

Developer Name:  iCare.com LLC

Product Name(s):  iCare EHR

Version Number(s):  V 2.0

Certified Health IT Product List (CHPL) ID(s): 15.04.04.2617.iCar.02.00.1.200220 

Developer Real World Testing Page URL:  https://icare.com/realworldtesting

 

SUMMARY OF TESTING METHODS AND KEY FINDINGS

The iCare.com EHR is a pure cloud product, therefore our Real World Testing plan included considerations for our delivery methodology which differs significantly from on-premise (or simply hosted) software.  The iCare cloud environment permitted us to monitor and test most criteria of our certified technology without the requirement to involve our customers with significant portions of the testing methodologies.

The annual Real World Testing for 2023 was performed by iCare.com using metrics from our production environment which serves actual customers using our software to manage the healthcare of real patients.  Customers participated in testing those criteria which require their involvement.

During Real World Testing activities, we anticipated that there could be instances where we may find a need to modify our testing methodologies, approaches, or expected outcomes that were originally laid out in our plans to address unexpected changes during the testing period.  In such instances, we would adjust our Real World Testing approaches and our results report will include a description of these types of changes, the reasons for them, and how intended outcomes were more efficiently met as a result.  No modifications to the original testing plan were required.

Please note that the Real World Testing Plan referenced in the CMS CHPL may not reflect the currently implemented plan.

The following list is a summary of the Real World Testing Approach.

  1. The live production environment was used for Real World Testing.  Both hospital and clinic settings were included in the testing.
  2. Customers used the iCare system as usual and customer involvement was limited to those areas where it was necessary.
  3. Data was recorded quarterly. We also used the EHR Measure Report (g1 g2) to generate data.
  4. The received CCDA reports were validated, incorporated, and reconciled and the measure reports were reviewed.
  5. The Data Export functionality was tested periodically as required.
  6. Logs were analyzed to measure successful messages sent for Syndromic Surveillance.
  7. ELR – Checked for data sent from the production environment.
  8. e-Prescribe – the production system connection to Surescripts was monitored and the e-prescribe measure report was generated.
  9. CQM – CMS 72, CMS 105, CMS 108 and CMS 111 was tested in the live production environment.

Care Setting(s)

Our Real World Testing was conducted within clinical settings in which our certified health IT is marketed. While health IT developers are not required to test their certified health IT in every setting in which it is marketed for use, we tested in various settings. 

The following care settings were included in testing.

  1. Hospital
  2. Ambulatory & Emergency Care
  3. Clinic
  4. Physician Practice

 

Relied Upon Software

The following third party software was relied upon to perform portions of the Real World Testing for several measures:

  1. popHealth

170.315(c)(1) – record and export

170.315(c)(2) – import and calculate

170.315(c)(3) – View Report

  1. phiMail Server

170.315(h)(1) – Direct Project

Metrics and Outcomes

We have detailed outcomes from our testing that successfully demonstrate that our certified health IT:

  1. is compliant with the certification criteria, including the required technical standards and vocabulary code sets;
  2. is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and/or,
  3. EHI is received by and used in the certified health IT.

This section describes how the specific data collected from their Real World Testing measures demonstrate our results.

Measurement

/Metric

Associated

Criterion(a)

Relied Upon Software

(if applicable)

Outcomes

Challenges Encountered

(if applicable)

170.315(b)(2)-Clinical Information Reconciliation and Incorporate

170.315(b)(1) Transitions of care,

170.315(b)(7) Security tags – summary of care – send,

170.315(b)(8) Security tags – summary of care – receive

170.315(h)(1) Direct Project

 

Generated and analyzed the G1/G2 reports for 3rd and 4th quarter in the production environment. There were no errors or issues reported for invalid formats.

Summary of care, Transition of care sent (b.1, b.7) and received (b.8, h.1) did not produce sufficient data for statistically significant results. The functionality was tested in the production environment and found to produce no errors for clinical reconciliation (b.2).

 

170.315(b)(3)- e-Prescribe

 

Sure-scripts

e-Prescriptions in Ambulatory and Hospital sites were analyzed.

In the 3rd Quarter, e-Prescriptions were submitted and received electronic acknowledgements for all with a 100% success rate.

In the 4th quarter e-Prescriptions submissions and acknowledgements were analyzed with a 100% success rate.

 

170.315(b)(6)- Data Export

 

 

For the 3rd and 4th quarter, logs were analyzed showing bulk patient data export had a 100% success rate with no errors shown.

 

170.315- c(3) – View Report

170.315(c)(1)—record and export

170.315(c)(2)—import and calculate

 

For CQM measures 105, 108, 72 and 111 no data was generated by customers during the 3rdand 4th quarters, but this functionality was tested in the production environment with no errors noted.

 

170.315(e)(1)- View, Download and Transmit to third party

 

 

The G1/G2 report for the 3rd and 4th quarters was generated and analyzed. Data and functionality for view, download and transfer was analyzed and tested. The error rate was found to be 0%.

 

170.315(f)(2)-Transmission to Public – Syndromic Surveillance

 

 

During the 3rd and 4th quarter, Messages for Syndromic Surveillance were analyzed including participation by customers and reporting agencies. Messages were sent successfully.

 

170.315(f)(3)-Transmission to Public- Reportable Laboratory Test Values/Results

 

 

During the testing periods, the capability of the iCare system to transmit laboratory results to public agencies was tested in the production environment with no errors noted.

 

170.315(g)(9)- Application Access- All Data Request

170.315(g)(7) Application access—patient selection

170.315(g)(8) Application access—data category request

 

 

Generated and analyzed the G1/G2 reports for the 3rd and 4th quarters. We performed testing of this functionality in the production environment with no errors noted.

 

 

 

KEY MILESTONES

The following is a list of key milestones that were met during the Real World Testing process.  Details are included on how and when we implemented measures and collected data.

Key Milestone

Care Setting

Date/Timeframe

During the 3rd Quarter of 2023, the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients.

Ambulatory and Hospital

1st July – 31st Oct 2023

We started report generation during the first week of October 2023 and evaluation was performed as applicable for the 3rd quarter.

Ambulatory and Hospital

First week of October 2023

During the 4th Quarter of 2023 the testing for all required measures started as outlined in the plan. The production environment users kept their routine data entry practices in place for actual patients.

Ambulatory and Hospital

1st Oct – 31st Dec 2023

We started final report generation during the first week of Jan 2024 and performed evaluation and analysis as applicable for the 4th quarter.

Ambulatory and Hospital

First week of January 2024

Upload of final report

Ambulatory and Hospital

First week of February 2024

Powered by iCare