Data quality assessment in emergency medical services: an objective approach

Abstract

Introduction

Data quality is a complex topic thatincludes various dimensions such as accuracy, reliability, precision, completeness, timeliness, integrity and confidentiality [1, 2]. However, it is still a real challenge in many organizations including healthcare facilities [3]. In the healthcare industry, measuring data quality indicators are necessary to provide policymakers with reliable evidence for better decision-making and planning to deliver high quality healthcare services [4]. The quality of healthcare data is not only important for patient care, but also increases the effectiveness and efficiency of healthcare professionals and their services. High quality data should also be considered an essential prerequisite of information systems that are used to support healthcare services [5, 6].

Emergency department (ED) is one of the most important departments in a hospital, and a proportion of the patients, who refer to this department, are in critical conditions. In public health services, the performance of EDs is considered as a key benchmark for healthcare delivery, and meeting the key performance indicators such as providing high quality data in this department is regarded as one of the important issues for measuring the performance of the whole health system [7, 8]. In some studies, the use of electronic health records (EHR) has been suggested to measure performance of EDs, because the quality of electronic data is regularly evaluated and can provide managers with reliable reports [9–11].

On the other hand, the unique features of emergency care services, such as tasks complexity, high speed healthcare delivery, frequent transfer of care from one provider to another, multiple interruptions, high turn-over of patients and sometimes dealing with unknown or complex cases in the emergency environment, make this type of care prone to errors [12]. As a result, problems with data quality may occur more frequently in the ED than in other departments of the hospital and the process of data collection and quality assurance encounters a number of bottlenecks, which in turn, leads to unintended consequences [13]. However, the dynamic nature of emergency care services requires complete and accurate documentation of the care processes [14].

According to the literature, data quality in emergency medical services is not at a high level, and the needed data are not available at the point of need to the healthcare professionals and managers [15, 16] Therefore, it is necessary to assess the quality of data routinely to be able to improve it. The present study aimed to assess data quality in emergency medical services using an objective approach.

Methods

This study was a retrospective quantitative study conducted in 2019 in which three dimensions of data quality; namely, completeness, accuracy and timeliness of data were assessed by reviewing different forms in the emergency medical records. Before conducting the research, ethics approval was obtained from the university ethics committee.

Results

In the following sections, the results of data quality assessment are reported for data completeness, data accuracy, and data timeliness, separately.

Data accuracy

To examine the level of data accuracy, data consistency among different data sources was measured. Data sources included emergency medical records (as a gold standard), pre-hospital emergency care form, and hospital information systems. Initially, the common data elements among these data sources were identified. To measure the level of data accuracy, the number of discrepancies between two data sources was divided by the number of common data elements. Initially, data consistency between the emergency medical records and hospital information systems was examined. However, the only common data elements between these two resources were demographic data which were consistent in 100% of cases. In fact, when a patient was admitted to the emergency department, the demographic data were entered into the hospital information system. Then, these data were printed and attached to the patient emergency medical records. Therefore, it was decided to examine the consistency of 6 common data elements in the pre-hospital emergency care form and emergency medical records (Table ​(Table33).

As Table ​Table33 shows, the highest consistency was related to the chief complaint (n = 353, 92.4%) and the lowest consistency belonged to reporting drug allergies (n = 74, 29.2%). In addition, there was no statistically significant difference between the data consistency of the chief complaint, final diagnosis, drug allergies and medication history. However, there was a statistically significant difference between reporting the external cause of accidents (P-value = 0.000) and medical history (P-value = 0.029) in these two data sources. It means that these two data elements were more documented by the pre-hospital emergency care staff compared to the ED staff.

Data timeliness

To measure data timeliness, three sources of data; namely, emergency medical records, hospital information system (HIS), and picture archiving and communication system (PACS) were used to extract the time of some para-clinical procedures (laboratory, radiology and CT scan services) performed in the EDs (Table ​(Table44).

Usually, the sequence of ordering and accessibility of the results should be as follows: paper-based physician ordering, order entry into the hospital information system (HIS), and the accessibility of the results in the HIS/PACS. After extracting the time documented for each procedure, the time interval between them was measured.

According to the results, there were positive and negative time intervals. The positive time interval indicated that the processes were performed sequentially; however, the negative time intervals indicated that the next process was performed before the previous one and they were not performed sequentially. As the results showed, in some cases the sequence of ordering and accessibility of the results was not maintained. As Table ​Table44 shows, the highest mean value (144 ± 78.6) was related to the positive time interval between the paper-based physician laboratory ordering and the accessibility of the laboratory results in the HIS and the lowest mean value (22.5 ± 35.4) belonged to the time interval between the paper-based physician radiography ordering and order entry into the HIS. Moreover, most of the negative time intervals (n = 31, 18%) was related to the interval between the radiography order entry into the HIS and the accessibility of the results in PACS, which showed radiology orders were not documented in the HIS on a timely basis.

Discussion

In this study, data quality was assessed in three emergency departments using an objective approach. To measure data quality, different data sources including a triage form, emergency medical records, pre-hospital emergency care form, hospital information systems, and picture archiving and communication systems were used, and data quality dimensions, which included data completeness, accuracy, and timeliness were assessed. As the results showed, the demographic data were more complete than the clinical data in three data sources; namely the triage form, emergency medical records, and pre-hospital emergency care form. Among the clinical data, the level of data completeness for some important data, such as life threatening conditions, drug allergies and vital signs (except blood pressure in the triage form) was not satisfactory, and among the above mentioned data sources, the level of data completeness in pre-hospital emergency care forms (70%) was higher than the other data sources.

It is noteworthy that in the present study and other similar ones, patient medical records were considered as a gold standard [19]. As a result, it is expected to see a higher level of data completeness in this data source. However, according to the results, the level of data completeness in the patient emergency medical records was 57.3% by average compared to the pre-hospital emergency care forms (70%). Although the completeness of the pre-hospital emergency care form was higher than the other data sources, according to Nisingizwe et al., if the completeness of data is less than 80%, it is considered poor [20]. The higher level of data completeness in the pre-hospital emergency care forms might be due to the importance of documenting each data element for a patient who is in a critical condition, strong dependency between documenting data in the pre-hospital care services and the continuity of care in the ED, and the importance of the legal issues.

The results are also consistent with the findings of other similar studies which showed that emergency medical records were usually incomplete and there were a number of missing data in the related electronic databases [14, 21]. This issue, in turn, can adversely affect quality of patient care [22]. The reasons for low quality documentation in the ED can be related to the high workload and low usage of digital infrastructure [23].

To measure data accuracy, the level of data consistency was examined in three different data sources. In fact, examining the level of data consistency is used to increase interoperability between different data sources [21, 24]. In addition, the best way to determine data accuracy is to determine the accuracy of data against another data source [20, 25, 26]. The results showed that most of the data elements were accurate (consistent), and there was no statistically significant difference between the data consistency in the emergency medical records and the pre-hospital emergency care form, except for the external cause of accidents and patient medical history which were more completed in the pre-hospital emergency care form. The importance of data consistency between different data sources, particularly in the emergency medical services has also been highlighted in other studies and it has been introduced as a predictor to determine the level of data completeness and accuracy [21, 27]. Understanding types and frequency of data errors in emergency care records helps healthcare organizations to develop practical strategies to improve data quality. Therefore, more attention should be paid to improve data consistency between emergency care data sources.

To measure data timeliness, the time interval between the paper-based physician ordering and order entry into the HIS, order entry into the HIS and the accessibility of the results in the HIS/PACS, and paper-based physician ordering and the accessibility of the results in the HIS/PACS were calculated and compared. The results showed that the time spent on the laboratory services was more than radiography and CT scan services, which can be related to the nature of the laboratory tests. However, there were some positive and negative time intervals. The negative time intervals showed that the sequence of procedures was not maintained. One of the reasons for this problem might be related to the complexity of the workflows in the ED. In addition, this might be due the ED workload, the severity of the patient condition and the urgency of providing some services which can interrupt routine workflows, priority of using verbal communications among the ED staff, lack of system integration, or paying less attention to the data documentation either in the paper-based or computer-based records [20]. Moreover, difficulties with continuous use of both paper-based and computer-based records can influence poor data quality [28]. As timely documentation of procedures is highly important in emergency care services, especially treatment, interventions, forensic issues, and quality assurance activities [29], process optimization should be taken into account to overcome current challenges.

Overall, it seems that data quality in emergency medical services was not satisfactory and there were some weaknesses in the documentation processes. Therefore, more practical strategies need to be developed to improve data quality. The use of health information technology in a wider scope can be one of these strategies that has been mentioned in several studies [29, 30]. Health information technology leads to improve data quality dimensions, such as completeness, accuracy, timeliness and accessibility by improving processes and facilitating clinical workflows. Moreover, providing emergency care staff with adequate training on documentation and conducting regular audits of data quality are other strategies which can help to improve data quality [31–33].

Conclusions

In this study, data quality was assessed in three emergency departments using an objective approach. The results showed that data quality in emergency medical services was not satisfactory. The demographic data had the highest level of completeness compared to the clinical data, and the pre-hospital emergency care form was more complete than other data sources. In addition, most of the examined data elements were accurate/ consistent. In terms of timeliness, the negative time intervals between some procedures suggested that there is a need to pay more attention to the sequence of para-clinical procedures in the EDs. These findings can be used to investigate bottlenecks and weaknesses in the documentation processes. Given that high quality data are necessary for providing high quality care, and a number of influencing factors may hinder proper documentation in emergency care services, it seems that providing a national minimum data set for emergency care services, training the emergency care staff in generating high quality data, and routine data quality audit can be useful for improving data quality. Further investigations are needed to use a combination of observational and qualitative research methodologies to identify bottlenecks in the process of emergency care documentation.

Acknowledgements

Abbreviations

Authors’ contributions

M.M. conducted the research and drafted the manuscript. H.A. supervised the research and commented on the manuscript. D.K.Z. participated in the critical review of the manuscript. T.T.A.B. helped with revising the manuscript. All authors reviewed the manuscript.

Funding

This study was funded and supported by Iran University of Medical Sciences, Tehran, Iran (IUMS/SHMIS_93/10).

Availability of data and materials

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Declarations

Footnotes

References