Pulse oximetry screening for critical congenital heart defects in Ontario, Canada: a cost-effectiveness analysis

Abstract

Résumé

Introduction

Congenital heart defects are structural fetal malformations that occur in approximately 8–9 per 1000 live births (Hoffman and Kaplan 2002; van der Linde et al. 2011). Of these, up to 30% are “critical”, associated with significant risk of mortality and morbidity (Heron and Smith 2007; Rosano et al. 2000), especially if missed (Brown et al. 2006). Routine antenatal ultrasounds can detect critical congenital heart defects (CCHDs), but many remain undiagnosed (Sharland 2012). Prior to closure of the patent ductus arteriosus, many CCHD lesions may remain without overt clinical symptoms. It has been estimated that 20–30% of CCHDs are missed by routine physical assessment in early newborn period (Meberg et al. 2009).

Screening for CCHD using pulse oximetry—henceforth referred to as pulse oximetry screening (POS)—is an attractive tool to screen for clinically undetectable CCHDs (Narayen et al. 2016). A meta-analysis of studies on POS found an overall sensitivity of 76.5% for detection of CCHD and overall specificity of 99.9% with a false positive rate of 0.14% (Thangaratinam et al. 2012). Similar sensitivity and specificity rates were reported in a more recent Cochrane review (Plana et al. 2018). Based on the evidence, the American Academy of Pediatrics as well as the Canadian Paediatric Society have endorsed routine screening for CCHD using POS (Mahle et al. 2012; Wong et al. 2017), and POS screening is being implemented in Ontario in tandem based on institutional capabilities (Newborn Screening Ontario n.d.).

POS requires real-time interpretation, and in cases of positive result, immediate action to provide timely diagnosis and appropriate management. Due to resource implications, particularly lack of availability of neonatal echocardiography at all centres as well as the burden of potentially large numbers of false positive results, there have been a number of cost-effectiveness analyses of POS for CCHD (de-Wahl Granelli et al. 2009; Knowles et al. 2005; Griebsch et al. 2007; Ewer et al. 2012; Roberts et al. 2012; Peterson et al. 2013). Details of these analyses are shown in Supplemental File 1, Table S1.1. All these models suggest that screening for CCHD is likely to be cost-effective. However, with the exception of the Swedish analysis by de-Wahl Granelli et al. (de-Wahl Granelli et al. 2009) and the US analysis by Peterson et al. (Peterson et al. 2013), all were from the United Kingdom, where availability and distribution of clinical resources are much different than in Ontario, Canada. Another limitation of previous studies is that the models represent time until diagnosis, except the study by Peterson et al. where the time horizon was first year following birth (Peterson et al. 2013). In addition, quality of life parameters associated with CCHDs have not been incorporated previously in any analyses. In light of these limitations, as well as in consideration of some unique logistical challenges towards screening for CCHDs, our objective was to determine whether POS implementation would be a cost-effective endeavour in Ontario.

Methods

A model-based cost-effectiveness analysis comparing POS to no POS was conducted from the Ontario healthcare payer perspective (Ministry of Health). A Markov model was developed to predict lifetime events, including quality-adjusted life months (QALMs), lifetime costs, and incremental cost-effectiveness ratios (ICER) [ΔCost/ΔQALMs], for a well-appearing newborn infant at 24 h of age. Ontario-specific data were used wherever possible. We followed Canadian guidelines for economic evaluations in health (Health CAfDaTi 2006). No individual patient-level data were utilized; this study was approved by the Research Oversight and Compliance Office - Human Research Ethics Program, University of Toronto, protocol reference number 32846.

Model structure

TreeAge Pro v2015 (TreeAge Software Inc., Williamstown, MA, USA) was used to develop a Markov decision model with 1-month time steps (each “cycle” representing a one-month step), allowing lifetime follow-up. Monthly intervals were chosen as during the first years of life, there are significant variations in mortality rates, morbidities and costs that would not be ascertainable using yearly time cycles (Ohuchi et al. 2011; Simeone et al. 2014). A lifetime horizon was chosen as, increasingly, studies are reporting the longer-term outcomes of CCHDs well into adulthood (Lee et al. 2014). The following exhaustive, mutually exclusive health states were created: (1) CCHD (pre-operative); (2) no CCHD (≤ 1 month); (3) post-operative CCHD (with morbidity); (4) post-operative CCHD (no morbidity); (5) no CCHD (> 1 month); and (6) death. At any given time, a simulated individual could only be in one of the six health states.

At the beginning of the first cycle, a simulated individual could only be in one of the following: (1) CCHD (pre-operative) or (2) no CCHD (≤ 1 month). An individual with (undiagnosed) CCHD at 24 h could have the screening performed and yield a (true) positive or a (false) negative result, the latter being a case of missed CCHD. Similarly, an individual with no CCHD could have a (false) positive or a (true) negative screening result. All positive results (whether true or false) were designed to require follow-up.

In the initial cycle, the model depicted possibilities of patient location and requirement of transfer(s) in detail (Fig. 1), and in cases of confirmed CCHD, possible outcomes from surgery (Fig. 2a). The model also depicted possible outcomes in event of a missed CCHD presenting with symptoms at home (Fig. ​(Fig.2b).2b). End of the first cycle resulted in an individual transitioning to 1 of the following health states: (1) Post-operative CCHD (no morbidity); (2) post-operative CCHD (with morbidity); (3) no CCHD (> 1 month); or (4) death. For purposes of this model, morbidity referred to any neurodevelopmental impairment (detailed in Supplemental File 2). In subsequent model cycles, a simulated individual could remain in the same health state, or transition to another. The cycles continued until all individuals transitioned to “Death”. Figure ​Figure33 depicts possible transitions among health states in subsequent Markov cycles.

Open in a separate window

Fig. 1

Illustration of decision tree during first Markov cycle and various possible outcomes (including need for transfers) in a simulated individual who has CCHD

Open in a separate window

Fig. 2

Panel a illustrates possible outcomes after missed CCHD incorporated into first cycle of Markov decision model. Panel b shows the possible outcomes of CCHD surgery incorporated into first cycle of Markov model. Note than in both Figs. 1 and 2, the circles represent a “chance” node with a certain probability associated with either arm emanating from that node being chosen (determined by the values for probabilities inputted into the model) while the triangles represent a “terminal” node, culminating in the transition to another health state

Open in a separate window

Fig. 3

Illustration of the possible transitions (denoted by arrows) among the 4 health states in subsequent Markov cycles

Results

In the base case of a well-appearing newborn, performing POS at 24 h of life was the superior strategy. There are approximately 150,000 births province-wide annually (Canadian Institute for Health Information; https://www.cihi.ca/en). Based on incidence of missed CCHD of 0.0004 (Cohen et al. 2015) and probability of POS detecting 0.843 of all CCHDs (Supplemental File 2), it was estimated that an additional 51 cases of CCHD will be detected annually in a timely fashion with POS implementation. The lifetime cost to the healthcare payer per individual was estimated to be $284,002.58 with POS implementation and $283,975.31 without POS implementation, yielding an incremental cost of performing POS of $27.27 per individual ($284,002.58–$283,975.31). Similarly, the expected QALMs per individual with and without POS implementation were expected to be 554.53 and 554.50, respectively, with a resulting gain of 0.03 QALMs. Based on 150,000 births per year, this would lead to an overall gain of 3682 QALMs or 307 quality-adjusted life years (QALYs) per birth cohort. The incremental cost and QALMs yielded an ICER of $1110.79, well below the cost-effectiveness threshold.

The model was not sensitive to any variable in one-way sensitivity analyses, i.e., the implementation of POS was superior for all variables across their plausible ranges. “Threshold analyses” were conducted for variables indicated in Table ​Table1,1, and thresholds (where identified) above or below which POS implementation is no longer expected to be cost-effective are indicated. It was predicted that POS implementation would no longer be cost-effective if (a) POS detects < 23.2% of CCHD lesions (well below the plausible lower limit of 80.2%); (b) POS falsely positive rate exceeds 11.8% (well above the estimated upper limit of 0.994%); (c) if probability of death at home in case of missed CCHD is < 3.5%, below the estimated lower limit of 7.4%; or (d) incidence of CCHD in base case is < 0.00009 (just below the estimated lower limit of 0.0001, but well below point estimate value of 0.0004).

In probabilistic sensitivity analysis of 10,000 model simulations, POS implementation was considered cost-effective 92.3% of the time at the cost-effectiveness threshold of $4166.67 (Fig. 4). Figure ​Figure55 shows that it is more likely to be cost-effective than no POS implementation (i.e., > 50% chance) at a cost-effectiveness threshold value as low as $1175.

Open in a separate window

Fig. 4

Cost-effectiveness scatter plot. Note: Each dot represents the incremental cost and incremental effectiveness (QALM) from each simulation, and all simulation values “below” the line of cost-effectiveness threshold are deemed to be cost-effective, whereas the ones above the threshold line represent simulations where ICER for POS implementation was above the pre-determined threshold value

Open in a separate window

Fig. 5

Cost-effectiveness acceptability at various cost-effectiveness (or “willingness to pay”) thresholds

Discussion

Province-wide implementation of POS for CCHD in Ontario appears to be a cost-effective endeavour with an estimated 51 additional cases of CCHD diagnosed in a timely fashion annually and a 92.3% chance of being cost-effective at an ICER threshold of $4166.67 per QALM. The findings of this model are consistent with what might be expected in context of biological plausibility as patients with delayed diagnosis of CCHD are more likely to experience hemodynamic compromise, resulting in prolonged hypoxemia to vital organs. They are more likely to not survive, as well as have a higher chance of morbidity (Fixler et al. 2014).

The likelihood of POS being cost-effective, along with its safe and non-invasive method, makes it a suitable screening tool for early diagnosis of CCHD. These criteria constitute the tenets of the Wilson and Jungner screening criteria (Wilson and Jungner 1968). Our model also shows that despite many cases being detected antenatally and postnatally prior to 24 h of age, POS still remains cost-effective despite the relative rarity of missed CCHD. In order to maximize effectiveness of POS, standardized guidelines for POS including recommended measurement techniques and standard procedures for follow-up are imperative (Wong et al. 2017). These procedures would minimize the cost to the healthcare system by minimizing the effect of ambiguity in steps that need to be taken and would maximize the effect of POS.

A number of studies have previously assessed the cost-effectiveness of POS implementation. Most such studies have been done in the UK (Knowles et al. 2005; Griebsch et al. 2007; Ewer et al. 2012; Roberts et al. 2012), but also in the United States and Sweden (de-Wahl Granelli et al. 2009; Peterson et al. 2013). Previous studies have evaluated a very limited time horizon, with all being until the point of diagnosis of CCHD, except the study by Peterson et al., which employed a 1-year time horizon (Peterson et al. 2013). The ICER values for these studies varied from as low as £3430 to as high as £24,900 per additional case of CCHD diagnosed. The reason for wide variability in the costs may relate to the nature of models created and data used, but all suggested that POS was likely to be cost-effective. The study by Peterson et al. determined the ICER to be $40,385 per life year gained (which converts to $3365.42 per month gained in US dollars). It should be noted that, unlike our model, none of the aforementioned studies employed utilities to account for morbidities that may be associated with CCHD and its outcomes (i.e., the various possible health states)—and as such do not present the costs for quality-adjusted life units gained and, therefore, cannot truly evaluate cost-effectiveness. In addition, our study provides a lifetime horizon which provides a clear sense of total lifetime costs and quality-adjusted life units to be gained by implementation of POS. In addition, model structures in aforementioned studies were rather simple, which did not take into account various possible pathways to the ultimate diagnosis of CCHD following a positive screen. Despite these differences from our study, these studies affirmed cost-effectiveness of POS, and our study was consistent with this in the Canadian context.

Limitations of the model framework itself relate largely to assumptions made, and include (a) assumption that individual with CCHD with positive screen would remain asymptomatic until surgery, (b) individual with CCHD requires only 1 surgery, (c) morbidity related to CCHD limited to neurodevelopmental impairment and that no new morbidity would arise after adolescence, and (d) inability to identify values for some variables requiring assumptions regarding their point estimates and ranges. We also acknowledge that while a targeted literature search was conducted for many of the variables, these were not systematic searches (due to feasibility) and therefore may not be representative of the entire body of literature for each variable. Another important limitation is decreasing confidence in point estimates of many variables the farther out the time horizon gets, due to lack of available data. This also adds to increased uncertainty with respect to the overall results of the model. Additionally, evaluating a lifetime span, particularly when accounting for an annual discount rate, may contribute to diluting the incremental gain in QALMs. The notion of quality adjustment for pediatric population may be considered a limitation, as it is difficult to ascertain the true impact of morbidities of personal assessment of life quality at such a young age. Finally, no spillover (caregiver) effects were considered as it relates to utility and cost variables (with a few exceptions indicated previously), although this makes our analysis conservative (i.e., if including spillover effects, it would have been expected that the ICER would be lower still). The most important strength of the model is the detailed framework that represents a realistic pathway from screening to diagnosis to outcomes in the first Markov cycle. Another important strength is the employment of time-varying variables that allow the values to change over time for transitional probabilities (Supplemental File 2). This is close to the realistic progression through life following CCHD. Finally, the model is representative of Ontario’s geography with a unique distribution of population (and resources)—whereby a few small areas are extremely densely populated, whereas large regions are sparsely populated.

Conclusion

POS implementation in Ontario is likely to be a cost-effective endeavour, with an estimated ICER of CAD$1110.79 per QALM gained. Despite creation of a realistic model framework, limitations in available data mean that the robustness of this analysis could be enhanced by incorporating or obtaining more local data, especially in light of POS implementation having been initiated in Ontario. In addition, it may be important to be mindful of thresholds for variables presented at which POS is no longer estimated to be cost-effective, and monitoring of these values over time may be warranted to ensure POS continues to remain a cost-effective endeavour.

Electronic supplementary material

Acknowledgements

Compliance with ethical standards

Footnotes

References