Use of a pediatric risk score for cardiac catheterization in a Spanish population with congenital heart disease

INTRODUCTION

Before 2008, many attempts were made to create a standardized risk score to assess the likelihood of serious adverse events (SAEs) during pediatric cardiac catheterization. The field of pediatric cardiac catheterization has jointly designed and implemented several large, multicenter clinical registries including the Congenital Cardiac Interventional Study Consortium (CCISC) and the Congenital Cardiac Catheterization Project on Outcomes (C3PO).

The former registry started a project back in 2006 to define adverse events and risk adjustment.

This effort has yielded important information concerning the risk associated with the type of procedure being performed.1,2 Back in 2015, the CCISC developed and validated an empirical preoperative catheterization risk score for pediatrics (CRISP). A 21-point scale that estimates the risk of procedural-related SAEs defined as any event leading to mortality, permanent morbidity, need for further procedures or extended lengths of stay. The CRISP derived from nearly 15 000 procedures added to the CCISC database from 27 centers of North and South America and a few European centers (eg, Evelina London Children’s Hospital, London, United Kingdom).3

As this system has not been validated in most European countries—Spain included—we tried to use and validate the CRISP by assessing its predictive performance in 1 of the Spanish centers that runs pediatric cardiac catheterizations.

METHODS

The study was conducted after approval from our center Institutional Review Board and consent from the Research Ethics Committee to access patients’ relevant health records including data storage and confidentiality. The participants’ informed consent was not deemed necessary as it was a retrospective study handling anonymized data. We collected data retrospectively assessing all electronic and written health records available. We obtained information on all diagnostic and interventional catheterizations performed from January 2016 through May 2017. We excluded procedures with patients > 18 years-old, electrophysiological studies, transesophageal echocardiographies, vascular access, and pericardiocentesis.

Use of CRISP

We have translated the CRISP to Spanish without altering its original variables (note: although the tables may look different, the variables remain unchanged).

The patients’ characteristics and type of procedure used were recorded as follows (= CRISP variables): age, weight, pre-catheterization diagnosis, systemic illness/organ failure, hemodynamic variables, inotropic support, procedural category, and case type (diagnostic, interventional, hybrid).3

Both the physiological parameters and anatomical diagnoses were defined based on the Society of Thoracic Surgeons/European Association for Cardio-thoracic Surgery shortlist. We divided them into 3 levels of perceived increased risk, procedural category, and case type. Multiple procedures were assigned to the highest-risk category.1,3

SAE was defined as any adverse event requiring further procedures, extended lengths of stay, causing permanent morbidity or eventual death.3 The lead operator/investigator identified all SAEs and reviewed treatment modalities to apply severity definitions.

For practical purposes, Nykanen et al. suggested clustering the risk category into 5 different SAE risk groups: 1.0%, 2.6%, 6.2%, 14.4%, and 36.8%, respectively, rounded to the nearest whole percentage.3

The points assigned to each variable and the overall CRISP score were estimated and recorded in the database.

Statistical analysis

All data are expressed as percentages, mean, and median. SAEs were grouped according to severity as a percentage of the overall number of SAEs. These were estimated as the percentage of the overall number of procedures or population at risk for an event. All statistical analyses were performed using IBM SPSS Statistics statistical software platform, version 21.

To assess the performance of CRISP in our population, we created a receiver operating characteristic curve. To estimate its ability to predict SAEs, we applied Pearson’s chi-square test (P < .08 were considered significant).4

RESULTS

From January 2016 through May 2017, a total of 669 patients we identified in our exploratory database. After applying exclusion criteria, 516 patients remained, 26.6% of whom were < 1 year-old [range, 1 day to 18 years-old], and 56.2% were males. The mean and median weights were 23.2 kg and 19.0 kg with 17% of the patients < 5 kg [range, 1.1 kg to 82.0 kg]. Procedures were diagnostic, interventional, hybrid or cardiac biopsies in 20.3%, 62.8%, 0.8%, and 16.1%, respectively (table 1).

Table 1. Patient and procedural characteristics from January 1, 2016 through May 31, 2017

All catheterizations were performed under general anesthesia. During the period at stake, 3 operators performed the procedures, which gives robustness and homogeneity to our results.

A total of 23% of the population included patients with single-ventricle physiology while 14.9% had complex CHD with biventricular anatomy (eg, outflow tract obstruction and intra-cardiac shunts). A total of 32% had isolated injuries like atrial septal defect, patent ductus arteriosus or valvular abnormalities. In the overall cohort, 25.2% and 1.5% had non-structural heart diseases (eg, cardiomyopathy or heart transplant) and non-cardiac conditions (eg, retrieval of a foreign body), respectively.

Most procedures were categorized as category risk 3 and 5 (67.6%). These are the highest risk categories, which shows the complexity of the procedures as risk categories increase. Our study area under the receiver operator curve (AUC) is 0.71, which is similar to the CRISP AUC of 0.74 (95% confidence interval (95%CI) of 0.66 to 0.91). No statistically significant differences were seen between the predicted risk and events obtained and Pearson chi-square test of 8.26 (P < .08).

Serious adverse events

A total of 41 SAEs (7.9%) were documented (description and frequencies shown are on table 2). First and foremost, of all the adverse events recorded, only 1 event per procedure was identified. No deaths or need for emergency surgery associated with cardiac catheterization were ever reported. Device embolization occurred in 6 patients; all were retrieved using percutaneous techniques. No device migration was reported after discharge. Arrhythmias requiring procedures were treated medically. Complete heart block was resolved with temporary pacemaker implantation.

Table 2. Rate of serious adverse events by proposed risk category

Two cases of unexpected cardiac arrest requiring emergency extracorporeal membrane oxygenation support were reported. A non-syndromic 18-month child (CRISP category 4; risk of SAE, 14.4%) with a post-natal diagnosis of Shones syndrome (critical aortic stenosis and ventricular dysfunction). He underwent multiple procedures including aortic angioplasty, pulmonary bi-banding, and PDA stenting in the neonatal period followed by Ross–Konno procedure when he turned 9 months old. During catheterization, he showed electromechanical dissociation without response to cardiopulmonary resuscitation requiring extracorporeal membrane oxygenation. After stabilization, stenting was performed in both pulmonary arteries, left anterior descending, and left circumflex coronary arteries.

The other patient was a 10-year-old with suspected myocarditis in the context of an influenza A infection and cardiogenic shock (CRISP category 5; risk of SAE, 36.8%) who required inotropic support during cardiac catheterization. The procedure included an atrioseptoplasty with cardiac perforation and cardiac tamponade. He underwent 2-min cardiopulmonary resuscitation that needed extracorporeal membrane oxygenation support.

Not surprisingly, the rate of SAEs increased with higher risk scores (table 3). Two patients received 19 points, but no one a maximum score (21 points).

Table 3. Adverse events

DISCUSSION

The original CRISP has already proven to be a robust risk predictor of SAEs.4 Comparison with CCISC data was intentional trying to establish a benchmark. Therefore, it’s not surprising to see that many features and outcomes are analogous.

A potential drawback of CRISP is the introduction of physiological variables like pulmonary vascular resistance, right ventricular systolic pressure, and anemia requiring intraoperative transfusion. Although these variables can sometimes be estimated non-invasively beforehand, they may not be reliable as their true quantification is unknown before cardiac catheterization. A recent refinement of CRISP (rCRISP) published back in 2018 excluded those 3 physiological markers. However, it performed well regarding risk prediction with an AUC of 0.70 and observed/expected SAE ratios from 0.71 to 1.18. However, the revised model demonstrated a worse overall fit compared to the original CRISP (based on lower Akaike’s [AIC], Schwarz’s [BIC] information criteria, and -2log likelihood ratio [N2LL]).5

In this study, we have validated the CRISP scoring system, revealing decent predictive accuracy for SAEs. Our results are equivalent to the original scoring system confirmed by the agreement measure AUC 0.74 and 0.71 (CRISP and our study, respectively) [95%CI, 0.66-0.91].

We identified all SAEs recorded, those liable to immediate correction, and those that resulted in an unexpectedly extended length of stay, and need for medical or interventional treatment (whether surgical or transcatheter).

Although our study was implemented retrospectively, we would like to highlight that the corresponding risk categories in cases requiring extracorporeal membrane oxygenation were 4 and 5, the 2 with the highest risk of adverse events. If CRISP were available, these outcomes would be rather predictable. Therefore, greater attention would be paid to the need for highly differentiated care. Obviously, in the cases indicated, the outcome was very favorable. Still, the utility of this tool is evident in the preparation and anticipation of all the necessary care in more complex and high-risk procedures.

The mortality definition associated with CHD catheterization still needs to be elucidated since assessing mortality is challenging in this field.3 Although, in this study, mortality rate is 0% and minimal in other studies, it should be interpreted with caution.

Furthermore, physicians are understandably concerned with unfair comparisons of sensitive topics, eg, critical evaluation of unfavorable outcomes and events used for compensation and accreditation. Consequently, comparing centers or operators to identify the best practices can be facilitated, thus acknowledging that confidence intervals will be broader in low-volume centers.

Interestingly, if we compare the rate of adverse events by risk category/group, we’ll be seeing more complications from groups 1 to 4. Surprisingly or not, that is not true for group 5. These findings could be due to the smaller volume of patients in this group and the heterogeneity of complexity regarding the procedures performed in such group.

Still, CRISP has an appropriate goodness of fit (Person’s chi-square test of 8.26 (P < .08).

Knowing that extrapolations are informal, and unquantified arguments are limited by assumptions,6 we still believe that CRISP can be extrapolated to all pediatric populations undergoing cardiac catheterization in Spain as our results are rather consistent with the original study.

As a matter of fact, these data show that CRISP positively impacts patient care as it can be used to plan procedures in advance. It helps prepare the required equipment, technical staff, need for special care after the catheterization like intensive care unit admission. It can also be used reliably for parent counseling before each procedure.5

Study limitations

In addition to the limitations inherent to retrospective studies, a substantial data limitation is the lack of systematic assessment of procedural success as in former studies. To this date, a standardized assessment of procedural success is lacking, but it will likely be the focus of future studies.3

For a more inclusive analysis, we disclosed all SAEs recorded, although adverse events are readily identified outcomes and perceived to have a meaningful impact on patient outcomes. However, without systematic auditing, there’s still a potential for loss of adverse events, especially after patient discharge.

It can be argued that sample size is crucial for a more accurate outcome and less estimation error. However, more is only, sometimes, better in sample sizing since we can estimate size effect based on previously reported or preclinical studies. For future reference, in retrospective and observational studies, if the original study already proved to be a strong predictor, a smaller sample would be adequate to prove the same effect.7

We aim to conduct a prospective registry to refine the analysis and make this model more robust in our center. In conclusion, it would be interesting to join a nationwide Spanish registry as we believe data will be more representative of a multicentric approach and with prospective experience so we could overcome the limitations mentioned above.

CONCLUSIONS

The CRISP system is a relatively simple tool for risk assessment before catheterization in the CHD domain. Despite our study assessed the risk of stratification based on a single-center database, this model has proven accurate. Therefore, we are confident that this score could also be extrapolated to all pediatric populations in Spain. We strongly believe that this scoring system can become a handy tool for risk prediction, thus planning and preparing procedures in advance. In addition, it can be used for benchmarking. Our outcomes reveal that we perform highly complex procedures with similar results compared to those previously published. Finally, we suggest the use of CRISP before cardiac catheterization for procedural risk assessment planning.

FUNDING

This research did not receive specific grants from any funding agencies whether private or non-profit.

AUTHORS’ CONTRIBUTIONS

All authors reviewed all data analysis and the manuscript previous version, critically reviewed significant intellectual content, and approved the manuscript final version for publication. They’ve all made themselves accountable on all aspects of the study while making sure that questions associated with the accuracy or integrity of any part of the study are properly investigated and resolved.

CONFLICTS OF INTEREST

J.L. Zunzunegui Martínez is a proctor for PFM Medical and St. Jude Medical. The remaining authors declared no conflicts of interest whatsoever.

ACKNOWLEDGMENTS

The authors would like to acknowledge the important contribution made by the statistical department of Hospital General Universitario Gregorio Marañón, Madrid, Spain, and the nursing and technical staff involved in data collection and entry into the database.

REFERENCES