Select your language

Advanced Search

Impact Factor: 1.4

Article

Structural intervention

REC Interv Cardiol. 2024;6:20-24

Use of a pediatric risk score for cardiac catheterization in a Spanish population with congenital heart disease

Aplicación de una puntuación de riesgo pediátrico para cateterismo cardiaco en una población española con cardiopatía congénita

Paulo Éden Santos,a,. Fernando Ballesteros,b Alexandro Rodríguez,b and José Luis Zunzuneguib

aPaediatric Cardiology Department, Royal Manchester Children’s Hospital - Manchester Foundation Trust, Manchester, United Kingdom

bServicio de Cardiología Infantil, Centro Materno-Infantil – Hospital General Universitario Gregorio Marañón, Madrid, Spain

ABSTRACT

Introduction and objectives: Performing cardiac catheterization can be challenging regarding the management of congenital heart disease. Therefore, the use of risk scoring or grading systems can help us plan the procedure. Back in 2015, the Congenital Cardiac Interventional Study Consortium developed and validated a system called CRISP (Catheterization risk score for pediatrics), which predicted the risk of serious adverse events (SAEs) prior to cardiac catheterization. Our aim was to use and validate the same scoring system to predict SAEs associated with cardiac catheterization in a Spanish pediatric hospital.

Methods: A retrospective descriptive study was performed between January 2016 and May 2017. To create the area under the curve, the expected number of events was correlated with the overall number of cases (compared to the original CRISP). Pearson’s chi-square test was used to assess the performance of the scoring system.

Results: A total of 516 patients were successfully enrolled, 26.6% of whom were < 1 year-old [range, 1 day to 18 years], 56.5% were males, and 17% weighed < 5 kg. Around 63.3% of the procedures performed were percutaneous compared to 1.2% that were hybrid. A total of 40 SAEs were found to be amenable to immediate correction with no associated mortality. CRISP showed good discrimination with an area under the curve of 0.71 (95%CI, - 0.66-0.91) compared to the original score of 0.74, and adequate goodness of fit with Pearson’s chi-square test of 8.26 (P < .08).

Conclusions: Despite the performance of highly complex procedures, the rate of SAEs was similar to the one previously published. CRISP has proven to be a good benchmarking and risk stratification tool. Therefore, it can be successfully used in the Spanish pediatric population and have a positive impact on patient care like helping during pre- and post-catheterization care planning.

Keywords: Risk score Cardiac catheterization Congenital heart disease Pediatric population

RESUMEN

Introducción y objetivos: La realización de cateterismos cardiacos puede ser un reto en las cardiopatías congénitas, por lo que el uso de sistemas de puntuación o graduación del riesgo puede ayudar en la planificación del procedimiento. En 2015, el Congenital Cardiac Interventional Study Consortium desarrolló y validó un sistema llamado CRISP (Catheterization Risk Score for Pediatrics), que predecía el riesgo de eventos adversos graves previo a la realización del cateterismo cardiaco. Nuestro objetivo fue aplicar y validar el mismo sistema de puntuación para predecir eventos adversos graves relacionados con el cateterismo cardiaco en un hospital pediátrico español.

Métodos: Se realizó un estudio descriptivo retrospectivo desde enero de 2016 hasta mayo de 2017. Para obtener el área bajo la curva se correlacionó el número esperado de eventos con el número total de casos (comparados con el CRISP original). Se utilizó la prueba χ2 de Pearson para evaluar el desempeño del sistema de puntuación.

Resultados: Se consiguió captar a 516 pacientes, de los cuales el 26,6% eran menores de 1 año (rango 1 día a 18 años), el 56,5% eran varones y el 17% tenían un peso inferior a 5 kg. En torno al 63,3% fueron intervenciones percutáneas y el 1,2% fueron procedimientos híbridos. Se constataron 40 eventos adversos graves susceptibles de corrección inmediata, sin mortalidad asociada. CRISP mostró una buena discriminación, con un área bajo la curva de 0,71 (IC95% −0,66 a 0,91), en contraste con la del sistema original de 0,74, y una adecuada bondad de ajuste, con test de Pearson de 8,26 (p < 0,08).

Conclusiones: A pesar de la realización de procedimientos altamente complejos, los resultados en cuanto a eventos adversos graves fueron similares a los previamente publicados. CRISP ha demostrado ser una buena herramienta de evaluación comparativa y estratificación de riesgo, por lo que puede usarse con éxito en la población pediátrica española e impactar positivamente en la atención al paciente; por ejemplo, ayudando en la planificación de cuidados antes y después del cateterismo.

Palabras clave: Puntuación de riesgo Cateterización cardiaca Cardiopatía congénita Población pediátrica

Abbreviations

CCISC:Congenital Cardiac Interventional Study Consortium. CHARM:Congenital Heart Disease Adjustment for Risk Method. CHD:congenital heart disease. CRISP:Catheterization Risk Score for Pediatrics. C3PO:Congenital Cardiac Catheterization Project on Outcomes. rCRISP:refinement of CRISP.

INTRODUCTION

Before 2008, many attempts were made to create a standardized risk score to assess the likelihood of serious adverse events (SAEs) during pediatric cardiac catheterization. The field of pediatric cardiac catheterization has jointly designed and implemented several large, multicenter clinical registries including the Congenital Cardiac Interventional Study Consortium (CCISC) and the Congenital Cardiac Catheterization Project on Outcomes (C3PO).

The former registry started a project back in 2006 to define adverse events and risk adjustment.

This effort has yielded important information concerning the risk associated with the type of procedure being performed.1,2 Back in 2015, the CCISC developed and validated an empirical preoperative catheterization risk score for pediatrics (CRISP). A 21-point scale that estimates the risk of procedural-related SAEs defined as any event leading to mortality, permanent morbidity, need for further procedures or extended lengths of stay. The CRISP derived from nearly 15 000 procedures added to the CCISC database from 27 centers of North and South America and a few European centers (eg, Evelina London Children’s Hospital, London, United Kingdom).3

As this system has not been validated in most European countries—Spain included—we tried to use and validate the CRISP by assessing its predictive performance in 1 of the Spanish centers that runs pediatric cardiac catheterizations.

METHODS

The study was conducted after approval from our center Institutional Review Board and consent from the Research Ethics Committee to access patients’ relevant health records including data storage and confidentiality. The participants’ informed consent was not deemed necessary as it was a retrospective study handling anonymized data. We collected data retrospectively assessing all electronic and written health records available. We obtained information on all diagnostic and interventional catheterizations performed from January 2016 through May 2017. We excluded procedures with patients > 18 years-old, electrophysiological studies, transesophageal echocardiographies, vascular access, and pericardiocentesis.

Use of CRISP

We have translated the CRISP to Spanish without altering its original variables (note: although the tables may look different, the variables remain unchanged).

The patients’ characteristics and type of procedure used were recorded as follows (= CRISP variables): age, weight, pre-catheterization diagnosis, systemic illness/organ failure, hemodynamic variables, inotropic support, procedural category, and case type (diagnostic, interventional, hybrid).3

Both the physiological parameters and anatomical diagnoses were defined based on the Society of Thoracic Surgeons/European Association for Cardio-thoracic Surgery shortlist. We divided them into 3 levels of perceived increased risk, procedural category, and case type. Multiple procedures were assigned to the highest-risk category.1,3

SAE was defined as any adverse event requiring further procedures, extended lengths of stay, causing permanent morbidity or eventual death.3 The lead operator/investigator identified all SAEs and reviewed treatment modalities to apply severity definitions.

For practical purposes, Nykanen et al. suggested clustering the risk category into 5 different SAE risk groups: 1.0%, 2.6%, 6.2%, 14.4%, and 36.8%, respectively, rounded to the nearest whole percentage.3

The points assigned to each variable and the overall CRISP score were estimated and recorded in the database.

Statistical analysis

All data are expressed as percentages, mean, and median. SAEs were grouped according to severity as a percentage of the overall number of SAEs. These were estimated as the percentage of the overall number of procedures or population at risk for an event. All statistical analyses were performed using IBM SPSS Statistics statistical software platform, version 21.

To assess the performance of CRISP in our population, we created a receiver operating characteristic curve. To estimate its ability to predict SAEs, we applied Pearson’s chi-square test (P < .08 were considered significant).4

RESULTS

From January 2016 through May 2017, a total of 669 patients we identified in our exploratory database. After applying exclusion criteria, 516 patients remained, 26.6% of whom were < 1 year-old [range, 1 day to 18 years-old], and 56.2% were males. The mean and median weights were 23.2 kg and 19.0 kg with 17% of the patients < 5 kg [range, 1.1 kg to 82.0 kg]. Procedures were diagnostic, interventional, hybrid or cardiac biopsies in 20.3%, 62.8%, 0.8%, and 16.1%, respectively (table 1).

 

Table 1. Patient and procedural characteristics from January 1, 2016 through May 31, 2017

Variables N (%)
Age  
      < 1 months 32 (6.3%)
      1 to 11 months 105 (20.3%)
      < 1 years 137 (26.6%)
      1-9 years 216 (41.8%)
      10 to 18 years 165 (31.6%)
Male 290 (56.2%)
Female 226 (43.8%)
Weight  
      < 2.49 kg 15 (2.9%)
      2.5 to 4.9 kg 71 (13.8%)
      < 5 Kg 86 (16.7%)
      5 to 9 kg 72 (14%)
      > 10 kg 358 (69.3%)
Diagnosis  
      No structural heart diseasea 130 (25.2%)
      Pulmonary hypertension 11 (2.1%)
      Isolated defects 169 (32.8%)
      Complex defect with 2 ventricles 77 (14.9%)
      Complex defect with 1 ventricle 121 (23.5%)
      Noncardiac issuesb 8 (1.5%)
Procedural characteristics  
Case type  
      Diagnostic 105 (20.3%)
      Interventional 324 (62.8%)
      Hybrid 4 (0.8%)
      Biopsy 83 (16.1%)

a Examples include patients with cardiomyopathy or post-heart transplantation.
b Including the snare technique foreign body removal procedure.

 

All catheterizations were performed under general anesthesia. During the period at stake, 3 operators performed the procedures, which gives robustness and homogeneity to our results.

A total of 23% of the population included patients with single-ventricle physiology while 14.9% had complex CHD with biventricular anatomy (eg, outflow tract obstruction and intra-cardiac shunts). A total of 32% had isolated injuries like atrial septal defect, patent ductus arteriosus or valvular abnormalities. In the overall cohort, 25.2% and 1.5% had non-structural heart diseases (eg, cardiomyopathy or heart transplant) and non-cardiac conditions (eg, retrieval of a foreign body), respectively.

Most procedures were categorized as category risk 3 and 5 (67.6%). These are the highest risk categories, which shows the complexity of the procedures as risk categories increase. Our study area under the receiver operator curve (AUC) is 0.71, which is similar to the CRISP AUC of 0.74 (95% confidence interval (95%CI) of 0.66 to 0.91). No statistically significant differences were seen between the predicted risk and events obtained and Pearson chi-square test of 8.26 (P < .08).

Serious adverse events

A total of 41 SAEs (7.9%) were documented (description and frequencies shown are on table 2). First and foremost, of all the adverse events recorded, only 1 event per procedure was identified. No deaths or need for emergency surgery associated with cardiac catheterization were ever reported. Device embolization occurred in 6 patients; all were retrieved using percutaneous techniques. No device migration was reported after discharge. Arrhythmias requiring procedures were treated medically. Complete heart block was resolved with temporary pacemaker implantation.

 

Table 2. Rate of serious adverse events by proposed risk category

Risk category Procedures (%) Adverse events (%) Expected adverse events (%)
CRISP 1 135 (26.2%) 3 (7.3%) 1 (1.0%)
CRISP 2 107 (20.7%) 5 (12.2%) 3 (2.6%)
CRISP 3 175 (33.9%) 15 (36.6%) 11 (6.2%)
CRISP 4 81 (15.7%) 14 (34.2%) 12 (14.4%)
CRISP 5 18 (3.5%) 4 (9.7%) 7 (36.8%)

CRISP, catheterization risk score for pediatrics.

 

Two cases of unexpected cardiac arrest requiring emergency extracorporeal membrane oxygenation support were reported. A non-syndromic 18-month child (CRISP category 4; risk of SAE, 14.4%) with a post-natal diagnosis of Shones syndrome (critical aortic stenosis and ventricular dysfunction). He underwent multiple procedures including aortic angioplasty, pulmonary bi-banding, and PDA stenting in the neonatal period followed by Ross–Konno procedure when he turned 9 months old. During catheterization, he showed electromechanical dissociation without response to cardiopulmonary resuscitation requiring extracorporeal membrane oxygenation. After stabilization, stenting was performed in both pulmonary arteries, left anterior descending, and left circumflex coronary arteries.

The other patient was a 10-year-old with suspected myocarditis in the context of an influenza A infection and cardiogenic shock (CRISP category 5; risk of SAE, 36.8%) who required inotropic support during cardiac catheterization. The procedure included an atrioseptoplasty with cardiac perforation and cardiac tamponade. He underwent 2-min cardiopulmonary resuscitation that needed extracorporeal membrane oxygenation support.

Not surprisingly, the rate of SAEs increased with higher risk scores (table 3). Two patients received 19 points, but no one a maximum score (21 points).

 

Table 3. Adverse events

SAE Overall SAE, %
Hemodynamically unstable arrhythmia requiring pharmacological therapy 5 (12.2%)
Balloon rupture with no vascular damage or need
to interrupt the procedure
2 (4.9%)
Unexpected cardiac arrest with emergency ECMO support 2 (4.9%)
Cardiac injury (mild left atrial posterior wall dissection) 2 (4.9%)
Complete heart block unresolved at the end of the procedure 1 (2.4%)
Deaths 0 (0.0%)
Device migration requiring removal via cut down
or transcatheter retrieval
5 (12.2%)
Postoperative device migration 1 (2.4%)
Ductal dissection (wall hematoma without contrast extravasation) 1 (2.4%)
Hematoma requiring monitorization 4 (9.8%)
Hemodynamic instability with loss of pulse wave
and impaired LV function
1 (2.4%)
Mild hypotension 1 (2.4%)
Mild vascular access dissection (no need to interrupt
the procedure)
3 (4.9%)
TTE-related self-limited pulmonary bleeding 1 (2.4%)
Severe desaturation with bradycardia 1 (2.4%)
Severe hypotension with/or bradycardia 8 (17.1%)
Vasospasm 1 (2.4%)
Venous damage requiring transcatheter procedure 1 (2.4%)
  40 (7.7%)

ECMO, extracorporeal membrane oxygenation; SAE, serious adverse events; TTE, transthoracic echocardiogram.

 

DISCUSSION

The original CRISP has already proven to be a robust risk predictor of SAEs.4 Comparison with CCISC data was intentional trying to establish a benchmark. Therefore, it’s not surprising to see that many features and outcomes are analogous.

A potential drawback of CRISP is the introduction of physiological variables like pulmonary vascular resistance, right ventricular systolic pressure, and anemia requiring intraoperative transfusion. Although these variables can sometimes be estimated non-invasively beforehand, they may not be reliable as their true quantification is unknown before cardiac catheterization. A recent refinement of CRISP (rCRISP) published back in 2018 excluded those 3 physiological markers. However, it performed well regarding risk prediction with an AUC of 0.70 and observed/expected SAE ratios from 0.71 to 1.18. However, the revised model demonstrated a worse overall fit compared to the original CRISP (based on lower Akaike’s [AIC], Schwarz’s [BIC] information criteria, and -2log likelihood ratio [N2LL]).5

In this study, we have validated the CRISP scoring system, revealing decent predictive accuracy for SAEs. Our results are equivalent to the original scoring system confirmed by the agreement measure AUC 0.74 and 0.71 (CRISP and our study, respectively) [95%CI, 0.66-0.91].

We identified all SAEs recorded, those liable to immediate correction, and those that resulted in an unexpectedly extended length of stay, and need for medical or interventional treatment (whether surgical or transcatheter).

Although our study was implemented retrospectively, we would like to highlight that the corresponding risk categories in cases requiring extracorporeal membrane oxygenation were 4 and 5, the 2 with the highest risk of adverse events. If CRISP were available, these outcomes would be rather predictable. Therefore, greater attention would be paid to the need for highly differentiated care. Obviously, in the cases indicated, the outcome was very favorable. Still, the utility of this tool is evident in the preparation and anticipation of all the necessary care in more complex and high-risk procedures.

The mortality definition associated with CHD catheterization still needs to be elucidated since assessing mortality is challenging in this field.3 Although, in this study, mortality rate is 0% and minimal in other studies, it should be interpreted with caution.

Furthermore, physicians are understandably concerned with unfair comparisons of sensitive topics, eg, critical evaluation of unfavorable outcomes and events used for compensation and accreditation. Consequently, comparing centers or operators to identify the best practices can be facilitated, thus acknowledging that confidence intervals will be broader in low-volume centers.

Interestingly, if we compare the rate of adverse events by risk category/group, we’ll be seeing more complications from groups 1 to 4. Surprisingly or not, that is not true for group 5. These findings could be due to the smaller volume of patients in this group and the heterogeneity of complexity regarding the procedures performed in such group.

Still, CRISP has an appropriate goodness of fit (Person’s chi-square test of 8.26 (P < .08).

Knowing that extrapolations are informal, and unquantified arguments are limited by assumptions,6 we still believe that CRISP can be extrapolated to all pediatric populations undergoing cardiac catheterization in Spain as our results are rather consistent with the original study.

As a matter of fact, these data show that CRISP positively impacts patient care as it can be used to plan procedures in advance. It helps prepare the required equipment, technical staff, need for special care after the catheterization like intensive care unit admission. It can also be used reliably for parent counseling before each procedure.5

Study limitations

In addition to the limitations inherent to retrospective studies, a substantial data limitation is the lack of systematic assessment of procedural success as in former studies. To this date, a standardized assessment of procedural success is lacking, but it will likely be the focus of future studies.3

For a more inclusive analysis, we disclosed all SAEs recorded, although adverse events are readily identified outcomes and perceived to have a meaningful impact on patient outcomes. However, without systematic auditing, there’s still a potential for loss of adverse events, especially after patient discharge.

It can be argued that sample size is crucial for a more accurate outcome and less estimation error. However, more is only, sometimes, better in sample sizing since we can estimate size effect based on previously reported or preclinical studies. For future reference, in retrospective and observational studies, if the original study already proved to be a strong predictor, a smaller sample would be adequate to prove the same effect.7

We aim to conduct a prospective registry to refine the analysis and make this model more robust in our center. In conclusion, it would be interesting to join a nationwide Spanish registry as we believe data will be more representative of a multicentric approach and with prospective experience so we could overcome the limitations mentioned above.

CONCLUSIONS

The CRISP system is a relatively simple tool for risk assessment before catheterization in the CHD domain. Despite our study assessed the risk of stratification based on a single-center database, this model has proven accurate. Therefore, we are confident that this score could also be extrapolated to all pediatric populations in Spain. We strongly believe that this scoring system can become a handy tool for risk prediction, thus planning and preparing procedures in advance. In addition, it can be used for benchmarking. Our outcomes reveal that we perform highly complex procedures with similar results compared to those previously published. Finally, we suggest the use of CRISP before cardiac catheterization for procedural risk assessment planning.

FUNDING

This research did not receive specific grants from any funding agencies whether private or non-profit.

AUTHORS’ CONTRIBUTIONS

All authors reviewed all data analysis and the manuscript previous version, critically reviewed significant intellectual content, and approved the manuscript final version for publication. They’ve all made themselves accountable on all aspects of the study while making sure that questions associated with the accuracy or integrity of any part of the study are properly investigated and resolved.

CONFLICTS OF INTEREST

J.L. Zunzunegui Martínez is a proctor for PFM Medical and St. Jude Medical. The remaining authors declared no conflicts of interest whatsoever.

ACKNOWLEDGMENTS

The authors would like to acknowledge the important contribution made by the statistical department of Hospital General Universitario Gregorio Marañón, Madrid, Spain, and the nursing and technical staff involved in data collection and entry into the database.

WHAT IS KNOWN ABOUT THE TOPIC?

  • Several reports suggested risk factors regarding adverse events associated with pediatric cardiac catheterizations. The C3PO study group started a project in the United States back in 2006 that has yielded important information concerning the risk associated with the type of procedure. Also, it developed the multivariate CHARM model for outcome risk adjustment. On the other hand, the CCISC group developed a pre-catheterization risk scoring system for individual pediatric patients undergoing cardiac catheterization procedures.

WHAT DOES THIS STUDY ADD?

  • The catheterization risk scoring system for the pediatric population is a relatively simple model that has yet to be sampled and validated in many European countries. We tested and proved that this RISK score is valid in our subgroup and could be used, in the near future, in all Spanish pediatric populations undergoing cardiac catheterizations. This score can be a handy tool to predict risk before cardiac catheterization, which helps prepare pre- and postoperative care, thus positively impacting patient care.

REFERENCES

1. Bergersen L, Gauvreau K, Foerster SR, et al. Catheterization for Congenital Heart Disease Adjustment for Risk Method (CHARM). JACC Cardiovasc Interv. 2011;4:1037-1046.

2. Bergersen L, Gauvreau K, Marshall A, et al. Procedure-type risk categories for pediatric and congenital cardiac catheterization. Circ Cardiovasc Interv. 2011;4:188-194

3. Nykanen DG, Forbes TJ, Du W, et al. Congenital Cardiac Interventional Study Consortium (CCISC). CRISP: Catheterization RISk score for Pediatrics: A Report from the Congenital Cardiac Interventional Study Consortium (CCISC). Catheter Cardiovasc Interv. 2016 ;87:302-309.

4. Burnbaum KP, Anderson DR. Model Selection and Multimodal Inference: A Practical Information-Theoretic Approach. 2nd ed. New York, NY: Springer-Verlag; 2002. P 68-78.

5. Hill KD, Du W, Fleming GA, et al. Validation and refinement of the catheterization RISk score for pediatrics (CRISP score): An analysis from the congenital cardiac interventional study consortium. Catheter Cardiovasc Interv. 2019;93:97-104.

6. Benford D, Halldorsson T, John M, et al. EFSA Scientific Committee. Guidance on Uncertainty Analysis in Scientific Assessments. EFSA J. 2018;16:e05123.

7. Kadam P, Bhalerao S. Sample size calculation. Int J Ayurveda Res. 2010;1:55-57.

* Corresponding author.

E-mail address: (P. Éden Santos).

PreviousNext

Original articles


Review Articles


Original articles

Editorials


Special articles

Debate

“Orbiting” around the management of stable angina

Editorials

Review Articles

Scientific letters

Images in cardiology