iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: http://pubmed.ncbi.nlm.nih.gov/38073535/
Fusion Modeling: Combining Clinical and Imaging Data to Advance Cardiac Care - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Dec;16(12):e014533.
doi: 10.1161/CIRCIMAGING.122.014533. Epub 2023 Dec 11.

Fusion Modeling: Combining Clinical and Imaging Data to Advance Cardiac Care

Marly van Assen  1 , Amara Tariq  2 , Alexander C Razavi  1   3 , Carl Yang  4 , Imon Banerjee  2 , Carlo N De Cecco  1   5
Affiliations
Review

Fusion Modeling: Combining Clinical and Imaging Data to Advance Cardiac Care

Marly van Assen et al. Circ Cardiovasc Imaging. 2023 Dec.

Abstract

In addition to the traditional clinical risk factors, an increasing amount of imaging biomarkers have shown value for cardiovascular risk prediction. Clinical and imaging data are captured from a variety of data sources during multiple patient encounters and are often analyzed independently. Initial studies showed that fusion of both clinical and imaging features results in superior prognostic performance compared with traditional scores. There are different approaches to fusion modeling, combining multiple data resources to optimize predictions, each with its own advantages and disadvantages. However, manual extraction of clinical and imaging data is time and labor intensive and often not feasible in clinical practice. An automated approach for clinical and imaging data extraction is highly desirable. Convolutional neural networks and natural language processing can be utilized for the extraction of electronic medical record data, imaging studies, and free-text data. This review outlines the current status of cardiovascular risk prediction and fusion modeling; and in addition gives an overview of different artificial intelligence approaches to automatically extract data from images and electronic medical records for this purpose.

Keywords: artificial intelligence; big data; cardiac imaging techniques; heart disease risk factors; prognosis.

PubMed Disclaimer

Conflict of interest statement

Disclosures Drs De Cecco and van Assen receive funding from Siemens Healthineers. Dr De Cecco is a consultant of Covanos, Inc. and Bayer The other authors report no conflicts.

Figures

Figure 1.
Figure 1.
Coronary Artery Calcium (CAC) acquisitions and Coronary CT angiography (CCTA) allow a direct measure of atherosclerosis and plaque vulnerability features strongly associated with Cardiovascular Disease (CVD) risk. Panel A, Agatston calcium and MESA score are a powerful cardiovascular risk predictor. Panel B, CCTA can concurrently identify obstructive and non-obstructive atherosclerotic lesions. (LM- left main, LAD-Left Anterior Descending, CX- Circumflex, RCA- Right Coronary Artery)
Figure 2.
Figure 2.
AI-based algorithms for coronary evaluation and plaque burden quantification on CCTA. Panel A, AI prototype for automated plaque detection, stenosis severity quantification, and Coronary Artery Disease – Reporting and Data System (CAD-RADS) classification (HeartAI Siemens Healthineers). Panel B, plaque burden quantification and identification of high-risk plaque features (blue arrow) can be automated using AI approaches (Cleerly Inc). Panel C, shows an example of plaque component quantification (Elucid Bioimaging Inc.) and several imaging biomarkers of plaque vulnerability, that may be used for Cardiovascular Disease (CVD) risk stratification, including low plaque attenuation (C), positive remodeling (D), and spotty calcifications (E).
Figure 3.
Figure 3.
Cardiac MR AI-based algorithms for ventricular volume and function assessment can provide rapid and accurate analysis for cardiovascular risk prediction. Images show automated cardiac chambers segmentation for left ventricle volumes and segmental contraction evaluation (Circle Cardiovascular Imaging Inc.).
Figure 4.
Figure 4.
The use of demographics, risk factors, symptoms and clinical risk scores can be combined with information from non-invasive imaging test such as CAC and CAD-RADS to create a personalized treatment plan. This includes medication that is predicted to be more beneficial to specific patients, personalized treatment goals and follow-up schedules. (CTA- CT Angiography, CAC-Coronary Artery Calcium, CAD-RADS- Coronary Artery Disease – Reporting and Data System, RF- Risk Factors, DM2- Diabetes Mellitus type 2, CHD- Coronary Heart Disease, Lp(a)- Lipoprotein(a), ASA-Aspirin, LDL-C- Low-Density Lipoprotein Cholesterol)
Figure 5.
Figure 5.
By using the CT and MRI images for segmentation and anatomical modeling and using ECG and electrophysiology mapping to create a digital twin of the heart, the combination of these data sources can be used to identify ablation target and plan out the procedure and visualize the outcomes before the procedure to identify the most optimal procedure strategy. The virtual model can be integrated into the procedure to guide the actual procedure. Figures courtesy of Siemens Healthineers. (EP- Electrophysiology, ECG- Electrocardiogram, EAM-Electroanatomic Mapping, VT-Ventricular Tachycardia)
Figure 6.
Figure 6.
Traditional fusion modeling approaches; late fusion, early fusion, and middle fusion. Late fusion methods can preserve the complete information of each data modality, but they cannot fully explore the interactions among data modalities. Early fusion methods, on the other hand, can potentially find complex cross-modality features but are often harder to properly supervise. In consequence, middle fusion methods offer a compromise, however, their designs are often ad hoc and require domain knowledge toward relations between the modalities
Figure 7.
Figure 7.
Graph neural network for fusion modeling. Different data elements may be used as node and edge feature vectors. Edge features vectors of two samples are evaluated for similarity to decide an edge between the nodes corresponding to these two samples. Graph neural network learns updated node representations based on original node features as well as edge structure formed based on edge features, hence, fusing the two data elements.
Figure 8.
Figure 8.
Broad classes of image processing models for cardiac imaging studies with main purpose of 1) segmentation, 2) numerical prediction and 3) image generation. Depending on the purpose, convolutional neural networks (CNNs) consists out of convolutional and fully connected layers and are the current standard to process imaging data.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Mensah GA, Roth GA, Fuster V. The Global Burden of Cardiovascular Diseases and Risk Factors: 2020 and Beyond. J Am Coll Cardiol. 2019;74:2529–2532. doi: 10.1016/j.jacc.2019.10.009 - DOI - PubMed
    1. Patel SA, Winkel M, Ali MK, Narayan KM, Mehta NK. Cardiovascular mortality associated with 5 leading risk factors: national and state preventable fractions estimated from survey data. Ann Intern Med. 2015;163:245–253. doi: 10.7326/M14-1753 - DOI - PubMed
    1. Vasan RS, Sullivan LM, Wilson PW, Sempos CT, Sundstrom J, Kannel WB, Levy D, D’Agostino RB. Relative importance of borderline and elevated levels of coronary heart disease risk factors. Ann Intern Med. 2005;142:393–402. doi: 10.7326/0003-4819-142-6-200503150-00005 - DOI - PubMed
    1. Arnett DK, Blumenthal RS, Albert MA, Buroker AB, Goldberger ZD, Hahn EJ, Himmelfarb CD, Khera A, Lloyd-Jones D, McEvoy JW, et al. 2019 ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;140:e563–e595. doi: 10.1161/CIR.0000000000000677 - DOI - PMC - PubMed
    1. Knuuti J, Wijns W, Saraste A, Capodanno D, Barbato E, Funck-Brentano C, Prescott E, Storey RF, Deaton C, Cuisset T, et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes. Eur Heart J. 2020;41:407–477. doi: 10.1093/eurheartj/ehz425 - DOI - PubMed

LinkOut - more resources