Largest Model Trained On Echocardiography Images Assesses Heart Structure and Function

Foundation models represent an exciting frontier in generative artificial intelligence (AI), yet many lack the specialized medical data needed to make them applicable in healthcare settings. While there are existing AI models for echocardiograms, these are typically trained on relatively small datasets comprising tens of thousands of examples. In a significant advancement, a team of AI experts has now compiled a dataset that includes over one million echocardiograms, or cardiac ultrasound videos, complete with clinical interpretations. Utilizing this vast database, they have developed EchoCLIP, a sophisticated machine-learning algorithm capable of interpreting echocardiogram images and analyzing crucial findings.

The team of investigators at Cedars-Sinai (Los Angeles, CA, USA) built a dataset of 1,032,975 cardiac ultrasound videos and corresponding expert interpretations. This extensive collection enabled the development of EchoCLIP, which offers clinician-level evaluations of heart function, past surgeries, and implanted devices. Moreover, EchoCLIP can identify a single patient across multiple videos and timepoints, recognizing clinically significant changes in heart conditions. EchoCLIP holds promise in revolutionizing how cardiologists assess echocardiograms by providing preliminary cardiac assessments, tracking changes over time, and identifying common cardiac conditions.

Image: The powerful machine learning algorithm can “interpret” echocardiogram images and assess key findings (Photo courtesy of 123RF)

In studies, EchoCLIP has demonstrated high accuracy in measuring cardiac function and identifying devices such as pacemakers and repaired mitral and aortic valves. Additionally, it has proven capable of recognizing unique patients across different studies and detecting important clinical changes like post-surgical modifications. The development of EchoCLIP has also facilitated the generation of preliminary text interpretations of echocardiogram images, further enhancing its utility in clinical settings.

“To our knowledge, this is the largest model trained on echocardiography images,” said corresponding author David Ouyang, MD, a faculty member in the Department of Cardiology at the Smidt Heart Institute. “EchoCLIP’s uniquely strong performance in image interpretation is a result of its training on almost tenfold more data than existing models. Our results suggest that large datasets of medical imaging and expert-adjudicated interpretations can serve as the basis for training medical foundation models, which are a form of generative artificial intelligence.”

Related Links:
Cedars-Sinai