MIT Researchers Develop AI Tool That Compares 3D Scans

A team of researchers from the Massachusetts Institute of Technology (Cambridge, MA, USA) have submitted a pair of conference papers describing a machine-learning algorithm that can register brain scans and other 3D images more than 1,000 times more quickly using novel learning techniques. The algorithm works by “learning” while registering thousands of pairs of images and acquiring information during this process about how to align the images and estimating some optimal alignment parameters. After training, the algorithm uses those parameters to map all pixels of one image to another simultaneously. This reduces registration time to a minute or two using a normal computer, or less than a second using a GPU with accuracy similar to state-of-the-art systems.

In order to analyze variations in brain structures in patients with a particular disease or condition, neuroscientists often use the medical image registration technique. This involves overlaying two images, such as magnetic resonance imaging (MRI) scans, to compare and analyze the anatomical differences in great detail. However, this process usually takes two hours or more, as traditional systems meticulously align each of potentially a million pixels in the combined scans. Since MRI scans are basically hundreds of stacked 2D images that form massive 3D images, called “volumes,” containing a million or more 3D pixels, called “voxels,” it can be highly time-consuming to align all voxels in the first volume with those in the second. Additionally, scans coming from different machines and having different spatial orientations can make matching voxels even more computationally complex.


The researchers’ algorithm, called “VoxelMorph,” is powered by a convolutional neural network (CNN), a machine-learning approach commonly used for image processing. These networks consist of several nodes that process image and other information across various layers of computation. The researchers trained their algorithm on 7,000 publicly available MRI brain scans and then tested it on 250 additional scans. During training, the researchers fed the brain scans into the algorithm in pairs. Using a CNN and modified computation layer called a spatial transformer, the method captures similarities of voxels in one MRI scan with voxels in the other scan. This allows the algorithm to learn information about groups of voxels, such as anatomical shapes common to both scans, which it uses to calculate optimized parameters that can be applied to any scan pair.

When two new scans are fed into the algorithm, a simple mathematical “function” uses those optimized parameters to rapidly calculate the exact alignment of every voxel in both scans. Thus, the algorithm’s CNN component gains all the necessary information during training so that, during each new registration, the entire registration can be executed using one, easily computable function evaluation. The researchers found that their algorithm could accurately register all of their 250 test brain scans – those registered after the training set – within two minutes using a traditional central processing unit, and in under one second using a graphics-processing unit. What is particularly noteworthy is that the algorithm is “unsupervised,” which means it does not require additional information apart from image data. Some registration algorithms incorporate CNN models but require a “ground truth,” which means that another traditional algorithm is first run to compute accurate registrations. However, the algorithm developed by the MIT researchers maintains its accuracy without that data.

In addition to analyzing brain scans, the speedy algorithm could find a wide range of potential applications, according to the researchers. For instance, other researchers at MIT are currently running the algorithm on lung images. The algorithm could also allow for image registration during operations and allow surgeons to potentially register scans in near real-time, getting a much clearer picture on the progress of the surgery.

"This is a case where a big enough quantitative change [of image registration] – from hours to seconds – becomes a qualitative one, opening up new possibilities such as running the algorithm during a scan session while a patient is still in the scanner, enabling clinical decision making about what types of data needs to be acquired and where in the brain it should be focused without forcing the patient to come back days or weeks later," said Bruce Fischl, a professor in radiology at Harvard Medical School and a neuroscientist at Massachusetts General Hospital.
The papers were presented by the MIT researchers at the Conference on Computer Vision and Pattern Recognition (CVPR) and at the Medical Image Computing and Computer Assisted Interventions Conference (MICCAI).

Related Links:
Massachusetts Institute of Technology