Database with Physician Input and Innovative Eye-Tracking Techniques Devised

Digital archives of biomedical images could soon put vital data at physicians' fingertips within seconds, demonstrating how computers can improve the manner in which medicine is practiced.

The current reality, however, is not quite there yet, with databases virtually overwhelmed by the explosion of medical imaging. Rochester Institute of Technology (NY, USA) professor Anne Haake recently won grants from the U.S. National Science Foundation (NSF; Arlington, VA, USA) and the U.S. National Institutes of Health (Bethesda, MD, USA) to address this problem. Prof. Haake foresees an image database built on input from the intended end-users and designed from the beginning with flexible user interfaces. She and her interdisciplinary team will develop a prototype using input from dermatologists to refine the search process for images of various skin conditions.

"We need to involve users from the very beginning,” said Prof. Haake, professor of information sciences and technologies at the B. Thomas Golisano College of Computing and Information Sciences. "This is especially true in the biomedical area where there is so much domain knowledge that it will be specific to each particular specialty.”

Prof. Haake understands the genuine need to make biomedical images useful. She began her career as a developmental biologist before pursuing computing and biomedical informatics. This project combines her two strengths and was inspired by research she conducted while on sabbatical at the NIH National Library of Medicine.

Dr. Cara Calvelli, a dermatologist and a professor in the physician assistant (PA) program in RIT's College of Science, has recruited dermatologists, residents, and PA students for the project. She is also helping to describe accurately the sample images, some of which come from her own collection. "The best way to learn is to see patients again and again with various disorders,” Dr. Calvelli stated. "When you can't get the patients themselves, getting good pictures, and learning how to describe them is second best.”

Funding Prof. Haake won from the NSF will support visual perception research using eye tracking and the design of a content-based image retrieval system accessible through touch, gaze, voice, and gesture; the NIH portion of the project will be used to fuse image understanding and medical knowledge.

Bridging the "semantic gap” is the challenge facing researchers working in content-based image retrieval, according to Prof. Haake. Search functions can go awry when computer engineered algorithms trip on nuances and fail to differentiate between disparate objects, such as a ship and a whale. Constructing a system based on end-user knowledge can prevent semantic problems from occurring.

Pengcheng Shi, director for graduate studies and research in the Golisano College, is providing his expertise in image understanding. "For many years computing/technical people have said we can write algorithms such that it will work,” he noted. "But people start to realize that machines are not all that powerful. At the end of the day we need to put the human back into it. What are the physicians looking at and how are they looking at it in order to make their decisions?”

Related Links:
Rochester Institute of Technology