Through the Looking Glass: Using Artificial Intelligence to Recognize Cerebral Malaria in African Children

by Vinayak Joshi and Sarah Soliz

The idea that the retina is the window to the human brain may be a truism in the medical community, but the practical implications of this idea are more significant than ever. Since 2014, VisionQuest Biomedical Inc. has been working with leading researchers in tropical medicine to develop inexpensive and easy-to-use artificial intelligence–based software and portable retinal cameras capable of detecting the retinal abnormalities that indicate cerebral malaria. This system, known as ASPIRE, provides immediate feedback and allows any health-care provider to make an accurate diagnosis and ensure the correct treatment.

Together with doctors from Malawi, Zambia and Uganda, we are finding ways to reduce child mortality in regions of the world affected by malaria. We are applying artificial intelligence technology to enable existing health-care systems to meet the needs of the population. Doctors Terrie Taylor, Susan Lewallen, and Simon Harding have been instrumental in the process. “It all started,” says Taylor, with a “locally identified priority” and entrée provided by local doctors, which “gave us a platform on which to work.” Lewallen adds:

VisionQuest had something marvelous to offer doctors. When Vinayak approached me with an idea for developing a camera with an algorithm that could analyze pictures and determine whether [a child had] malarial retinopathy or not, I was very skeptical. Sometimes it’s nice to be proven wrong.

A Picture Begins to Emerge

As a young adult, Taylor accompanied her high-school church group on mission trips and became interested in the challenges faced by people living in resource-poor environments. Years later, when she was trying to decide what area of medicine to specialize in, she had the opportunity to work in the Sudan. She was struck by the improvisational nature of the Sudanese doctors’ work—their ability to develop workarounds to the problems they faced. Yet their flow of patients was so continuous, they were unable to step outside that flow to ask larger questions or address the source of their problems.

One of those problems is, of course, malaria. Cerebral malaria kills hundreds of thousands of children every year and is also a leading cause of neuro-disability in sub-Saharan Africa. For a decade, Taylor worked at the bedside of her patients in Malawi, studying the characteristics of cerebral malaria in the children she treated and always trying to improve their chances of survival. Eventually, she and her colleagues hit a wall: they couldn’t do more without knowing more. Over the next ten years, Taylor explored the shortcomings of the clinical diagnosis, seeking to understand why children who met the WHO criteria for cerebral malaria and were treated appropriately were still dying.

One evening, Taylor was having dinner with Lewallen, the only ophthalmologist in the southern part of the country at that time, and telling her about a child who had died that day. “What’s the pathology?” Lewallen asked, but no one knew. She then asked if she could examine the eyes of the children who had died under similar circumstances. Lewallen began visiting Taylor’s patients in the evenings and recording her retinal findings, several of which had never been described before. At the end of that malaria season, she had enough data to see that the children with the unusual retinal features were dying, whereas the ones with normal retinas had a higher rate of survival. As Lewallen and Taylor began to publish their findings on what Lewallen called malarial retinopathy, other interested doctors, including Professor Harding, directed resources and expertise toward solving the problem. After their studies were complete, a picture had begun to emerge: malarial retinopathy was a compelling indicator of cerebral malaria, that is, malaria parasites in the blood vessels of the brain.

Dr. Manda performing retinal imaging in the malaria ward in Malawi.

Following their discovery, however, the doctors could see that some children who had been clinically diagnosed with cerebral malaria but who had no sign of malarial retinopathy were still dying. In other words, they were still missing something. At least one study has reported that the WHO diagnostic criteria misclassify up to 39 percent of cerebral malaria cases, and this significant false positive rate leads to inadequate treatment of the underlying cause of illness. The missing piece of the cerebral malaria puzzle was overdiagnosis, as Lewallen and Taylor discovered through their autopsy study.

In 1993 Lewallen published her findings on the retinal lesions associated with cerebral malaria, and since then other researchers have confirmed the lesions’ unique association with cerebral malaria and thus their importance in the accurate diagnosis of this disease: comatose, malarial children with malarial retinopathy have cerebral malaria; comatose, malarial children without malarial retinopathy do not have cerebral malaria and can be investigated further for other non-malarial illnesses and treated accordingly. When combined with the WHO criteria, detection of malarial retinopathy by an ophthalmologist increases the accurate diagnosis of cerebral malaria to 95 percent or greater and means that children without cerebral malaria can be treated as quickly as possible for a range of other illnesses.

Lewallen became an ophthalmologist because she liked the directness of examination rather than the interpretation of test results: “You make your diagnosis by looking . . . and we have these beautiful tools in ophthalmology that allow you to examine the eye.” But in a country with only a handful of ophthalmologists to serve a population of over 18 million, the lack and cost of those tools can become barriers to diagnosis and care.

Overcoming Barriers to Treatment

For the past several years VisionQuest has been working with Taylor, Lewallen, and others to develop new tools that overcome barriers to access. We are testing the cameras and AI techniques that make up the ASPIRE system and that can be used by anyone, anywhere: doctors, nurses, and community health-care workers in the hospital and in the field. The portable cameras and laptop- or cloud-based software are unique. They use algorithms that we have engineered to detect the biomarkers of malarial retinopathy—hemorrhage, whitening, and vessel discoloration—and to provide results in real time. In other words, a health-care worker who travels to a rural clinic with the camera and software in their backpack can take retinal photographs, process the images, and receive the results of the analysis all in a few moments.

When things are like that, patients are more willing to accept treatment. If it means they can go to a district hospital, for example, be seen quickly, [have] pictures taken, a diagnosis made immediately—they’ll be more willing to come back for another checkup. For most of our patients, when there’s good news, they spread the good news. —Dr. Moira Gandiwa, ophthalmologist in Malawi

The collaboration between us—the doctors in Malawi and scientists at VisionQuest—could be characterized in a number of ways: it is a unique application of artificial intelligence and personalized medicine in the developing world; it is a project that aims to support, not supplant, the Malawian health-care system by putting powerful diagnostic tools into the hands of local caregivers; it is an effort to bring inexpensive, portable, easy-to-use technology into remote and underserved communities; and above all, it is an attempt to save children’s lives.

The Road Ahead

Working in Malawi is exciting, says Taylor, because of the way that the results of scientific research are quickly transformed into policy. With funding from the US National Institute of Allergy and Infectious Diseases and support from the Malawi Ministry of Health, we have been working with Taylor and other doctors in Malawi to demonstrate the effectiveness of ASPIRE in diagnosing malarial retinopathy and screening out other confounding diseases that are common in sub-Saharan Africa. This process is also enabling us to make improvements and prepare for commercialization. We plan to launch the product throughout the areas of Africa and the world where cerebral malaria harms so many, particularly children. We also plan to add previously developed tools to the ASPIRE system to increase its usefulness and cost-effectiveness in Malawi and surrounding countries. We are currently approaching NGOs and other organizations that would be able to assist us in reaching the populations most in need of this singular technology. A tool like this, observes Lewallen, “allows somebody who’s not an ophthalmologist to take account of specific eye findings, yes, and to treat patients more holistically.”

The first time VisionQuest’s Joshi worked on developing software for malarial retinopathy diagnosis was during his PhD program, but at that point it was just another piece of code. The real work began when he visited a malaria clinic in Malawi. When he actually saw a child dying of cerebral malaria, probably due to misdiagnosis, he realized the value of the work. He realized that rather than focusing only on getting the best scientific results, he would also need to think about developing the tools to ensure that the solution would be affordable and accessible to the patients who actually need it. The work has been challenging, both technically and personally, as well as incredibly rewarding. Says

We’ve taken the first steps toward solving an important problem, and we’re ready to put this technology into the hands of the people who need it most.