Multimodal AI is the Future of Health, Wellness and Biomedicine

?“The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease.” This @NatureMedicine is a must-read article! https://www.nature.com/articles/s41591-022-01981-2

Here are my top 5 takeaways:

1- From external to internal Human Insight AI! I’ve spent most of my career researching and developing outward signals for understanding humans. Examples include optical sensors / cameras and computer vision to analyze facial expressions and emotion, voice data to understand prosody and intonation, and wearable physiological sensors that measure skin conductance, heart rate and heart rate variability. But there are novel technologies that measure the INSIDE state of humans. These are often referred to as the omes’, and include the genome, proteome, transcriptome, immunome, epigenome, metabolome and microbiome. One of my investments, ReviveMed, use machine learning to map our metabolites (cholesterol, glucose etc) and accelerate drug discovery. Leila, the founder and CEO, is an awesome Iranian-American computational biologist who spun out of MIT.

2- Multi-modal Human Insight AI is the future! The best (human) doctors use multi-modal data to diagnosis disease. They integrate disparate and complex data from multiple disciplines to make an accurate diagnosis and personalize treatment. That is obviously challenging to do because the data is often not available in a way that is easily consumable, and science is advancing so fast making it difficult for doctors to stay on top of all these advances. AI, however, can pull these disparate datasets together, and ensure that the latest algorithms are integrated into the decision-making. This is no easy feat because the datasets are often disparate and distributed. But herein lies the opportunity for innovation - what startups are building this multi-modal platform?

3- Access to health services from the comfort of your home! Telehealth, remote patient monitoring, virtual health assistants, social robot nurses at home, and even digital clinical trials are already transforming how our health services get delivered, increasing accessibility. It is still early days for these services, re: scale of deployment and what data they use to make decisions. It is not hard to imagine an AI digital coach that integrates multiple data sources, such as wearable sensors, environmental sensors and continuous monitoring of blood biomarkers and metabolites, to promote behavior change, answer health-related questions, triage symptoms or communicate with healthcare providers when needed. Today in mental health, the gold standard is STILL a survey “On a scale from 1-10, how depressed are you? How suicidal?” But we know that there are facial, vocal, physiological biomarkers of mental health disease. VideraHealth is one of the companies I am excited about in the remote patient monitoring space for intervention in mental health, addiction and more.

4- Multimodal Data and Machine learning pipeline is a must. Most ML pipelines today cannot handle multi-modal data (why we @SmartEye acquired @iMotions, as they have spent the past 15 years building a multi-modal platform). Data and ML pipelines need to incorporate multi-modality from the ground up; it is almost impossible to build that in after the fact. Data acquisition needs to synchronize different sensors and signals; 2) Preprocessing to streamline this mess of data becomes key; 3) Data warehouses need to allow for different biobanks, 4) ML models need to be re-trained to incorporate multiple signals (not easy since signal might have different granularity, time domains) and finally 5) new visualization tools are needed to bring multimodal insights to life. Who is building this??

5- Data privacy and ethic are paramount: Finally, privacy is paramount. So is the ethical development and deployment of these technologies. Much of the data in medicine is biased due to underrepresentation in clinical trials and biomedical research, which s leads to disparity and bias in diagnosis and treatment. So key to be intentional about mitigating these biases.