ASSIST Research Lab

Backed by NASA funding, Associate Professor Siddhartha Bhattacharyya and his Ph.D. students are advancing vision-based autonomous systems for aircraft or cars. Their research focuses on automating data labeling and developing a new framework for image-based machine learning models—paving the way for smarter, more capable autonomous systems.

I am super excited to share that the ATLAS Lab at Florida Tech, in collaboration with Siddhartha Bhattacharyya's ASSIST Research Lab at Florida Institute of Technology, has been selected for award of one of NASA's University Leadership Initiatives (ULI) focused on developing trustworthy autonomous agents for learning-enabled increasingly autonomous systems. We have a great multidisciplinary team including Amy Pritchett from Penn State, Sandip Ray from the University of Florida, Abdollah Homaifar from North Carolina A&T State University, Marco Pavone from Stanford, Cong Liu from Collin's Aerospace, Matthew Synborski from ResilienX, Melody Graveen, Ed.D. from Santa Fe Community College and Chris Fernando from Florida Tech. I can’t wait to get this research started!

I am delighted to share my experience at the Defense Advanced Research Projects Agency (DARPA) Intrinsic Cognitive Security (ICS) Principal Investigator Meeting in Los Angeles, California on January 7-8, 2025! This gathering brought together leading researchers and organizations to explore innovative solutions for securing systems against cognitive vulnerabilities in mixed reality systems. At the event, we had the privilege of presenting our work on MATRICS (Modeling and Analysis Toolkit for Realizable Intrinsic Cognitive Security). MATRICS focuses on integrating cognitive behavior models with formal analysis techniques to safeguard against cognitive cyber-attacks. It was exciting to discuss how our toolkit addresses vulnerabilities within perception, physiology, and attention, and to showcase our progress in developing mixed-reality prototypes and assurance mechanisms. We participated in dynamic working group sessions during the event, exploring guarantees and models for cognitive vulnerabilities while identifying synergies across various teams. The discussions reinforced the importance of integrating human factors into system design for secure solutions. Inspired by DARPA's vision for Mixed Reality Systems, MATRICS aligns with efforts to develop adaptive, intelligent technologies capable of mitigating cognitive threats in real-world contexts. It was empowering to share our progress and contribute to the broader conversation on cognitive security. Grateful to Defense Advanced Research Projects Agency (DARPA), our collaborators, Dr. Siddhartha Bhattacharyya, my fellow ASSIST Research Lab teammate, Candice Chambers, as well as the incredible community driving innovation in this space. Excited for what lies ahead! #CognitiveSecurity #MixedReality #DARPA #MATRICS #Innovation

Machine learning is, at its core, statistics in a much more sophisticated form. Traditional deep learning models perform well when the data they encounter during testing is statistically representative of what they were trained on. But what happens when the real world presents unexpected challenges—when the data distribution shifts, when the domains differ? As 2024 winds down, I’m excited to share some pictures from the kickoff of our project, Automated Data Labeling and Assurance for Aircraft Vision Systems (ADLAAAVS), at NASA Langley Research Center (LaRC), along with moments from our recent data collection flights at MLB Orlando International Airport and Grant Valkaria Airport. We are exploring how Transfer Learning can elevate vision-based classification for autonomous systems across different domains. Imagine the lane markings on a highway compared to those on a taxiway or runway. Both guide movement, but they differ in lane widths, patterns, surface textures, and environments. Highways have multiple lanes for various vehicles, while taxiways and runways are designed specifically for aircraft with unique marking conventions and fewer lanes. Can a model trained to navigate highway lanes excel in interpreting those on a taxiway or runway? Our research focuses on understanding the common knowledge between these domains and determining how much of it can or should be transferred for a model to perform well in a fundamentally different context. Additionally, we have begun exploring few-shot learning (FSL) approaches to enable models to perform well across domains with minimal labeled examples. This addresses domain-specific data dependencies and facilitates adaptation to diverse scenarios. By comparing and analyzing characteristics of car road lane markings against taxiway and runway markings, we aim to develop models that are not just effective but adaptable across domains. Through our research, we aim to generalize machine vision algorithms, enabling advances in perception-based systems. As we continue to test, explore, and push boundaries, I’m excited to see where this road (and runway) leads! A huge thank you to my professor Dr. Siddhartha Bhattacharyya for his mentorship, guidance, and the opportunity, and to Dr. Natasha Neogi Dr. Sarah Lehman, Parth Ganeriwala, and the team at ASSIST Research Lab for their continuous support, guidance and immense contributions to the project.

A Florida Tech team led by computer scientist Siddhartha Bhattacharyya has received funding from the renowned Defense Advanced Research Projects Agency (DARPA) to help develop ways to prevent cybersickness in a cognitive attack when using mixed reality (MR) systems. Their research is vital as it helps mitigate potential cognitive attacks before military personnel rely heavily on the MR systems. Learn about their approach and expectations, below.