The AirLab develops sensing, localization, perception and motion planning algorithms for autonomous aerial robots ("drones") to enable them to operate without requiring human operators in challenging environments.
The AirLab at Carnegie Mellon University的外部链接
5000 Forbes Ave
US，Pennsylvania，Pittsburgh，15213
The AirLab at Carnegie Mellon University转发了
CMU Ph.D. Student in Robotics, Studying Neuro-Symbolic AI for safe/interpretable Robot Behavior
Humans can learn to reason in an "unfamiliar" world, like new games. How far are LLMs from this? Check out our recent work @NeurIPS2024 D&B Track: "LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulation". Page: https://lnkd.in/gyRWgtC7
How can we drive fast ????? in new offroad environments ?? where we don't have data? Check out Eric Chen and Cherie Ho's work on ALTER at IROS! ??
Excited to present ALTER at IROS 2024! ???????How can we drive fast in new offroad environments where there is limited data? We quickly adapt a visual model in <1 minute in a new environment, by learning "on-the-drive". Online Learning from LiDAR + Model Selection = Good long-range prediction in < 1 minute Introducing ALTER, an online self-supervised perception framework that adapts rapidly to new environments by learning from current LIDAR geometry data for robust long-range visual perception. We use pre-trained image features and model selection to enable fast training and to retain the knowledge of past learned environments. Amazing collab with Eric Chen, Mukhtar Maulimov, Chen Wang and Sebastian Scherer! ?? Actionable insights even if you don't work on offroad driving! ? Have complementary sensors that fail in different situations? Have them learn from each other! Use online self-supervised labeling to bridge both sensors. ? Need fast reactions to previously learned scenarios? Apply ALTER’s model selection method! Keep track of an ensemble of models to avoid forgetting. ? Need good fallback when a sensor fails? Select between sensors and model online using embedding similarity and model loss jointly as the signal. Check out our paper, poster, and video to learn more! Paper title: Learning-on-the-Drive: Self-supervised Adaptation of Visual Offroad Traversability Models ?? arXiv: https://lnkd.in/giCv9h8K? ?? Video: https://lnkd.in/g_BEGw_D #AirLab #CMU #cmurobotics #HarveyMudd #riss #robotics #ComputerVision #fieldrobotics
?? Ever wondered how to film a group of people with pixel-level precision, even as they split, join, and rearrange? Check out our work on “Greedy Perspectives: Multi-Drone View Planning for Collaborative Perception in Cluttered Environments.” presenting at IROS 2024. ?? Website:?https://lnkd.in/eSR-y7gt ?? arXiv:?https://lnkd.in/eWHmYkrs ?? We directly optimize views and drone motions over the planning horizon using greedy algorithms for submodular maximization. For pixel-level reasoning, we render views with OpenGL, skipping distance-based approximations and instead using a proxy for the reconstruction metric. ?? Greedy Perspectives distributes pixels across multiple actors in cluttered scenes, accounting for obstacles and occlusions, and outperforms traditional formation planning. Authors: Krishna Suresh, Aditya Rauniyar, Micah Corah, and Sebastian Scherer #AirLab #CMU #CSMines #robotics #drones
Can we inject symbolic rules into learning-based planners to improve out-of-distribution robustness? ?? We’re excited to introduce RuleFuser, a novel method for integrating traffic rules into IL-based motion planners using evidential deep learning. ?? RuleFuser is developed on the key insight that learning-based planners excel in in-distribution (ID) scenarios while rule-based planners provide greater robustness in out-of-distribution (OOD) scenarios. Integrating these two into a framework that strategically deploys each planner in the appropriate regime can enhance overall performance. ?? We’ve tested RuleFuser on the nuPlan dataset, where it adapts effectively to OOD situations, achieving safety levels beyond those of IL or rule-based planners alone. ?? This work is a collaboration between the Autonomous Vehicles Research Group at NVIDIA and The AirLab at Carnegie Mellon University?? Congratulations Jay Patrikar Sushant Veer Apoorva Sharma Marco Pavone Sebastian Scherer Interested to learn more? ?? Paper: “RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners for Robustness under Distribution Shifts” Link: https://lnkd.in/dYTwv4Q5 #AutonomousDriving #AI #MachineLearning #AVSafety #SafeAI
The AirLab at Carnegie Mellon University转发了
?? Meet Amelia,? our largest dataset and model for multi-agent, multi-airport Aircraft Trajectory Prediction! ?? Introducing “Amelia ??:? A Large Dataset and Model for Airport Surface Movement Forecasting! ?? In a world where our skies are more crowded than ever, we face airport congestion and frequent close calls between aircraft. Can we leverage generative AI to design safer and more efficient NAS? ?? What makes up Amelia? ? Amelia-48: More than a year’s worth of curated data from 48 airports across the US National Airspace System, totaling a staggering ~50TB of raw data to date. ? Amelia-TF: A transformer-based model trained on 300 days (over 9 billion tokens) of data for multi-agent, multi-airport trajectory forecasting. ? Amelia Toolkit: A comprehensive set of tools to download, process, analyze, and visualize aviation trajectory data! This project is a collaboration between the CMU BIG @ Carnegie Mellon University Robotics Institute and The AirLab at Carnegie Mellon University! ??Here’s to ensuring safer and more efficient skies! Congratulations to Ingrid Navarro* Pablo Ortega Kral* Jay Patrikar* Haichuan(Jack) Wang ZELIN YE Jong Hoon Park, EIT Jean Oh Sebastian Scherer! Read more:? ???Website: https://lnkd.in/djn8Tw7H? ??Paper: https://lnkd.in/dmPhfEha? ????Codebase: https://lnkd.in/dbTg3NCF
Congratulations to Team Amelia for winning the Best Student Paper Award at AIAA AVIATION 2024!
Congratulations to Ingrid Navarro, Pablo Ortega Kral, Jay Patrikar, Haichuan(Jack) Wang, ZELIN YE, Jong Hoon Park, EIT, Jean Oh, and Sebastian Scherer for winning Best Student Paper at the 2024 American Institute of Aeronautics (AIAA) Conference! ?? The paper, a collaboration between the CMU BIG lab and The AirLab at Carnegie Mellon University, was recognized in the Air Transportation Systems technical discipline. Read: "Amelia: A Large Model and Dataset for Airport Surface Movement Forecasting" https://lnkd.in/dmPhfEha
?? Meet Amelia,? our largest dataset and model for multi-agent, multi-airport Aircraft Trajectory Prediction! ?? Introducing “Amelia ??:? A Large Dataset and Model for Airport Surface Movement Forecasting! ?? In a world where our skies are more crowded than ever, we face airport congestion and frequent close calls between aircraft. Can we leverage generative AI to design safer and more efficient NAS? ?? What makes up Amelia? ? Amelia-48: More than a year’s worth of curated data from 48 airports across the US National Airspace System, totaling a staggering ~50TB of raw data to date. ? Amelia-TF: A transformer-based model trained on 300 days (over 9 billion tokens) of data for multi-agent, multi-airport trajectory forecasting. ? Amelia Toolkit: A comprehensive set of tools to download, process, analyze, and visualize aviation trajectory data! This project is a collaboration between the CMU BIG @ Carnegie Mellon University Robotics Institute and The AirLab at Carnegie Mellon University! ??Here’s to ensuring safer and more efficient skies! Congratulations to Ingrid Navarro* Pablo Ortega Kral* Jay Patrikar* Haichuan(Jack) Wang ZELIN YE Jong Hoon Park, EIT Jean Oh Sebastian Scherer! Read more:? ???Website: https://lnkd.in/djn8Tw7H? ??Paper: https://lnkd.in/dmPhfEha? ????Codebase: https://lnkd.in/dbTg3NCF
The AirLab at Carnegie Mellon University转发了
Excited to share MIA! How do we Map ?? It Anywhere? The trick is to flip the script from using principally limited self-collected datasets to using readily available worldwide maps. mapitanywhere.github.io Introducing the MapItAnywhere (MIA) data engine, an automatic way to curate map prediction data from two public worldwide scale data platforms. We use Mapillary to source FPV images and OpenStreetMap for BEV semantic maps. MIA can empower both generalizable training and benchmarking. We use MIA to curate from 6 cities easily –> 1M+ Image/Map Dataset with wide diversity. Using the dataset, we train a simple Mapper model with the dataset and show it can produce good map predictions out-of-the-box. Try it on your images today! We believe MIA complements existing datasets to provide an initial and scalable approach towards bringing map prediction anywhere. This is just the start! We are curating a bigger multi-modal dataset to go more anywhere ;) Excited to see how the community can build on this! Amazing collab with Jiaye Zou, Omar Alama, Sai Mitheran Jagadesh Kumar, Benjamin Chiang, Taneesh Gupta, Chen Wang, Nikhil Varma Keetha, katia Sycara and Sebastian Scherer! Head over to our website to generate your own data or see our FPV Mapper in action! Website: mapitanywhere.github.io? Data Engine Demo: https://lnkd.in/gUv-5Rve? Map Predictor Demo: https://lnkd.in/g5RTjwfY? ArXiv: https://lnkd.in/gDY_EPDr The AirLab at Carnegie Mellon University Carnegie Mellon University #ComputerVision #Geospatial
Please see below and apply if you are qualified:
We have an open software/robotics engineering position in the Air Lab. Please consider applying: https://lnkd.in/eqQqX47D Thank you!
CMU AirLab will be presenting 10 papers at the ICRA 2024! (Yokohama, Japan, May 13-17) 4 ICRA papers, 4 IEEE RA-L papers, 2 IEEE T-RO papers [1] “Aerial Interaction with Tactile Sensing” Xiaofeng Guo, Guanqi He, Mohammadreza Mousaei, Junyi Geng, Ph.D., Guanya Shi, Sebastian Scherer Paper: https://lnkd.in/g-sEGi4a [2] “TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks” Matthew Sivaprakasam, Parv Maheshwari, Mateo Guaman Castro, Samuel Triest, Micah Nye, Steven Willits, Andrew Saba, Wenshan Wang, Sebastian Scherer Paper: https://lnkd.in/g6P4xZGu [3] “Geometry-Informed Distance Candidate Selection for Adaptive Lightweight Omnidirectional Stereo Vision with Fisheye Images” Conner Pulling, Je Hon Tan, Yaoyu Hu, Sebastian Scherer Paper: https://lnkd.in/gZS-gNBH [4] “UNRealNet: Learning Uncertainty-Aware Navigation Features from High-Fidelity Scans of Real Environments” Samuel Triest, David D. Fan, Sebastian Scherer, Ali Agha [5] ?“MUI-TARE: Cooperative Multi-Agent Exploration with Unknown Initial Position” (RA-L) Jingtian Yan, Xingqiao Lin, Zhongqiang Ren, Shiqi Zhao, Jieqiong Yu, Chao Cao, Peng Yin, Ji Zhang, Sebastian Scherer Paper: https://lnkd.in/gPAFAhR5 [6] “Time-Optimal Path Planning in a Constant Wind for Uncrewed Aerial Vehicles Using Dubins Set Classification” (RA-L) Brady Moon, Sagar Sachdev, Junbin Yuan, Sebastian Scherer Paper: https://lnkd.in/g8Cf7RdJ [7] “Multi-Robot Multi-Room Exploration with Geometric Cue Extraction and Circular Decomposition” (RA-L) Seungchan Kim, Micah Corah, John Keller, Graeme Best, Sebastian Scherer Paper: https://lnkd.in/guUqYPKD [8] “AnyLoc: Towards Universal Visual Place Recognition” (RA-L) Nikhil Varma Keetha, Avneesh Mishra, Jay Karhade, Krishna Murthy Jatavallabhula, Sebastian Scherer, Madhava Krishna, Sourav Garg Paper: https://lnkd.in/gPUwFJCh [9] “AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale Environments” (T-RO) Max (鹏） Yin (殷）, Shiqi Zhao, Haowen Lai, Ruohai Ge, Ji Zhang, Howie Choset, Sebastian Scherer. Paper: https://lnkd.in/ghTzw8f7 [10] “BioSLAM: A Bio-Inspired Lifelong Memory System for General Place Recognition” (T-RO) Max (鹏） Yin (殷）, Abulikemu Abuduweili, Shiqi Zhao, Lingyun Xu, Changliu Liu, Sebastian Scherer. Paper: https://lnkd.in/gRkB2-ep