The AirLab at Carnegie Mellon University

We had a great time exploring the online mapping results of our learning-based stereo VO, MAC-VO, after uncertainty filtering—some exciting results! ?? We provide a dense mapping feature for MAC-VO in our GitHub repo. While the resulting map is stunningly consistent, the implementation is extremely simple - just project pixels with predicted depth uncertainty < 0.5cm to the 3D space! Check it out: mac-vo.github.io

?? The camera-ready version of MAC-VO is now live! Excited for #ICRA2025! By leveraging its data-driven uncertainty estimation module, MAC-VO achieves SOTA performance in pose estimation across diverse environments—including indoor, outdoor, and dynamic scenes. The most exciting part??MAC-VO is trained on the synthetic TartanAir dataset and requires **no fine-tuning** on different real-world data! Check it out: * website: mac-vo.github.io * Code: github.com/MAC-VO/MAC-VO

Excited to announce MapEx has been accepted to ICRA 2025! ?? ?????♀? How can a robot explore to build accurate maps without seeing everything? mapex-explorer.github.io We explore by jointly reasoning on prediction uncertainty and potential sensor coverage using multiple map predictions ?????? of unseen areas. ? ?? TL;DR - Predict multiple maps → estimate uncertainty & sensor coverage → explore to build accurate maps faster. Indoor environments have structure, predictability, and repeating patterns. We introduce MapEx, a new exploration framework to leverage such predictability by predicting maps to form probabilistic sensor models as information gain metrics for exploration planning. MapEx generates multiple predicted maps from which we estimate prediction uncertainty and potential sensor coverage. ??We plug in our new probabilistic information gain metric to a simple frontier-based exploration framework. The result? Better exploration than state-of-the-art map prediction-based exploration baselines on a real-world floorplan dataset simulator. ?? We believe MapEx complements many exciting research directions in robot exploration! We have open-sourced our code and simulator setup. Excited to see how the community can build on this. Sneak peek: we’re exploring learning from humans and RL :) Amazing collab with Seungchan Kim, Brady Moon, Aditya Parandekar, Narek Harutyunyan, Chen Wang, katia Sycara, Graeme Best, Sebastian Scherer ??Website: mapex-explorer.github.io ??Code: https://lnkd.in/ga_FrVJ7? ??ArXiv: https://lnkd.in/gi6rydcf The AirLab at Carnegie Mellon University CMU Robotics Institute Summer Scholars Program Carnegie Mellon University Robotics Institute Carnegie Mellon University #machineLearning #robotExploration

Exciting news from CMU AirLab! ?? We're glad to announce that five of our papers have been accepted at the International Conference on Robotics and Automation (ICRA) 2025! [1] MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictions Cherie Ho*, Seungchan Kim*, Brady Moon, Aditya Parandekar, Narek Harutyunyan, Chen Wang, Katia Sycara, Graeme Best, Sebastian Scherer Paper: https://lnkd.in/eGnWHa8c [2] MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometry Yuheng Qiu*, Yutian Chen*, Zihao Zhang, Wenshan Wang, Sebastian Scherer Paper: https://lnkd.in/eZEjgSCg [3] SuperLoc: The Key to Robust LiDAR-Inertial Localization Lies in Predicting Alignment Risks Shibo Zhao*, Honghao Zhu*, Yuanjun Gao, Beomsoo Kim, Yuheng Qiu, Aaron Johnson, Sebastian Scherer Paper: https://lnkd.in/ey9DPHiZ [4] SALON: Self-supervised Adaptive Learning for Off-road Navigation Matthew Sivaprakasam, Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Jieyi Lew, Isaiah Adu, Wenshan Wang, Sebastian Scherer Paper: https://lnkd.in/eJhhsPYq [5] BETTY Dataset: A Multi-modal Dataset for Full-stack Autonomy Micah Nye, Ayoub Raji, Andrew Saba, Eidan Erlich, Robert Exley, Aragya Goyal, Alexander Matros, Ritesh Misra, Matthew Sivaprakasam, Deva Ramanan, Sebastian Scherer We have two more RA-L papers that will be presented at ICRA. [1] I2D-Loc++: Camera Pose Tracking in LiDAR Maps with Multi-View Motion Flows Huai Yu, Kuangyi Chen, Wen Yang, Sebastian Scherer, Gui-Song Xia? Paper: https://lnkd.in/ejfrN4Xx [2] Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulation Xiaofeng Guo*, Guanqi He*, Jiahe Xu, Mohammadreza Mousaei, Junyi Geng, Ph.D., Sebastian Scherer, Guanya Shi Paper: https://lnkd.in/eWtjDydG Huge congratulations to authors and collaborators! Looking forward to sharing our work at ICRA. See you in Atlanta! #ICRA2025 #Robotics #Autonomy #Exploration #AirLab #CMU #LiDAR #SLAM

Happy New Year from CMU Airlab Aerial Manipulation Team! #CMU #AirLab #Robotics

Excited to introduce SALON! ?????? ?????? How can we achieve autonomous driving in off-road environments without hand-labels? Turns out that coupling visual foundation models with proprioception, and a few simple design choices, can lead to the same performance on a full scale all-terrain vehicle (ATV) as previous baselines trained on 100-1000x more data - https://lnkd.in/g44u54Sh. ?? Simply plug in a proprioceptive signal for self-supervision, initialize the system with as little as one manual label, and let it go! SALON quickly adapts to new experiences and learns what terrain types are most preferable, as well as how fast it should drive over them. ???? It’s so simple in fact, that beyond our own ATV autonomy results, we also demonstrate promising open-loop results on other ~very~ different robot configurations teleoperated in various environments. Huge thanks to my collaborators that helped make this possible: Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Lew, Isaiah Adu, Wenshan Wang, and Sebastian Scherer Website + Full Video: https://lnkd.in/g44u54Sh? ArXiv: https://lnkd.in/gJpgbg8q #AirLab #CMU #cmurobotics #robotics

Excited to present MapItAnywhere (MIA) at NeurIPS 2024! ??How do we Map It Anywhere? The trick is to flip the script from using principally limited self-collected datasets to using readily available worldwide maps. mapitanywhere.github.io? ??Insight: The data is already out there for map prediction! We just need to curate it! We introduce the MapItAnywhere (MIA) data engine, an automatic way to curate map prediction data from two public worldwide scale data platforms. We use Mapillary to source FPV images and OpenStreetMap for BEV semantic maps. ?? MIA can empower both generalizable training and benchmarking. We use MIA to easily curate from 6 cities –> 1M+ Image/Map Dataset with wide diversity. Using the dataset, we train a simple Mapper model with the dataset and show it can produce good map predictions out-of-the-box. Try it on your images today! ?? We believe MIA complements existing datasets to provide an initial and scalable approach towards bringing map prediction anywhere. This is just the start! We are curating a bigger multi-modal dataset to go more anywhere ;) Excited to see how the community can build on this! Amazing collab with Jiaye Zou, Omar Alama, Sai Mitheran Jagadesh Kumar, Cheng-Yu Chiang, Taneesh Gupta, Chen Wang, Nikhil Varma Keetha, katia Sycara and Sebastian Scherer! Head over to our website to generate your own data or see our FPV Mapper in action! Website: mapitanywhere.github.io? Data Engine Demo: https://lnkd.in/gUv-5Rve? Mapper Demo: https://lnkd.in/g5RTjwfY? ArXiv: https://lnkd.in/gDY_EPDr? #MachineLearning #ComputerVision #Geospatial #OpenData #robotics #MapItAnywhere The AirLab at Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University Robotics Institute

Humans can learn to reason in an "unfamiliar" world, like new games. How far are LLMs from this? Check out our recent work @NeurIPS2024 D&B Track: "LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulation". Page: https://lnkd.in/gyRWgtC7

How can we drive fast ????? in new offroad environments ?? where we don't have data? Check out Eric Chen and Cherie Ho's work on ALTER at IROS! ??

?? Ever wondered how to film a group of people with pixel-level precision, even as they split, join, and rearrange? Check out our work on “Greedy Perspectives: Multi-Drone View Planning for Collaborative Perception in Cluttered Environments.” presenting at IROS 2024. ?? Website:?https://lnkd.in/eSR-y7gt ?? arXiv:?https://lnkd.in/eWHmYkrs ?? We directly optimize views and drone motions over the planning horizon using greedy algorithms for submodular maximization. For pixel-level reasoning, we render views with OpenGL, skipping distance-based approximations and instead using a proxy for the reconstruction metric. ?? Greedy Perspectives distributes pixels across multiple actors in cluttered scenes, accounting for obstacles and occlusions, and outperforms traditional formation planning. Authors: Krishna Suresh, Aditya Rauniyar, Micah Corah, and Sebastian Scherer #AirLab #CMU #CSMines #robotics #drones