Carnegie Mellon University Robotics Institute

Low-cost teleoperation systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Meanwhile, many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. ?? A low-cost, ?force-feedback-enabled teleoperation system.? 2. ?? Force-Attending Curriculum Training (FACTR) uses force to improve generalization in complex, contact-rich tasks. ??Our teleop system (~$1K per leader arm) lets users intuitively feel forces experienced by robots, without additional sensor hardware. This can be done for many arms. We also integrate gravity compensation, redundancy resolution, etc. in the teleop leader arm, augmenting the teleop experience. ??Behavior cloning policies often overfit to vision input and ignore force when added naively, limiting performance in contact-rich tasks. We propose FACTR, a curriculum that prevents overfitting to vision and guides the policy to properly attend to force. FACTR policies generalize to unseen objects significantly better than baseline force policies without FACTR. FACTR is easy to set up on existing systems. Both the hardware and code of FACTR will be open-sourced soon! This work is my first PhD project at Carnegie Mellon University with co-lead Yulong Li as well as Kenny Shaw, Tony Tao, Russ Salakhutdinov, and Deepak Pathak. Webpage: https://lnkd.in/dSTyavk4 Paper: https://lnkd.in/d4iYjgmp

Chief Technology Officer at the Robotics and AI institute (RAI) Alfred Rizzi visited the Robotics Institute to give a fascinating seminar, titled “Developing Physically Capable and Intelligent Robots.” He also visited some RI labs, where students had the opportunity to demonstrate their current work. Huge thanks to Al for visiting and for such an informative talk! The recording of his seminar, as well as all other seminars, can be found on the CMU RI YouTube channel. We encourage you to check out the amazing speakers we have had the privilege of hosting this academic year and in the past! https://lnkd.in/ewmQqRwQ

Great news! ?? AirLab’s Tartan Air Rescue was named as a Stage 1 Winner and a US University Innovation Award Winner of the GoAERO Prize! The AirLab at Carnegie Mellon University develops state-of-the-art methods for drone autonomy, and the Tartan Air Rescue aircraft is designed to make a life-saving impact when it matters most. Congratulations to the team! Learn more about Tartan Air Rescue and the GoAERO competition: https://lnkd.in/gGz7sFQk Watch the GoAERO video celebrating the stage 1 winners: https://lnkd.in/gmrRgs36

Congratulations to RI Ph.D. students Angela H. Chen and Bardienus Duisterhof, who both received the Generative AI in Healthcare Fellowship from the CMU Center for Machine Learning and Health! Chen and Duisterhof utilize artificial intelligence in unique ways to expand their research at the intersection of robotics and healthcare. You can read about their individual work and how the fellowship will support them below! https://lnkd.in/dqBrgemu

Pittsburgh's role as the "Robotics Cradle" is fueled by groundbreaking research, collaboration, and a community dedicated to advancing robotics and AI ?? Discover how our city became a global robotics hub and what’s driving innovation today in Jon Swartz's Digital CxO article: https://buff.ly/3EfT927 Pennsylvania Department of Community & Economic Development | Michael Gerber | Techstrong Group | Carnegie Mellon University | Carnegie Mellon University Robotics Institute | Innovation Works | New Economy Collaborative | Robotics Factory | Allegheny Conference on Community Development | Pittsburgh Region. Next is Now. | Southwestern Pennsylvania Commission | #Robotics #Innovation #Automation #TechTransformation #WorkforceOfTheFuture #RoboInnovation #PittsburghRobotics #RoboticsInnovation #PittsburghTech #RoboticsLeadership #PGHRobotics #Roboburgh #PGHTech #PittsburghRoboticsNetwork #RoboticsPGH #PittsburghTechies #DeepTech #DeepTechPGH #RoboPGH #PittsburghRoboticsHub

?? Can robotics redefine surgical precision? Over the past 1.5 years, my?Masters in Robotic Systems Development (MRSD) Capstone team?modified a?KUKA robotic arm?with an?Intel Corporation Realsense camera?and?Dremel tool?to assist surgeons in?Total Knee Arthroplasties (TKAs).? Our goal: improve accuracy, reduce procedure time, and enhance outcomes for patients and physicians. I am proud to have worked with?my Team ParaDocs?(Shivangi Gupta, Li-Wei Yang, Abhishek Warrier, Qilin Wu) and industry mentors from?Smith+Nephew?(Branislav Jaramaz, Kaushik Balasundar, Sundaram Seivur, Parker Hill) to bridge robotics and clinical needs. ?? Explore our work:?https://lnkd.in/gda8F-vM ???We will make our codebase public soon! If you’re curious about?surgical robotics,?healthcare innovation, or want to brainstorm, I’d love to connect! Carnegie Mellon University Carnegie Mellon University Robotics Institute Carnegie Mellon University School of Computer Science #Robotics #SurgicalInnovation #MedTech #Healthcare #CMU

Today, we are honoring Warren “Chuck” Whittaker and his incredible contributions to the Robotics Institute as he steps into retirement. Chuck’s journey with Carnegie Mellon University began in 1985 when he joined a team developing robots to inspect the contaminated basement of the Three Mile Island nuclear plant near Harrisburg, Pennsylvania. He later became a lead engineer at Redzone Robotics, a company founded by his brother, Red Whittaker. In 2004, Chuck returned to CMU as the senior field robotics engineer at the CMU Field Robotics Center (FRC), where his work spanned from autonomous vehicles to developing robots for mining and outer space exploration. Below, he is pictured giving his final FRC lecture. Chuck also made a lasting impact by mentoring high school students through the Girls of Steel robotics team, led by George Kantor. The Robotics Institute has been profoundly shaped by Chuck’s dedication, enthusiasm, and his expertise in field robotics. His commitment to supporting students, researchers, and fellow staff has left an enduring legacy. Please join us in celebrating Chuck Whittaker’s remarkable career and wishing him all the best as he embarks on this exciting new chapter of retirement!

Excited to present our new work on using diffusion priors for video amodal segmentation and content completion! with Deva Ramanan and Tarasha Khurana. arXiv: https://lnkd.in/e5UDFW8s project page: https://lnkd.in/eMGAGxhi

Excited to be at NeurIPS'24, where I'll be presenting at several workshops! Looking forward to chatting about (time series & tabular) foundation models, data science agents, or ML for healthcare! Also, I'm also on the industry job market, looking forward to connect ??!