Lessons learned from stress testing autonomous vehicles and other robots

Lessons learned from stress testing autonomous vehicles and other robots

Philip Koopman Philip Koopman

Philip Koopman

Embedded System Software & Safety, Self-Driving Vehicles, Consulting.

发布日期: 2018年7月13日
+ 关注

The ASTAA team at CMU/NREC stress tested 11 autonomy projects and logged 150 bugs. Here are the lessons learned.

1) Protect your robots from data assumptions

  • Don’t trust that your configuration is valid
  • Time is not always monotonic
  • Violations can happen between semantically redundant fields

2) Floats and NaNs are useful but dangerous

  • Do not use floats as iterators
  • NaNs propagate

3) Plan for the system to fail

  • Nodes should not fail silent
  • Good logging is invaluable

While these findings might seem like common sense, we've seen problems in these areas over and over across many systems.

See the full paper and conference presentation here:


要查看或添加评论，请登录

Philip Koopman的更多文章

  • Job Selection Advice
    2021年5月8日

    Job Selection Advice

    A connection on LinkedIn asked me for help deciding between job offers. I can't provide personalize advice, but here…

    4 条评论
  • The Heavy Tail Safety Ceiling
    2018年6月21日

    The Heavy Tail Safety Ceiling

    Creating safe autonomous vehicles will require not only extensive training and testing against realistic operational…

    3 条评论
  • A More Comprehensive Look at Autonomous Vehicle Testing and Validation
    2018年3月2日

    A More Comprehensive Look at Autonomous Vehicle Testing and Validation

    Developers should create a transparent safety argument based on testing, simulation, and good engineering practices…

    13 条评论
  • Robustness Testing of Autonomy Software
    2018年2月16日

    Robustness Testing of Autonomy Software

    I'm very pleased that our research team will present a paper on Robustness Testing of Autonomy Software at the ICSE…

    1 条评论
  • Peer Review Checklist for Embedded Software
    2018年1月30日

    Peer Review Checklist for Embedded Software

    Here's a new peer review checklist to help improve the quality of your embedded C code. (Blog version of this posting;…

    1 条评论
  • Top Five Embedded Software Management Misconceptions
    2017年10月10日

    Top Five Embedded Software Management Misconceptions

    Here are five common management-level misconceptions I run into when I do design reviews of embedded systems. How many…

    6 条评论
  • The Spaghetti Factor: A Software Complexity Metric Proposal
    2017年8月28日

    The Spaghetti Factor: A Software Complexity Metric Proposal

    I've had to review code that has spaghetti-level complexity in control flow (too high cyclomatic complexity). And I've…

  • Optimize for V&V, not for writing code
    2017年5月8日

    Optimize for V&V, not for writing code

    I first heard this notion years ago at a workshop in which several folks from industry who build high assurance…

    5 条评论
  • SCAV 2017 Keynote: Challenges in Autonomous Vehicle Validation
    2017年4月21日

    SCAV 2017 Keynote: Challenges in Autonomous Vehicle Validation

    Philip Koopman, Carnegie Mellon University; Edge Case Research LLC Developers of autonomous systems face distinct…

    8 条评论
  • Autonomous Vehicle Safety: An Interdisciplinary Challenge
    2017年1月25日

    Autonomous Vehicle Safety: An Interdisciplinary Challenge

    IEEE Intelligent Transportation Systems Magazine ( Volume: 9, Issue: 1, Spring 2017, Pp. 90-96).

    3 条评论

社区洞察

其他会员也浏览了