The importance of timing when working with sensors and robotics

The purpose of this article is to emphasize the importance of timing when working with robotics.

Some of the key tasks for a robotic system is:

• Sensing (sensors)
• Processing (computer)
• Actions (motions e.g. move from A to B, brake, turn, pitch forward etc.)

But what about time?

Let us use the following example:

Unmanned aerial vehicles (UAV) interacting with moving objects/platforms/targets.

Before delving into the example, we must first think of what is important for robotics that are interacting with the world:

• Sensing
• Motion estimation
• Object localization

To achieve the above-mentioned bullet points with high precision and accuracy, accurate timing is necessary.

A common problem is that latencies from sensing to localization (of objects/platform/targets) and action is a problem. With knowledge of such latencies, the control system compensates for delayed measurements using motion predictions. If not, the control system will struggle making decisions to achieve its given task at the right time. E.g. moving to a position, applying automatic braking, initiating a turn or tilting the multirotor UAV to change position in a given direction.?

Splitting the challenges down to pieces

Let us now split up the latency from sensor measurement and all the way to the robotic system starts to perform an action. This is illustrated in the image below.

Latency accounting. Summarizing the latencies relative measurement time of validity (TOV)

+Latency before transport (TOT-TOV)

+Sensor data transport time over communication/data link (TOA-TOT)

+Latency at compute before processing start (TOP-TOA)

+Sensor data processing time (TOD-TOP)

+Time from decision to action (TOAc-TOD)

Total latency = TOAc-TOV

How do these latencies affect my system? It works fine, right?

That depends. The negative effect of unhandled timing errors and latency increases with:

+Relative dynamics of objects

+Timing errors

+Unmitigated latencies

= Increased action errors related to time

How can the SentiBoard (SB) by SentiSystems technology help with this?

• Sensing, timing and sensor processing must be tightly integrated to achieve full potential of your agile cyber-physical system
• Hardware-based timing is superior to software-based timing
• The SB times sensor measurements with: time of validity (TOV), time of arrival (TOA), TOV in UTC time if GNSS time is available
• With the SB and software utilities, time syncing protocols such as IEEE1588 can be used to accurately sync the PC time with SB time
• This enables SentiSystems to estimate sensor data latencies from measurement validity, via transport, to onboard processing and to action, providing this to the user
• Having access to latencies, vehicle (UAV, vessel, robot, car, vessel, etc.) motion states/signal and (track of) identified objects/platforms enables SentiSystems to do real-time localization and motion prediction of objects and planforms
• Relative timing errors among sensor matters also for slow/low-dynamic applications

Thank you for reading this article!