The Significance of the MIPI CSI-2 Interface in Embedded Vision Applications

An increasing number of embedded vision applications have appeared on the market in recent years. These are systems based on an embedded computing board and a camera module. Such a system can manage vision tasks more affordably and efficiently.

The interface, which is the connection between the camera module and host, plays a key role in the setup of such an embedded vision system. The plug-and-play USB 3.0 interface can be used here, for example. LVDS interfaces (Low Voltage Differential Signaling) are also worth considering, especially for FPGA-based systems. But in very many cases, the MIPI CSI-2 interface is the appropriate choice. This White Paper provides an overview of the significance and features of this important interface for the embedded vision field.

1. What is MIPI?

The MIPI Alliance – or Mobile Industry Processor Interface Alliance – is an association of nearly all manufacturers of applications or hardware in the mobile communication and entertainment electronics industry. Its goal is to standardize all important interfaces between the mobile processor and peripherals (position sensors, cameras, input interfaces, displays, etc.). This makes it easier for manufacturers of mobile peripherals to adjust their hardware to various types of processors. It creates a larger range of potentially compatible peripherals for processor manufacturers. This way both sides can profit with more economical development and production processes.

Well-known MIPI standards include the DSI (Display Serial Interface) specification and the CSI (Camera Serial Interface), which we will look at in more detail.

2. What defines MIPI CSI-2? What doesn’t?

The CSI specification has already existed for several years. By now the third generation has appeared, namely CSI-3. However, the lack of hardware support has prevented CSI-3 from catching on in the industry, although it has indisputable advantages compared to the predecessors’standards (e.g. a much higher maximum bandwidth). The fact that the most widely used CSI-2 standard at this time is completely sufficient for current hardware requirements may also play a role here.

The MIPI CSI-2 specification describes the physical layer of the signal transfer (D-PHY or C-PHY) as well as the CSI-2 protocol for image data transfers which are based on it. This standard also specifies an interface for a camera configuration via I2C, namely CCI (Camera Control Interface).

2.1 The physical layer – C-PHY vs. D-PHY

With MIPI CSI-2, the image data are transferred serially through individual lanes. It is customary to connect image sensors or camera modules with two or four lanes. The maximum possible bandwidth is linearly scaled with the number of lanes, i.e. four lanes offer twice as much possible bandwidth as two lanes. This standard originated in the world of smartphones, where a high-resolution camera on the back of the cell phone was connected with four lanes but the lower-resolution camera on the front with only two lanes. As a result, nearly all corresponding on-chip processors already have one CSI-2 interface with two and one with four lanes.

Physical image data transfer via C-PHY:

In C-PHY, a lane consists of three conductors, realizing a 3-phase symbol coding with an embedded clock. According to the latest specification, this can theoretically result in 5.7 GBit/s per lane.

Physical image data transfer via D-PHY:

D-PHY has a simpler structure compared to C-PHY – similarly to LVDS, the data streams are transferred deferentially via two conductors. A shared external clock line is then available for all lanes. As a result, this simpler setup only allows lower data rates of up to 2.5 GBit/s/lane. Despite the lower bandwidth (which is still completely adequate for most applications), D-PHY in particular has caught on as a standard layer.

But D-PHY and C-PHY can also coexist in the respective hardware, and individual sensors can, for example, be operated with one or the other technology.

Camera Control Interface (CCI):

CSI-2 also offers a standard protocol for the configuration of the sensor/camera module in the form of CCI (Camera Control Interface). CCI is physically based on I2C and in general, the camera module could be configured through an arbitrary I2C interface, which is present in multiple instances on virtually all SoCs. However, there is a lack of uniformity in the area of sensors/camera modules as well as the SoCs themselves: Some SoCs have a dedicated I2C/CCI interface that must be used for the configuration; at the same time, many sensors/camera modules are not configured via the standardized CCI but through manufacturer-specific, proprietary implementations.

2.2 The CSI-2 protocol layer

CSI-2 is a packet-oriented protocol. For that reason, the specification describes the packet data format in particular.

The possible pixel formats (e.g. RGB, YUV, RAW and JPEG etc.) that can normally be used by an application are also specified. However, the MIPI CSI-2 drivers of a mobile processor generally only support a few pixel formats (which are sometimes pixel formats that tend to be unconventional in the market of industrial image processing).

2.3 MIPI CSI-2 compared to industry standards (GenICam etc.)

The MIPI CSI-2 is more of a description of the “standard on the wire” and in this respect is comparable with the GigEVision specification (image data transfer through Gigabit Ethernet). Unlike the standards from the GenICam family commonly used in the image processing market, MIPI CSI-2 lacks a standardized software stack, a standardized programming interface (Application Programming Interface, API) – as represented by GenAPI, for example – and a standardized image data interface (such as GenTL). With MIPI CSI-2, the image data are usually delivered through Video4Linux (V4L), but this is also neither standardized nor universally the case.

3. What are the advantages?

The mobile market is huge, mobile processors are produced in extremely high quantities and the market size as well as competitive pressure are resulting in increasingly efficient processors in short innovation cycles. Even very inexpensive, low-end processors now generally have two MIPI CSI-2 interfaces on a chip with two or four lanes.

Many manufacturers who once only produced mobile processors (e.g. Qualcomm, Rockchip, Samsung) have discovered an interesting market in the image processing industry and now also offer the products – sometimes through module partners – in smaller quantities and with long-term availability for industrial applications. At the same time, the more typical industrial embedded processors (e.g. the i.MX family from NXP, some Sitarra SoCs from TI, NVIDIA, Intel Atom SoCs etc.) are also equipped with MIPI CSI-2 interfaces by now, so that this new camera interface is virtually unavoidable for the image processing market. CSI-2 enables extremely slim and inexpensive machine vision designs, which make it possible to use sensors or camera modules with a very high bandwidth. But the fact that processors are affordable and getting still more affordable is not the only reason why the unit costs are decreasing. As a result of the multitude of processors, the developers can freely choose the design of the optimal system depending on the application. This can also lead to reduced energy consumption by the embedded system, for example.

MIPI CSI-2 technologically enables very small design sizes, which can be implemented with a flat flex cable through a board-to-board connection. Otherwise this is only possible via LVDS-based connections and certainly not through a connection via USB 3.0, whose plugs make them take up significantly more space. The available assembly space is one of the most relevant factors for many products based on embedded systems, so USB 3.0 may not be considered here at all.

With the MIPI CSI-2 interface, the image data can be transferred directly from the camera module or sensor to the processor. This eliminates the need for interim processing or a data conversion as is the case with USB 3.0 i.e. the data doesn’t “take a detour” that affects the processor’s capacities. The corresponding hardware (micro controller etc.) is also omitted. These conditions make it possible to implement much slimmer embedded systems.

4. Which obstacles might arise?

As seductive as the offer of an efficient MIPI CSI-2 camera interface is, being available virtually without additional costs, there are some obstacles that should be considered during the development:

• Cable length: Unfortunately, the physical image data transfer (D-PHY) only permits shorter cable lengths, usually not more than 20 cm. For mobile applications like smartphones, this is not an issue, but for many industrial applications this might be a non-option.
• Plugs: Unfortunately, the MIPI Alliance does not standardize a plug for MIPI CSI-2. This means that a sensor/camera module must always be connected individually and in a proprietary way.
• Driver support: The lack of a standardized driver and software stack requires the individual adjustment of a sensor or camera module to the CSI-2 driver of a given SoC via a proprietary I2C driver as a Video4Linux sub-device. In other words: When you select a sensor/camera module designed for an SoC, make sure that this module also has a corresponding I2C driver. This significantly limits the choice of sensors/modules and SoCs.
• Pixel formats: As described above, most CSI-2 drivers only support very few pixel formats, which aren’t necessarily those required on the image processing market.
• Camera API: The standards of the GenICam family have caught on in the image processing market. Thanks to GenICam, the camera API is standardized and full access to all camera features is ensured. The designations for the camera features are also standardized independently of the manufacturers, which allows the user to change the camera model and even the interface technology (e.g. USB3 Vision instead of GigEVision) without having to make significant changes in the program code. In contrast, however, the lack of standardized feature designations and a standardized API makes it more difficult to reuse existing code when migrating from one camera module to another or one SoC to another.

All of these obstacles can be overcome with the appropriate expenditure, but this can drive up the development and manufacturing costs (e.g. to implement longer cables).

Individual industrial camera manufacturers are working on the development of a GenICam interface for MIPI CSI-2-based camera solutions as well, which would provide access to all camera features and pixel formats and at the same time permit much longer cables (over one meter).

5. Summary

The MIPI CSI-2 interface is suitable for embedded vision systems in which a lean and cost-optimized structure is desired. But the integration of a corresponding camera module takes effort, particularly on the software side. Drivers and image data interfaces may need to be adjusted to the applications. Other restrictions, for example those posed by hardware, may also need to be considered.

To make sure that system developers can reliably estimate their workload, they must therefore consider the scope and compatibility of the drivers as well as potentially existing camera software and its SW interfaces. If the right options are available here, an efficient vision system can be developed, resulting in a well-performing end product.