A Comprehensive Guide to Axis Network Camera Edge Application SDKs: Choosing the Right SDK for Your Application.

This article provides a comprehensive overview of the different software development kits (SDKs) available for edge application on Axis network cameras. The author of this article has over 7 years of professional experience in developing Axis camera applications. This expertise provides valuable insights into the SDK options available for Axis cameras and how to navigate the complexities of choosing the right SDK for a specific application. The article covers topics such as deployment, programming language, web pages and parameter handling. The article also includes background information on the evolution of Axis SDKs, from the early days of SDK 1 to the current options of SDK 3 and 4.

After reading this article you will understand the differences of:

• Deployment as?an .eap?file or?a Docker?image
• Using?the C/C++?or?Python?programming language
• Building your own parameter handler or using the built-in parameter handler
• Creating web pages or APIs with axhttp,?fastCGI?or a reverse proxy

Background

Axis cameras are known for their high-quality hardware, advanced features such as edge analytics and audio capabilities, and their compatibility with third-party software and systems. When the built in functionality is not enough the Axis Camera Application Platform (ACAP) allows the creation of custom edge applications that can be installed and run in the cameras. These applications can be installed individually in each camera or in thousands of cameras at the same time using the Axis device managing tools or a third party Video Management Solution (VMS).

Developing Axis ACAP applications over the past few years has been an exciting experience, especially with the continuous development of the SDKs and APIs. However, this also leads to some confusion and complexity when deciding which SDK to use. Once you have a clear understanding of your needs and knows the differences between the newer and older SDKs, selecting the right choice for you will be easier.

The new SDKs are they easier to install and use in continuous integration pipelines, but they also offer more functionality and with a better API than earlier versions. The newer SDKs will however put strict limitations on which camera models the applications can be deployed to and there is a limitation on the compatibility with tools used to install and manage the applications. It is thus very important that you understand the properties of the different SDKs so you can select the right one to use already from the beginning.

The old days

Going back more than a decade (to 2009) ACAP SDK 1 was released, a game changer for Axis cameras and the network camera industry. Before then there was no (supported) way of building edge applications for Axis cameras. Although fragile hacks could be used to make custom code run in the cameras, it was not possible in a consistent and robust way, therefore preventing most of the real use-cases. The launch of the ACAP SDK also made it easy for end customers or integrators to install the applications into the cameras themselves.

The usage of ACAP SDK 1 continued for many years, until firmware 5.70 was released. At that point, a few significant changes in the architecture of the firmware and the camera runtime was made. As an example, the?Boa?web server was replaced with the?Apache?web server and the?telnet?remote access was replaced by?SSH. These changes resulted in a strict breaking point where the application needed to be built with the correct choice of SDK depending on if it was to run on firmware before 5.70 or newer. This way it continued for many years: SDK 1 was used for low-end cameras and SDK 2 for the new or high-end cameras. After the initial release of SDK 2, only a few minor releases was done during all these years.?

The new days

A few years back we could see the release of ACAP SDK 3. This marked another era of the ACAP ecosystem where the view on the SDKs changed from being something static that could be used for years to being a software component that should be kept up to date. While SDK 1 and SDK 2 had years between minor released SDK 3 was frequently updated with new features, bug fixes and security improvements. SDK 3 was also quickly followed up with two new major releases, Native SDK 4 and Computer Vision SDK 4.

With a frequently evolving SDK the power of the platform was quickly increasing, but at the cost of an exploding complexity. Several active SDKs was suddenly available which targets different use case and with frequent additions of new functionality in the minor releases the firmware compatibility was frequently broken. This means that you can no longer rely on using the latest minor of the SDK but you do instead need to understand a complex compatibility matrix and make decisions based on your business requirements.

The ACAP SDK 3 packaging

With official support from firmware 9.70, SDK 3 was under the hood very similar to SDK 2 but with an improved packaging. The ACAP runtime in the cameras and the build environment was mostly the same as with SDK 2, this means that the breaking point in firmware support between SDK 2 and SDK 3 was not as strict as between SDK 1 and SDK 2. Building applications with SDK 3 and running in cameras equipped with firmware much older than the minimum supported 9.70 release does usually not pose any significant challenges. Care does however need to be taken to ensure compatibility with libc and dynamic libraries in the camera.?

The most obvious difference from SDK 2, which was distributed as a?.tar?or?.deb?package (expected to be installed on the developers computer), is that SDK 3 is distributed as a?Docker?image. This allows easy creation of dockerized build scripts which will be reproducible on every developers computer.

Other changes and additions worth of noting are that the configuration file?package.conf?file was replaced with a?manifest.json?file which is much nicer to work with, the compiler version being bumped from gcc 4 to gcc 8 and the introduction of a new build tool (in SDK 3.3). The new compilers introduced sought after functionality such as the?undefined behavior sanitizer?and?address sanitizer?which made debugging and quality assurance easier.

The new configuration files, the builder script and mostly the compilers are backwards compatible. This resulted in very easy transitions from SDK 2 to SDK 3.

SDK 3 is as of today still supported but will not receive any new features according to Axis. A significant ratio of ACAP developer companies are still forced to use SDK 3 as the default SDK due to the limited compatibility of the newer SDKs. Incorrect modification of the?embeddedSdkVersion?tag in the?manifest.json?file can be set to allow installation on (more or less) arbitrarily old firmwares, this is however effectively disabling the compatibility check for the camera and can often cause errors from the dynamic linker or even unexpected segfaults due to the lack of forward compatibility in libraries and libc.

The two versions of ACAP SDK 4

Applications for the SDKs 1-3 was traditionally implemented using the C-programming language. While the compilers also supports C++ only a few advanced applications was developed using this since all SDK libraries are written for C code.

In the end of 2021, there was two new SDKs released. Both of the new SDKs are named ACAP SDK 4 but they come in two different variants that are very different (and in some cases can be complementary to each other). The two variants are the?Native?SDK and the?Computer Vision?SDK.?With this release, another supported programming language, Python, was added.

ACAP?Native?SDK 4

Native?SDK 4 share a lot in common with SDK 3. The build environment and the ACAP runtime both are mostly the same. Native SDK 4 which have a more limited range of supported cameras, is likely to see some improvements regarding APIs and security. The current trend of modernizing the APIs and tools (as was seen with the introduction of the json based manifest file instead of the earlier key-value file) was continued by removing some of the older feeling APIs that was still available in SDK 3. There are three significant APIs that was removed that we have seen affecting many current ACAP developers:

1. Removal of the?axhttp?library: In SDK 2 and SDK 3 the preferred way to create dynamic web interfaces for an ACAP was to use either?transferCGI?or?fastCGI?configured so that certain requests sent to the camera's?Apache?web server was forwarded to the application binaries. This inter process communication was enabled by the Axis specific?axhttp?library which does not exist in?Native?SDK 4. The?new way of serving webpages?that was introduced, which is inspired from the microservice architecture, was to run a stand-alone web server within the application. This web server was run on another port than the cameras?Apache?server, bound to the loopback address, and with the?Apache?server configured as a reverse proxy for certain URL paths. The cameras?Apache?web server was also used to terminate the?encrypted SSL?connections (using the certificates installed through the cameras configuration tools) and Apache was used for user authentication based on the camera’s configured web users. While this could be seen as slightly more complicated, the benefits are many such as better separation of concerns leading to a more robust application and increased possibilities to do advanced configurations of the web server. Recently a third option was announced, yet again going back to the usage of fastCGI to make the application talk directly to the Apache process. This time, the application was recommended to use open-source libraries for the communication and no successor to axhttp was this announced. The choice between the two libraries are mostly down to backward compatibility versus complexity and potentially future compatibility.
2. Removal of the capture library: Traditionally image data and video data were acquired by the applications by the usage of?the libcapture?library. A while back the newer?vdostream?library was introduced which offers better control of what image formats are available in the specific platform. Native image formats differ between the different architectures in the different Axis cameras and the earlier library did not allow for introspection of the format, frequently leading to frustrated developers spending weeks to get a frame using the zero-copy API. The?vdostream?library is flexible (supports raw and encoded images in multiple formats), portable (compatible with multiple backends) and allows for zero-copy data access when performance is crucial. In?Native?SDK 4 this is the only available capture library (see also the?OpenCV?API in?Computer Vision?SDK 4 which can also be used in combination with?Native?SDK 4 as a video capture library).
3. Removal of the?axparameter?library: This change could have the potential to make the biggest dent in the design of ACAP applications. While the other removed libraries come with a new library to perform essentially the same functionality, this removal does in-fact remove the feature that it once enabled. The?axparameter?library was used as a parameter handler in the application, storing the parameters with this handler automatically enabled a configuration user interface. This user interface could be used by simple applications to directly in the application page in the camera’s web-based user interface configure the parameters and behavior of the application. Furthermore, it enabled the configuration of the parameters using the?VAPIX?web API which made bulk configuration of camera pools easy. With the?axparameter?library now removed it is up to each application developer to handle not only its own parameters, but also the configuration UI for changing the parameters. This is why we at fixedit.ai has created a reusable parameter handler and configuration template which can easily be adopted for different clients needs. This can be seen in the video demonstration below.

ACAP Computer Vision SDK 4

All the SDKs described so far (1, 2, 3 and?Native?SDK 4) are very similar in that they build the application into a package file with the?.eap?file extension. This package can then be installed in a camera using the cameras web interface. This allows for deployments where the end customer or a non-programmer integrator installs the application themselves in their cameras. Bulk installation of applications in multiple cameras are also supported by tools such as Axis Device Manager (ADM) and installation on remote cameras can be done with the Axis Companion software or the Axis Device Manager Extended (ADME).

What happened to the UniMatrix project? Is there a unified standard for running deep learning models in network cameras?

The?Computer Vision?SDK 4 uses a very different runtime with container-based packaging. This approach is even more inspired by the modern microservice architecture, and the principles often used in DevOps. The architecture of many components in ACAP?Computer Vision?SDK 4 and the container runtime in the cameras was driven by an?open-source project named?UniMatrix?which was a cooperation between multiple camera vendors. While this cooperation was a refreshing idea it does not seem to be as productive as initially envisioned. The cooperation aimed to create a unified environment for deep learning applications in the video surveillance industry, and at one point it looked like the project would become adopted by the Linux Foundation. The standardization of technologies so complex as hardware accelerated processing of deep neural networks with a unified distribution format did however not result in any broadly adopted standard. Today the activity in the Unimatrix project seems very low and different platforms even within the Axis ecosystem continues to be very segmented with vastly different requirements on the training frameworks and the model compilation process.

Together with the release of Computer Vision SDK 4 the support for containers in high-end cameras was announced. With the containerd daemon running in some selected cameras and the docker daemon available as an installable application it was now possible to run Docker containers in the cameras, just as we are used to on our developer computers or cloud environments.

There are a lot of benefits with containers such as easier development, easier software testing (due to isolation) and simple deployment of bundled libraries. The downsides are the low rate of support among camera models, the storage intense deployment units it produces that normally requires the cameras to be equipped with an SD-card and the lack of deployment, monitoring and management infrastructure. Currently only a few selected high-end cameras (as of writing, 19 models out of 214 in total) with lots of flash storage and firmware newer than 10.7 are capable of running the containers that the?Computer Vision?SDK 4 depends on.

The container runtime in the cameras does not come with any user interface where end-users or camera operators can install or manage applications as they could with the?.eap?packages produced by the other SDKs.

Applications built with the ACAP Computer Vision SDK 4 needs to be installed with an API based approach. A docker client needs to run on a host that has direct network access to the cameras and can then be used to download application images and start or stop the containers. This could be done in a few different ways:

1. A centralized system with direct access to the cameras can?control the run state of each camera, each camera can pull their images from the cloud centralized registry
2. An onsite controller can be responsible for pulling the image from the cloud and managing the run state of the containers in the cameras
3. An application built with?Native?SDK 4 can run in the camera and be responsible to?pull the container images?from a server and manage the run state of them, this would require direct communication with the docker daemon
4. A combined package where?Native?SDK 4 is used to build an?.eap?package file that contains?both a container image built with?Computer Vision?SDK 4 and a script that loads and manages the state of the containers, this would require direct communication with the docker daemon

Despite the deployment challenges, the new Computer Vision SDK has many benefits when it comes to prototyping and testing out ideas for new applications. Using containers to run applications in the cameras the filesystem as experienced by the application and the host filesystem is isolated from each other. This allows for easier deployment of software components aimed for other platforms, it also removes most challenges with conflicting library versions commonly known as "dependency hell". Combining this separation with the increase in both flash and RAM in the newer Axis cameras it is now easy to build applications in the?Python?language that will run in the cameras. The?Computer Vision?SDK 4 bundles libraries that are usable for computer vision applications for both?C++?and?Python.

Some of these libraries are:

1. OpenCV?video capture library: Everyone who are working with computer vision is probably familiar with?OpenCV?and their video capture API. This is now available as an option to do video and image capture in the Axis cameras. The special version of the library comes pre-compiled with the?Computer Vision?SDK 4 for both?C++?and?Python. Under the hood you can see that the?OpenCV?capture library relies on the?vdostream?library to get images from the Axis cameras.
2. TensorFlow Serving: Organizations that has deployed machine learning models in large scale can tell you that model deployment is not as simple as wrapping the model with a web server and you’re done. There are a lot of MLOps related activates around the deployment such as model monitoring, gradual deployment strategies and data serialization. To be able to focus on the data and the training of the model, most companies therefore make use of a model serving library or server. One such model server is?TensorFlow Serving?which is available for the ACAP Computer Vision?SDK 4 as a?Docker?container enabling the use the microservice architectures for model serving. The SDK also comes with a?Python?library for the?TensorFlow Serving?client. The Axis specific version of the?TensorFlow Serving?image (which is now also open source) makes use of?liblarod?(an open-source library under the UniMatrix project) which is also the library available in SDK 3 and?Native?SDK 4 used to run deep learning model inference on the neural network acceleration hardware (Deep Learning Processing Unit, DLPU) found in many Axis cameras.
3. NumPy?and?SciPy:?Computer Vision?SDK 4 also comes with a lot of other helpful libraries for?Python?such as the?NumPy?and?SciPy?libraries that most data scientists have made use of to create models and data processing steps.

# cv_edge_camera_app.py

import cv2

cap = cv2.VideoCapture(1)  # Select the camera interface
cap.set(cv2.CAP_PROP_FPS, 25)  # Select the frame rate
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 480)  # Select width and height
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 320)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"RGB3")) # Set format

while True:
    _, frame = cap.read()
    do_analytics(frame)        

Read more about Python applications and the OpenCV library in this article.

The container runtime in the cameras is built on the open-source?containerd?daemon. Many developers are much more familiar with the?Dockerd?daemon, therefore a separate ACAP application can be installed in the cameras that run the?Dockerd?daemon on-top of the?containerd?daemon. To create easily deployable microservice architectures also the?Docker Compose?tool can be installed as a separate ACAP.

Since the?ACAP Computer Vision?SDK 4 make use of a container based runtime, it allows you to implement the applications for any environment and any programming language you are comfortable with. Due to this, there are no cross-compilers deployed together with the SDK, but you can use the cross compilers available in the distribution your application builds in, e.g. from the ubuntu repository or you can combine the build environment with the Native SDK 4 and use the compilers from that.

Which SDK should you chose?

tl;dr If you skipped directly to this section, contact us at FixedIT Consulting and we will help you make the right choices with minimal effort. There is simply not a single recommendation short enough for a conclusion, or even an article, that works in all situations.

There are currently three supported ACAP SDKs to choose from, each with multiple minor releases. ACAP SDK 3 has been replaced by?Native?SDK 4 and does not receive any new features; this means that when designing a new ACAP application you should use this only if you need to which due to compatibility requirements is fairly frequent. One such requirement might be to support image analytics using the low-end Axis cameras. Doing so, you should be aware that this must be considered a legacy feature by now and compatibility might disappear with the releases of new camera firmware.

The?ACAP Computer Vision?SDK 4 is a powerful option to create portable applications with components reusable from cloud or desktop environments. The simplicity of using?Python?and a range of popular open-source libraries makes this a superb environment to experiment and develop in. The cost is however deployment complexities and a reduced camera compatibility where the older or cheaper cameras cannot be used. This excludes the SDK from most production use-cases.

Further considerations are:

• The support between firmware and SDK version is in many cases highly overlapping between the SDKs, allowing for a wide range of possible choices.
• UniMatrix is a relatively new open-source project but already seems to be stale.
• Both SDK 4 variants can be used together to make use of the benefits of the microservice architecture but the application deployment and management capabilities of the?Native?SDK 4. This is however uncharted territory.
• SDK 3 can be used in a way that it is fully compatible with?Native?SDK 4, but an application designed only for SDK 3 might be very hard to migrate for?Native?SDK 4.
• A few new features are available in?Native?SDK 4, compared to SDK 3, like the preuninstall script (which is however defacto usable in SDK 3 too). This is only usable if the camera firmware is new enough to support it.
• Applications built with?Computer Vision?SDK 4 is only supported on a few high-end cameras but with the release of ARTPEC-8 and CV25 the price point was reduced significantly.
• Computer Vision SDK 4 allows for a true microservice architecture and relative ease of porting cloud or desktop applications to the edge devices.
• Applications deployed as?Docker?images using the?Computer Vision?SDK 4 might require the camera to have an SD-card installed due to the large size of the deployment unit.

Hopefully this article has shed some light on the landscape of Axis ACAP SDKs and helped you understand how important it is to select the right SDK for your use case. The impact on the application architecture cannot be neglected and the options should therefore be explored in an early phase to allow efficient use of the Axis cameras. We have helped a lot of companies design the architecture of their computer vision systems which relies on Axis cameras, and you should reach out to us if you want to use your resources efficiently.

Find more information on?fixedit.ai or contact us directly.

This article is the second revision of an article by Daniel Falk found here, which in turn is a refactored transcription of his screen cast "Compuer Vision at the edge: Which AXIS ACAP SDK should you use?":