The power of vector databases on mobile & restricted devices

To unlock the potential of Edge AI, which is no less than the promise of AI anywhere anytime, developers need vector databases capable of running directly on handheld and embedded devices.

This isn’t just about keeping data closer to home; it’s about revolutionising how AI can operate independently of the cloud, right on the myriad of connected devices and smart things around us. On mobile phones, IoT gadgets, controllers, ECUs, medical equipment, industrial machines, you name it!

Enabling Advanced AI Anywhere, Anytime?

On-device vector databases bridge the gap between powerful AI capabilities and everyday devices. Recent AI language models (LLMs) already demonstrate impressive capabilities while being small enough to run on e.g. mobile phones. For example: Gemma 2b from Google, Phi-3 from Microsoft and OpenELM from apple. The next logical step is to use these LLMs for advanced AI applications that go beyond a mere chat on small devices. And for this, developers need an on-device vector database.

Advanced AI apps use data (e.g. user specific data, or company specific data) and multiple queries to enhance and personalise? the quality of the model’s response and perform complex tasks. And now, for the very first time, with the release of Objectbox 4.0, this will be possible locally on restricted devices.

This brings the added benefit of working offline as well as online and the data never needing to leave the device.

This pioneering technology allows AI language models to interact seamlessly with user-specific data —like text and images— directly on the device, without relying on an internet connection. It's the perfect solution for developers looking to create smarter apps that are not only efficient but also reliably functional in any environment, enhancing everything from personalized banking apps to robust automotive systems.

Efficiency is key - and it's the sustainable alternative too

We specialise on restricted devices. Where others see limitations, we see the potential. We have repeatedly demonstrated creating superefficient and highly capable software for small hardware that maximises speed while minimising resource use, thus saving battery life and CO2. With this knowledge, we approached vector search in a unique way. The result is not only a vector database that efficiently runs on restricted devices, but also a vector database that outperforms vector databases from the server space (we’ll share the benchmarks soon). We believe this is no small feat given that ObjectBox still holds up full ACID properties (meaning no data is lost, ever).

Also, keep in mind that ObjectBox is a fully capable database that allows you to store complex data objects along with vectors. This means, you have the full feature set of a database at hand and it empowers hybrid search, traceability, and powerful queries.?

A look at use cases

Fortunately, or unfortunately, ObjectBox can be used for a million different things, from empowering generative AI features in mobile apps to predictive maintenance on ECUs in cars to AI-enhanced games.?

Mobile Apps

On-device AI with an on-device vector database can significantly enhance mobile app experiences, making them smarter, faster, and more personalized. Here are some examples

• Advanced Personal Assistants and Real-Time Translation: Imagine a mobile assistant that not only understands and knows you but also provides real-time translation during voice and video calls. With on-device vector databases, AI can process and analyze language patterns, context, and user preferences on the fly, without any of that data ever leaving your device.?
• On-device Augmented Reality (AR) Enhancements: Future mobile devices will leverage on-device AI to enhance AR experiences. For instance, AR shopping apps could use vector databases to instantly recognize products and provide detailed information, reviews, and personalised recommendations while potentially taking into account availability as well, and suggest alternatives based on personal style.
• Contextual and Predictive Personalization: Mobile phones will soon become highly personalised. The AI will over time know the user better and better and use this knowledge in combination with current data to predict needs and offer timely suggestions. For example, your phone could recommend the best time to leave for an appointment based on real-time traffic data, your current location, and your punctuality habits. This level of personalization will rely heavily on on-device vector databases to quickly process and correlate diverse data sets.

Retail and Point of Sale (PoS)

On-device AI can significantly enhance the shopping experience at the Point of Sale in various ways. By leveraging an on-device vector database, AI can analyze historical data, real-time information, and other relevant data to optimize operations and improve customer satisfaction. Here are some use cases:

• Personalized Shopping Assistance: Imagine having an AI-driven assistant that can provide recommendations based on your shopping preferences and availability. For example, if a customer is looking for a specific item, the AI can quickly check availability and if not available suggest similar products based on their browsing history and preferences.
• Predictive Inventory Management: AI can enhance stock efficiency and reduce costs by analyzing historical sales data and real-time information such as current stock levels, seasonal trends, and customer preferences. This enables the AI to forecast demand accurately, preventing overstocking and stockouts, ensuring that the right products are always available.
• PoS Enhanced Security and Fraud Detection: In PoS systems, on-device AI can heighten security by detecting signs of fraud in real-time. By analysing transaction patterns and detecting anomalies, AI can identify suspicious activities such as false returns, identity theft, and employee theft.

Automotive (SDV)

The software-defined vehicle (SDV) benefits significantly from in-vehicle AI in various areas. Utilising an on-device vector database, the onboard AI can analyse historical data, real-time information, and other vehicle data, such as the driving situation, to determine appropriate actions. E.g.

• Premium digital driving experience: Imagine your car becoming a personal companion, like the iconic Knight Rider. With onboard AI, your car can offer a new driving experience by providing intelligent entertainment suggestions and acting as a co-pilot. This co-pilot can keep you company, suggest actions, or automatically adjust in-car settings to enhance your journey.

• Predictive car maintenance: This can greatly improve the maintenance experience of customers and lower costs tremendously. By analyzing both long-term and current data (e.g. mileage, battery condition, tire pressure, air temperature, vehicle acoustics, driver preferences), the AI has a feel of a car's current condition, can identify unusual occurrences, and predict maintenance needs.
• Advanced driver-assistance systems (ADAS): AI can detect signs of inattentive driving, such as unusual driver behaviour, or unusual eye movement etc. In combination with knowing the driver and all the other data, it can suggest corrective measures, or even take action, if need be.

AI is leaving the “mainframe” era

AI is in a “mainframe era” today. Just like clunky computers from decades before, AI is restricted to big and very expensive machines running far away from the user. In the future, AI will become decentralized, shifting to the user and their local devices. To support this shift, we created the ObjectBox vector database. Our vision is a future where AI can assist everyone, anytime, and anywhere, with efficiency, privacy, and sustainability at its core.