Visualising your IoT and Data using Immersive Analytics and Virtual Reality

Back in early 2000 when I started developing on mobile devices such as Palm Pilots, Windows CE hand held computers, I realised the future of these devices will be to act as full citizens on the internet. By definition, my interpretation of a full citizen on the internet is to be able to add information and be autonomous, can interact with other devices without the need to communicate to a central server. 

At that point in time, I had just begun my Ph.D. and had visions on how I can play a role in empowering mobile devices to interact with each other. The first task I had was to run a web server daemon on the mobile device, make sure it was able to listen for and post requests using, at the time, popular XML web services. We spent about two years developing a platform that empowered Windows Mobile devices to host web services. This research allowed us to submit several papers and gain international recognition for this concept. We coined the term Microservices since on a mobile device, the services needed to be lightweight and independent of each other without the need of state.

Figure 1: Our first international conference paper at Pervasive 2004

Mobile devices and embedded devices started to proliferate, we visited HP Labs and explored their “future lab” where the day starts and your mirror updates you on a delay to the airport and books a cab for you early so you can avoid the traffic congestion. Then we walk into the storeroom of the future were all items had RFID tags and you could just pick up the item and walk up to the counter, that automatically updates the employee on the item and price. 

However, with all these devices and items now internet enabled, how could we visualise all this information? Thousands of different sensors and data types pushing data into a data store, how can we visualise this?

That is when I started to work with the Data Mining team at Monash. The team had developed a way to visualise data on a mobile device that is clutter aware. The challenge the team had was that data mining and visualising on a mobile device was resource intensive and the battery life was poor. We decided to add an adaptive element to the algorithm and adjust the amount of resource consumption based on battery life. 

A video of the way the visualisation works is provided bellow (click on the picture), the device identifies when the clutter threshold is reached and then changes the visualisation from size to colour density (Video 1). 

Video 1: Clutter Adaptive Visualisation

We also added a battery management component to adjust a screen refresh and backlight, this gave us an extra 2 hours of battery life on the device (Figure 2). This resulted in a few publications (https://ieeexplore.ieee.org/document/5670093, https://dl.acm.org/citation.cfm?id=2430803) and us contracting work with Victoria Police. 

Figure 2: Power consumption with energy consumption being monitored  

To reflect over 10 years ago, these concepts are very real and now we have SAP Cloud Platform to provide this capability. SAP Cloud Platform can support device interoperability by exposing numerous protocols including HTTP, MQTT and other IoT based protocols. The SAP Cloud Platform can also scale and with the ability for us to exist on OpenStack, we can support non-SAP based languages including Python, Ruby and other popular environments. SAP Cloud Platform (Figure 3). This includes our IoT connected apps that is part of our Leonardo portfolio such as Vehicle Insights.

Figure 3: SAP Cloud Platform

At SAP, we used our SAP Vehicle Insight product to develop our MyShuttle app that can save SAP 410K euro/year and by optimising travel based on real-time demand through the app (Figure 5). These apps were developed using our new Fiori for iOS SDK, making it simple and elegant to use on our Apple devices.

Figure 5: SAP MyShuttle helping save costs

The SAP Cloud Platform can also expose SAP services to be consumed by non-SAP technologies. This is the approach required on how we could incorporate Immersive Analytics, a topic that I am collaborating with the Monash University SensiLab to provide full sensory approach to visualise analytics. Imagine being able to feel when there has been in a change in a sensor or a real-time financial feed. Imagine walking through your data, looking at your data using different angles and then interacting with colleagues how to visualise the data. 

Below I have provided my latest VR experience which is a the VRVault (Video 2). Recently I had developed a gaming front end for our Banking services, the ability to visualise your money, possible transactions physically appear/disappear. This could also be used to visualise your wealth, how a Wealth Manager could visualise your wealth over a projected period based on recommended investments/strategies. 

Video 2: The VRVault, a VR financial experience developed on top of SAP Cloud Platform

I am excited with the potential of the SAP Cloud Platform and its possibilities, from capturing data to immersing yourself in data, the SAP Cloud Platform is available so you can develop and test your ideas. You can start with a free trail and even register for free courses online using our openSAP training courses. 

Feel free to contact me via LinkedIn or Twitter if you’d any further information about developing on SAP Cloud Platform, here is the site to get you started.