Data Exchange, Open Innovation & AI/ML – The Inflection Point

The digitization of services has generated and continues to generate valuable data in large volumes. However, its potential remains largely unexploited, especially in the case of public data. Whether it's public transport, traffic management, citizen safety, healthcare, farming, or other citizen welfare schemes, the true power of data goes beyond deriving information, insights, and trends. By combining data from multiple sources and creating applications using AI/ML technologies, we can further enhance service delivery efficiency and end-user convenience. This makes data the "new oil" and a driving force of the digital economy.

Smart Solutions 1.0 : The primary objective of this phase is to transition from pen and paper to electronic records, thereby easing storage, retrieval, processing, and management of services using insights, trends, and some level of automation. This is similar to the analog-to-digital conversion (A/D) in the electronics domain and often relies on sensor-based technologies. The Internet of Things (IoT) plays a crucial role in this transition, requiring relatively higher capital investments and implementation time. Examples of such applications include:

1. Camera-based automated traffic violation detection and e-tickets
2. GPS-based vehicle tracking and electronic fare collection
3. Mobile app-based parking search, booking, and payment
4. Monitoring air quality, water quality, flood water levels, and sewerage
5. Healthcare infrastructure, treatment capabilities, and patient management

Smart Solutions 2.0 : This phase focuses on combining data from independent solutions to create applications that result in significant cost savings (20 to 30%) from service delivery efficiency and increased revenue from enhanced end-user convenience. Since data from the 1.0 solutions is the main input here, these applications often require less capital investment, aside from the computing infrastructure, and relatively lower implementation time. Examples of such applications include:

1. Multimodal Journey Planner: Integrates data from water/rail metro, bus, taxi, ride-share, and e-bike services for seamless end-to-end journey planning, ticketing, and payment.
2. Efficient Fleet Planning: Optimizes fleet operations for public transport and solid waste collection based on current status, demand patterns, and traffic scenarios, reducing costs and improving the end-user experience.
3. Flood Prediction and Warning System: Integrates elevation maps, actual and forecasted rainfall, and water levels in dams, rivers, and canals to assist city governance and disaster management.
4. Disease Spread and Medical Infrastructure Management: Combines healthcare datasets such as hospital infrastructure, patient information, air/water quality, digital elevation maps, population density, and traffic/commuter density.
5. Farmer Credit: Utilizes datasets such as land records, soil health, weather information, predicted yield, produce history, market prices, and access to storage and processing capabilities.

Broad Ecosystem Benefit : There is widespread interest and support from all stakeholders for data exchange because of its benefits to each of them.

1. Government Departments: Lower service delivery costs, increased revenue from data monetization, and freedom from vendor lock-in for applications.
2. End-users: Access to innovative and cost-effective services, greater convenience, improved user experience, and efficient service delivery.
3. Industry/Start-up Ecosystem: Accelerated development, reduced costs, and application portability facilitated by data availability, unified data formats, and standard APIs.
4. Academia and Research: Enhanced research opportunities and collaboration with industry through access to diverse datasets and reduced costs.

Challenges : Despite the great possibilities and impact, the following technical and non-technical challenges need to be addressed to realize the potential of the data marketplace and application innovation:

1. Since similar data is often represented in different forms across various systems, there is a need to establish a mechanism for accessing data in a unified format that protects data ownership, privacy, and enables seamless integration.
2. There is a lack of understanding about the quality and completeness of datasets for targeted applications, pricing frameworks, and business revenue models.
3. It is essential to have comprehensive legal, regulatory, and policy frameworks that address data ownership, consent, privacy, and ethical considerations.

Data Exchange Platform (DeX) : The lack of infrastructure to facilitate sharing of any type of data between parties is addressed by the Indian Institute of Science (IISc), Bengaluru, which has developed and deployed a fully open-source data exchange platform, supported by multiple ministries in the Indian government. This platform resolves critical issues that inhibit data sharing by accepting data in any format from any system, converting it to a unified format, preserving data ownership, and securely serving data consumers through standard APIs. This enables open innovation and allows applications to be developed once and deployed across disparate systems and processes with minimal or no changes.

Public and privately owned datasets in urban governance, mobility, healthcare, citizen welfare schemes, citizen security, and farming are currently being exchanged through the data exchange platform and more domains and datasets are continuously being added . The industry and startup ecosystem are leveraging these datasets to develop and deploy the aforementioned applications.

Data Monetization : As you know, the pricing of goods or services is one of the biggest dilemmas in any business, and data exchange is no exception. What is the best price for my data? Should I charge per gigabyte of data shared, or should I charge for a duration like a day or month? Can I ask for a revenue share from their business? If so, what percentage is appropriate? Sounds familiar?

Businesses like Google, Meta, and Amazon have clear pricing structures for their data, but they reached this point after a long journey of experimentation, market understanding, and end-user feedback. Public data sharing and data monetization are still in their early stages, and the diverse range of data types, coupled with numerous potential opportunities and customers, adds complexity to the landscape. To accelerate progress while exchanging value along the way, I suggest the following to public data providers:

1. Recognize the Value : Recognize that your data holds significant potential to create value, which can only be realized by making it accessible to potential users.
2. Showcase and Sample : Showcase your data to everyone and provide free samples to prospective data consumers, allowing them to assess its usefulness and similar to conventional business practices for typical goods or services.
3. Trial Agreements : If a prospective user has a compelling proposition, create a time-bound agreement to share data free of cost. Regularly review their progress and address any issues.
4. Share Value : Be patient and supportive throughout the process. As users create value, enhance your contract to receive a share of the value derived from your data.
5. Pricing Models : Over time, improve the quality, completeness, and ease of use of your data based on feedback and your own research. Develop a competitive pricing model that enables more users and accommodates diverse use cases.

Consider the solid waste collection service most of you are familiar with. The typical waste collection expenses for a year range from INR 40 crore (USD ~5 million ) in small cities to INR 200 crore (USD ~25 million) in larger cities. By combining multiple datasets—such as pickup truck location, distance covered, waste collected, waste yet to be collected, fuel consumed, citizen complaints, waste-generating events, and traffic information—the pickup cost could be optimized by at least 20%, resulting in savings of INR 8 crore to 40 crore. I'm not discussing a theoretical possibility but rather what has been implemented, tested, and consistently delivered results.

Only when such value is created does the worth of your data and a fair share of the savings become a worthwhile discussion. This approach will expedite trading this "new oil" in the emerging data economy. The same principles apply to fleet optimization, multi-modal journey planning, parking aggregation, flood prediction, farmer credit, revenue leakage reduction, and many other areas.

Internalize the value of your data, be generous in sharing it, help others create value, and be vigilant and patient in the process. The benefits will come in multiple forms to you and many others around you. Simple, intuitive, and elegant, right?

Regulatory Frameworks : Understand the regulatory frameworks relevant to your operational domain and location. These frameworks are continuously evolving, so it's crucial to strictly adhere to existing regulations while actively contributing to necessary changes for the benefit of all stakeholders. Don't wait for everything to be set in stone, as they will never be.

Most of the applications mentioned above can be implemented using public data. However, when Personally Identifiable Information (PII) such as names, addresses, or phone numbers of individuals are required, especially for use cases like providing a farm loan to a specific farmer, ensure that data usage is conducted only with explicit consent and handled with care.

The Opportunity : We are at a point of great possibility, an inflection point. This is the time for public and privately owned data providers, the industry/startup ecosystem, government departments, and academia/research to collaborate, exchange data, use AI/ML technologies to unlock its full potential, create impactful applications, and deploy them commercially under new business and revenue models which best suits the data economy.

Let's do it together!

Related Articles :

Megatrends Driving Technology (Urban Living)????????

Smart Townships : Win-win for Tenants and Investors

Smart Cities, Townships : What could be your share?

Trends and Technology will redefine the way we live