Case Study: UEnergyHub Canada - An Energy Collection Data Hub for Research

Introduction

UEnergyHub is a pioneering energy data hub designed for ethical research, providing real bill and meter data directly from utility account holders to universities. It is the first automated energy data service in Canada, and possibly the world, dedicated to unbiased research and education in the energy sector. This Case Study provides insight into the University of New Brunswick (UNB) being a third-party receiver of bill and meter data for ethical research.

Objectives

The primary goal of UEnergyHub is to empower various stakeholders, including energy customers, researchers, utilities, innovators, and governments, to participate in energy research activities in Canada. This collaboration aims to enhance understanding and management of energy consumption, ultimately contributing to creating solutions for the energy and climate crisis. UEnergyHub leverages Screaming Power’s EZGB service designed to act as a continuously updated energy data hub for third parties to access information from over 50 electric and natural gas utilities.

Stakeholders

Utility Customers: Authorize the use of their energy data for research purposes. The platform simplifies authorization and secure sharing / retrieval of bill and meter information. Researchers: Universities and researchers gain access to a centralized hub of multi-utility energy datasets, facilitating comprehensive analysis of real energy data. Utilities: The platform provides a link between utilities and research teams, helping to address issues in the energy ecosystem by supporting an automated process for researchers to access energy data through the utility customer. Innovators: New and existing solution providers can test and validate ideas using real-world data through universities, supporting a new opportunity for collaboration. Governments: The platform supports unbiased review, aiding in developing more effective energy policies.

Functionality and Process Flow

UEnergyHub operates by allowing utility account holders to authorize the reuse of their data while respecting data privacy. The platform provides an easy-to-use interface for managing and sharing data with researchers. This process has already been proven to support various research initiatives, including the analysis and comparison of energy consumption and billing patterns through unbiased University research.

UEnergyHub → Energy Customer - UEnergyHub provides electric and natural gas utility authorization services to the Energy Customer (account holder) once they register.

Energy Customer → Energy Utility - The Energy Customer authorizes the collection of their energy data.

Energy Utility → UEnergyHub - The Energy Utility sends energy data to UEnergyHub on an ongoing basis.

UEnergyHub → University - UEnergyHub shares data with the University for research and educational purposes in its natural form or translated to more user-friendly formats.

UEnergyHub → Energy Customer - UEnergyHub presents data and allows the customer and the University to reuse the data as it is automatically refreshed in UEnergyHub by the utility.

Case Study - Research Focus

Key areas of this case study include: University Research: Collaboration and education to identify data for opportunities. Data Authenticity: Ensuring the accuracy and reliability of data. Regulatory Compliance: Evaluating how well utilities comply with data access regulations and providing the ‘best available’ data.

Case Study Project Goals

Anomaly Detection: Developing methods to identify and resolve anomalies in energy consumption data using machine learning and other methods.

Data Validation: Ensuring utilities provide accurate and up-to-date information.

Educational Framework: Building a network to educate and take action on energy usage and emissions.

Collaboration: Creating new business relationships between stakeholders.

Insights and Research Findings

Ease of Data Donation: UEnergyHub simplified the process for electric and natural gas utility customers to donate their energy data, facilitating the collection of consumption and billing patterns for research and educational initiatives. This ease of use also supported continuous reauthorizations needed to manage and validate utility issues.

Costs: UEnergyHub avoids Utility setup (onboarding) overhead. It provides the University network a service that deals with Utility operational issues in an unbiased way. This allows the University to focus on data analysis while greatly reducing data acquisition costs and efforts on an ongoing basis to access real energy data. The Hub also allows the account holder to share their data at no cost in a matter of minutes.

Collaboration and Reporting: Although the University was not required to communicate the issues found, key problems were reported to the utilities and the government through Screaming Power and subject matter experts (SMEs). This added complexity to the research process, causing delays due to lengthy communication and collaboration with multiple stakeholders. However, it helped utilities improve their compliance with regulations and standard software testing practices. The EZGB platform, from which UEnergyHub was created, is now the main testing infrastructure in the Ontario market.

Continuous Improvement: UEnergyHub had to continually adapt to accommodate for ease of use and log utility issues, requiring constant monitoring and updates to the system. Ensuring consistency and adherence to a standard format for the data provided by utilities was challenging but crucial for validating the “Source of Truth” data. Changes by some utilities improved their systems as the study continued. Timely utility support was needed as the research continued, but was often lacking as utilities had competing priorities.

Utility Preparedness: Despite regulatory certifications required by Utilities and utilities confirming solutions were implemented by November 2023, many were not fully prepared to handle customer data requests. This gap between regulatory compliance and operational readiness posed significant challenges, delaying research activities and requiring adjustments to research goals. Screaming Power supported defining and resolving utility implementation gaps, with testing driven by UEnergyHub’s infrastructure using Screaming Power’s EZGB service which includes diagnostic tools.

Data Quality Issues: All utilities providing data to UEnergyHub faced operational and data challenges, including inconsistencies and inaccuracies. The research identified these issues and the unpredictable loss of connection to the utility infrastructure from time to time. These issues complicated analysis and validation processes, related to this new utility digital sharing landscape. The research itself served as an evaluation tool for the utilities’ infrastructure, revealing that neither the utilities nor the regulator had independent means to appropriately test and validate this new data marketplace. This led to necessary changes in the data hub toolset to identify, report, and manage irregularities.

Understanding Data Standards: The research emphasized the importance of understanding standardized processes and ensuring the validation of “Source of Truth”. The research team worked directly with the utility data source rather than the translated outputs available in UEnergyHub. This phase was crucial for accessing previously unavailable energy data. Although meter (usage) data comparison was key, analyzing the authenticity and operations of utility infrastructure was necessary to ensure the data and utility operations become trusted sources for future research and business practices. Researchers had access to both translated data for easy reuse and raw outputs within the Hub for comprehensive evaluation.

Sources of Truth: The research team used two different sources of truth from utilities: UEnergyHub, providing direct access to data defined as “Best Available” by the regulator, and data obtained through a legacy business-to-business process, which required a more manual authorization process and offline delivery to the researchers. Managing these different data sources required additional cost, effort, external support, and coordination. The UEnergyHub demonstrated a concrete benefit over the legacy route of data acquisition managed by the Ontario Energy Board (OEB) Electronic Business Transactions (EBT) infrastructure.

Support for University Research: UEnergyHub supported UNB in gathering data for phase 1 of their research. This research was financially and in-kind supported by Mission:data Coalition, Screaming Power, and multiple industry SMEs from May to October 2024.

Enhanced Features for Analysis: Due to research requirements, UEnergyHub expanded its feature set to include tools for comparing uploaded PDF bills with data exports from utility sources. Manual importing of source data was also created when automated retrieval processes were unavailable. Continuous improvements to the platform enhanced ease of use and logging of utility communications and data issues, benefiting all EZGB service customers.

Impact on Costs and Emissions: By sharing energy data and understanding usage patterns, the education community now has a data hub to identify opportunities to reduce costs and greenhouse gas emissions, contributing to a sustainable future.

Positive Surprises and Unexpected Benefits

Strong Engagement from Utility Customers: There was a high level of engagement from utility customers willing to donate their energy data for research purposes. Despite needing to reauthorize several times due to utility technical issues, this enthusiasm was maintained and helped gather data, crucial for the research. The research engagement also improved the evaluation of utility systems, as the UEnergyHub (EZGB) infrastructure assisted with utility communications and data testing.

Effective Collaboration: The collaboration between the University of New Brunswick (UNB), not-for-profit organizations, residential and institutional utility customers, and private SMEs was highly effective. Although utilities were not fully aware of the research itself, they were very interested in fixing their systems. This network continues to work together, enhancing the quality and scope of the research while improving the marketplace.

Government and Utility Support: Despite initial challenges and limited support from the OEB and the Ministry of Energy, utilities were receptive to resolving issues once the issues were communicated and proven. UEnergyHub’s tools provided the proof which facilitated improvements in data quality, communication, and regulation compliance by the Utilities.

Machine Learning Applications: Researchers successfully employed machine learning and data mining techniques to identify and resolve anomalies in energy consumption data. UEnergyHub also developed machine learning techniques to manage the non-standard datasets and the different operational processes of the utilities. This technological advancement added significant value to the research while working with different datasets.

Improved Data Standards: Two years of data history was provided through UEnergyHub, usually within a few minutes when the utility system was fully performing. Research highlighted the importance of data standards and oversight, leading to improvements in the quality and consistency of data provided by utilities. This improvement process is ongoing and expected to have long-term benefits for future research and energy/GHG management initiatives.

Increased Awareness: The project raised awareness about the importance of accurate and reliable energy data among utilities, regulators, and the public. This increased awareness is likely to drive further improvements in data quality and regulatory compliance. A research white paper created by the University provides detailed results for public review.

Enhanced Utility Operations: Feedback and data analysis from the research team, Screaming Power and SMEs helped utilities identify and address operational inefficiencies, leading to improvements in data management practices and overall service delivery.

Regulatory Compliance: The project highlighted areas where utilities needed to improve regulatory compliance and issue turnaround times. The research also emphasized the need for the government to focus on more effective and timely compliance measurement and action. A several month turnaround time is not adequate for this type of technical rollout.

Innovation and Development: Collaboration between researchers, utilities, and innovators led to the development of new tools and solutions for data management within UEnergyHub. Innovations and improvements have been scaled and implemented more broadly, benefiting the UEnergyHub and Screaming Power’s EZGB third-party hub platform.

Economic Benefits: By identifying inefficiencies and opportunities for improvement, UEnergyHub helped Universities and consumers reduce costs and effort. This economic benefit extends beyond the immediate participants to the broader energy community.

Community Engagement: The project raised public awareness about a new way to gather data from Utilities, which will assist everyone to manage energy consumption and sustainability, leading to more informed and proactive energy use behavior.

Policy Development: Insights from the research provided valuable data for policymakers, and regulators. Accurate and comprehensive information with unbiased assessment is the cornerstone of an effective and inclusive marketplace.

Strengthened Partnerships: The project fostered stronger relationships between universities, utilities, government agencies, and private sector innovators. These partnerships are expanding, leading to further collaborative projects and initiatives. The UEnergyHub pilot demonstrated the infrastructure’s reuse by multiple universities globally and provided a nexus for collaboration with the US and the Caribbean.

Technological Advancements: The need to handle and analyze large energy datasets led to advancements in data infrastructure and analytical capabilities. These technological improvements were applied to UEnergyHub.

Privacy Measures

User Authorization: Explicit Consent: Utility customers and their delegates had to explicitly authorize the use of their data. Utility customer consent was obtained through a clear and straightforward process within UEnergyHub that evolves as new use cases appear. User authorization ensures that customers are aware of what data was being shared and for what purpose(s).

Transparency: Clear Communication: UEnergyHub provided information to utility customers about how their data would be used. This included details on the types of research being conducted and the benefits of participating in the data sharing initiative. Privacy Policies: Privacy policies were made available within the UEnergyHub solution to outline the measures taken to protect data and rights as data providers and users of the UEnergyHub infrastructure.

User Control: Data Management Tools: Utility customers were provided with tools to manage and share their data. Revoking access to their data at any time is a feature that allows the user to go to the utility anytime to revoke access. UEnergyHub provided the ability for the Utility customer to view and download their data; giving them full control over their own information and allowed them to reuse the data themselves.

Educational Initiatives

Educational Impact: UEnergyHub fostered an educational community focused on energy data management and sustainability, helping identify opportunities to reduce energy costs and greenhouse gas emissions. Student Involvement: Students gained hands-on experience and valuable insights into real-world energy issues, benefiting their future careers and providing contributions to the energy sector.

Conclusion

The UEnergyHub project advanced energy research and education while delivering significant operational, economic, and community benefits. These advantages highlight the impact of collaborative energy research and its potential to drive positive change across multiple domains. This research also demonstrated the concrete benefit of the UEnergyHub, and in turn EZGB, in the form of cost and time savings relating to energy data acquisition.

The research findings underscore the importance of collaboration, data quality, and regulatory compliance. UEnergyHub’s ability to facilitate easy data donation and provide comprehensive analysis tools has been invaluable for advancing energy research and education.

By leveraging real-world data, UEnergyHub supports ongoing education and research efforts aimed at creating a sustainable and inclusive energy participation network while involving our future work force.

Unexpected challenges highlighted the need for utility preparedness, improved data quality, and effective collaboration. Addressing these issues were essential for the project’s success and ensuring the reliability and accuracy of future research activities.

Positive surprises laid a strong foundation for future collaboration with multiple Universities. The project demonstrated the potential for innovative solutions and effective partnerships to address energy and climate challenges. Public exposure to UEnergyHub has led to Memorandums of Understanding for additional energy research and collaboration in other energy markets.

Educational initiatives effectively informed utility customers about their rights and options regarding data sharing, building trust and encouraging participation. The research provided insights, which are being reviewed and implemented as UEnergyHub expands.

As part of a SaaS offering, UEnergyHub’s enhancements to the core EZGB solution have improved the review and communications infrastructure for everyone. EZGB provides centralized data access and a robust solution for over 50 electric and natural gas utilities.

The platform is looking to add more research groups to assist with ethical research.

See the first release of the Universities research here

For more information on UEnergyHub including University Research inquiries please reach out to me or visit our website.