SOLVE IT Yorkshire Water Hackathon
Towards the end of March a team from 1Spatial attended the SOLVE IT Yorkshire Water Hackathon, organised by Yorkshire Water and +ADD Strategy. The hackathon was organised so that teams entering could present ideas and innovations to help Yorkshire Water improve actionable insights from new network monitor systems.
One of our team, Aled Nicholas, Senior Consultant at 1Spatial provides an overview of the Hackathon…
In the coming months Yorkshire Water intend to invest in new sensors, network architecture and management systems, with the ideas presented at the hackathon potentially being utilised as solutions. The initiative will see the development of enhanced monitoring of critical sewage network assets and key points for asset failure, as well as building data from extending the sensor coverage in the network. This will provide the opportunity to review the process of event alarm management and event prediction.
With the above in mind, each team was provided historical data as well as recent sensor trial data in order to demonstrate the value of their approach in two main areas:
- Refinement and validation of a given logic model for prioritising event alerts.
- Demonstrating new approaches to integrated alarm management, with particular focus on analytics and insight for prioritising alerts and providing ‘actionable insight and information’.
With the two main areas highlighted, the teams entering would be assisting Yorkshire Water in establishing and developing new insights and creating prototype visualisations that assist in designing intelligent alert management models, with initial ideas being presented at the end of the hackathon.
Upon arrival on the first day of the hackathon, it was exciting to see what would be in store for the next two days. As the 1Spatial newbie (having only started in March) I knew 1Spatial had a wealth of experience in data and helping organisation solve their data management challenges using a combination of our own 1Spatial products and services as well as combining them with 3rd party products. I was therefore looking forward to seeing how our approach as a spatial systems integrator would help us to bring together a range of these products and services to solve the problems presented to us at the hackathon. Still firmly within my ‘induction’ period, I had seen most of these products in isolation so it would be an eye opener to see how these great products could be used in conjunction to solve a real world problem.
Day 1 began with presentations by Yorkshire Water, outlining some of the problems they encounter and what they hoped to see from those attending the hackathon. These problems focused on blockages within their networks that their teams needed to work on. There was particular focus on prioritising these blockages by their severity, how these blockages are highlighted and how actual blockages could be differentiated from false alarms (usually caused by high rainfall increasing the water level within the system). There was also focus on the causes of blockages, with the majority being caused by flushed wipes – even so-called flushable wipes are not actually flushable – and the consequences of the blockages, sometimes resulting in flooded homes. Living in an area served by Yorkshire Water, I was as keen as they were that things did not go wrong very often! Once all of the presentations were completed we were then given the data that we would be using as a base from which to provide potential solutions.
The data provided consisted of various spreadsheets showing blockage data, rainfall data, telemetry, watercourse information, flooded properties and incidents of pollution. This information went back a number of years, so not surprisingly there were literally millions of records. Having analysed and prepared the data, we decided what was relevant and agreed on what we would try to achieve. With a plan in place it was time to get down to business, put on our systems integrator hat, and look at which products might create a good combination and offer a solution to meet Yorkshire Water’s challenge. First up was FME – having been supplied with a MapInfo file, FME was used to convert it into a shapefile. Granted, this only scratches the surface of what FME is capable of but it meant that we could use this data further up the line.
We then needed to join the telemetry alarm data (which highlighted all alarms triggered through high water levels) with the rainfall data. This would allow us to identify which alarms had been triggered by high levels of rainfall (we decided on one value for the whole of the Yorkshire Water area, but this could easily be broken down into smaller areas). Those triggered by high levels of rainfall could then be discarded, allowing Yorkshire Water to focus solely on those alarms which are likely to have been caused by blockages. This was carried out by creating some rules in 1Spatial’s 1Integrate product that joined the two files together where the dates of the telemetry matched the dates of the rainfall level. While it sounds fairly simple, it meant processing around 1.7million objects and was completed in around 25 minutes. This worked out at roughly 1,000 objects being processed per second! That is exactly the kind of capability that 1Integrate has that can only really be witnessed when it is used in a real scenario. The resulting rules would allow Yorkshire Water to replace the historical rainfall data with forecast data, meaning any alarms triggered could be prioritised more efficiently.
Day 2 began with a fun #TeaTimeBanter session to get us all warmed up and then it was time to get back on with the task in hand.
At this point we already had the method of creating an efficient alarm system using both FME to prepare the data and 1Integrate to apply complex rules to this data. It was then time to consider other products to bring into the solution and at this point Geocortex entered the fray. Geocortex was going to be used as the viewer to display the types of blockages, with the most common blockage within each Drainage Area Zone (DAZ) shown. This would allow Yorkshire Water to identify the most likely cause of a blockage simply by looking at the map interface. This would be useful in isolation, allowing Yorkshire Water to identify any areas that could benefit from additional information – such as a leaflet drop about flushing wipes. It could also be used in conjunction with the alarm system, ensuring that any alarms raised would give Yorkshire Water a good idea of the cause of the blockage before arriving on site. With presentation time fast approaching, some example reports and print templates were created to further highlight the benefits of using Geocortex as a viewer.
The time for presentations was upon us, with an overview of the solution ready to be delivered. Team 1Spatial were fourth in the running order and having seen the first three solutions delivered, it was clear that all of the teams involved had had their own innovative ideas and that competition was of a high standard. Nevertheless, we were confident of the solution we had generated.
The presentation was delivered superbly by Jess, with Chris giving a demonstration of the business rules on 1Integrate and the mapping system on Geocortex. The initial focus was on showing how Yorkshire Water could have an effective alarm system that discarded false positives through the rainfall level, leaving only alarms that will have been triggered through blockages. Although the demonstration was carried out with historical data, this model could easily be modified to include forecast rainfall to give an indication of future incidents. The focus then moved to the interactive mapping system, showing base mapping overlaid with the DAZ/blockage type data and the additional functionality that Geocortex could offer. After discussions with the staff from Yorkshire Water, it was discovered that different roles within the organisation would be required to access the system and each role would have responsibilities for different areas and have differing priorities. This made Geocortex ideal, as each user could be assigned their own log in configured to their own requirements. The presentation concluded with an overview of what 1Spatial could achieve with Yorkshire Water and how we could improve their overall mapping of their network using our rules based system.
With the presentation delivered, we were confident that the benefits of the system were clearly understood and they were well received. All that remained was to view the remaining presentation by the other teams that had entered and, as with those that had presented before team 1Spatial, there were some fantastically innovative ideas presented.
Overall, the hackathon proved to be a great event which was brilliantly organised by +ADD Strategy and Yorkshire Water and attended by talented teams from various companies ranging from large engineering companies to newly formed sensor companies and everything in-between. On a personal level it was great education to see how 1Spatial can quickly and seamlessly bring together a range of products and services to create an end solution perfect to meet a customer’s challenge. There was one other reason why it was a great event – team 1Spatial won the award for Most Effective Alarm Prioritisation!
