National Topographic Database... On the shoulders of Giants... . It was 20 years ago today..

It’s 20 years to the day (or near enough) since we first achieved national coverage of the NZ Topographic Database.

At the time to have national coverage with 1:50,000 scale topographic mapping data like that was pretty much a world first. Along with initiatives like the first iteration of LandOnLine that meant that NZ was truly world leading.

While I’m no doubt biased, nevertheless I suggest it’s pretty incredible that that database survives pretty much in a similar structure to when it was first designed more than 20 years ago. I think there are some lessons to be learned from that...

Happy Birthday NZ Topo

Unfortunately I don’t think we recognise or remember the contribution of those that came before us. Even when the things they produced we still use as basemaps etc every day. So tonight I’m raising a glass of something bubbly in honour of those that produced the original NZ Topo Database 20 years ago. That data is still a cornerstone of much of the GIS data we use in NZ today.

Some of those pioneers are no longer with us, moe mai ra. Or they have retired. To absent friends.

So well done Paul Lundberg, Joyce Bailey, Lorraine Crocker, Baz Parker, Rob Parkin, Robin Pickering, Geoff Howard, Dave Mole, Russell Turner, Fran McNamara and Chris Kinzett for what we achieved. That result still resonates even to this day.

Also remembering all of the cartographers that drew the original paper maps, the surveyors, photogrammetrists and aerial photography people that all contributed as well.

Here’s to you NZTopo... Happy Birthday!

Below are some of the things that I learnt being the youngster on that team...

Lesson: Be Strategic: take the time to design things to last

I remember the amount of work that Fran and other members of the team put into the database design, which took months of effort and that continued to be tweaked for years. I think that one important lesson is that if you design something properly to begin with then it can pay dividends by lasting for decades. Amazing that even with the pace of technology change, that design has lasted relatively unchanged now for so long. How many other pieces of technology can say that? It’s a pretty big testament to those GIS pioneers in NZ that they had the foresight to achieve that. It wasn’t a fluke. It was planned.

There was time spent looking at various emerging international standards (but they often didn’t fit NZ). As we worked through digitising the 1:50,000 scale topographic maps we often found additional attributes that needed to be added. I guess the paper maps also provided the details for what needed to be captured so in a way there were probably many others that worked out the original cartography that indirectly contributed to that design. 

The main contribution I made was to publish the NZ Topo Data Dictionary on a new fangled thing called ‘the world wide web’ also known as ‘the information superhighway’. People used to ‘surf’ it. Using something called Netscape Navigator (rumour has it that there were other browsers available: but I think that that is an urban legend).

Lesson: To execute the design experiment with tools to work out the best solution, and keep on re-evaluating new tools

The first piece of geospatial technology that I used when doing the data capture for the NZ Topo Database was Laser-Scan VTRAK. This system used scans of printing plate colour separations to create a raster, which was then semi-automatically 'traced' to create vectors. After my day of university study I used to go to Heaphy House and use this system to trace contours: often until after midnight. For the most part the job was to start a vector line tracing over a raster contour line and then watch the cursor to make sure it didn't accidentally 'jump' onto the wrong contour line. If that happened then the vector contour wouldn't join up with it's point of origin and I would either have to delete and redo the whole contour or trace back manually trying to see where it had gone wrong. I remember a particularly painful instance where I had digitised around the base of Mt Taranaki and somewhere it had gone wrong: probably about 30 minutes of wasted effort!

I also remember going home at about 1AM and trying to sleep, but instead watching the echoes of glowing contours with cursors tracing around them on the inside of my eyelids. I used to dream of mapping the contours of our mountains...

Once the Laserscan process was complete we could use another system named GeoVision (or Vision) to do topology checking, create polygons and add attributes. Sometimes I would also connect it up to a digitising tablet with a puck to do some additional data capture of small areas that would take too much time to scan etc. It was a pretty good system for the time, but it took ages to do a lot of things that would take seconds these days. We would typically only work on one map sheet at a time (because working on anything bigger would take hours or days to load).

There was a lot of trial and error and lessons learned when we were doing all of this, and lots of good ideas that came up along the way to improve efficiency. We were innovating new approaches all the time, because a lot of what we were doing had never been done before.

It’s nice to think that even though some of that data we captured back then would have been completely superseded and replaced using more modern capture processes (particularly elevation data etc) on the other hand some of the data that we captured by painstaking manual processes 20 years ago is still probably still in use today.

Lesson: Stay focussed on the scope

Yes I know that in hindsight the NZ Topo database design could have been better and even at the time 20 years ago we discussed things like improving the road network connectivity and attribution. There were also debates about alignment with the DCDB roads. There were also many arguments about the role of government vs the private sector and what Linz should and should not be capturing (these debates probably continue to this day). But we had been given our scope, which was to replicate the data on the paper maps, and by focussing on that we didn't get distracted by lots of other 'nice to have' things that we could have done.

Lesson: Open Data

Another thing that I think people have forgotten is that this dataset was one of the first major nz datasets (of any type) to be released as open data. I remember when it used to cost a million dollars under market value cost recovery models and then when all of a sudden it was virtually free. The only company that had actually purchased the whole dataset was Telecom NZ. Everyone else had to choose small areas and limited layers. It has been so beneficial to GIS in New Zealand since this dataset became more open. I remember the shock in other countries at what NZ was doing. Many of you out there probably take this for granted now, but at the time it was a massive battle to make it happen. If that hadn’t have been successful then I wonder how much further back NZ open data would have been today.

All opinions are my own and do not necessarily reflect the views of my employer or any other party.