I was App-Centric, but I'm Alright Now

Great Failures I Was Part Of (but I can prove I didn't do it!) - Episode 1

I recently signed the Data-Centric Manifesto, which tries to get all of us to do an about face when it comes to the apps we use, and the data we put into them. I didn't have a snappy word for it, but I became data-centric early on.

I responded to an ad for an Excel 3 expert, and got a contract with a local government to streamline their process of preparing, comparing and tracking construction project bids.

On a lengthy spreadsheet, someone typed in all of the items in a project, such as 2 cubic metres of concrete, or 10 metres of drainage pipe. Pre-qualified construction companies had to bid on each individual item, so if it ended up that 3 cubic metres of concrete was required later on, a calculation could automatically be done for how much was owed.

At the time of bidding, only one number mattered - the grand total. By law, the lowest was the winner, even if just by a penny on a 20 million dollar project.

But first, the spreadsheet had to be printed, sometimes onto 50 or 100 pages, for distribution to the dozen or so pre-qualified companies. Each of these companies would give the paper to a typist for entry into a spreadsheet. Sometimes it was typed in twice so a comparison would reveal any mistakes. After prices for items were entered in these spreadsheets, each company would enter formulas, do some formatting, and print the result on paper for submission. 

After bidding, the numbers from the winner would be re-entered, twice, to verify the correctness of the grand total, and to track materials and cost changes as the project went on to completion.

Obviously there was a huge duplication of effort, which brought with it an increased chance of mistakes, which was to no one's benefit. Why not switch the bidding process to an electronic one where the spreadsheet used to print the traditional paper copy was also available on diskette, and details of bids would be accepted in the form of numbers entered in this spreadsheet.

Construction companies were contacted, and they were in near unanimous agreement that they would use this system. And, they were all using Excel 3.

So the ad went out and I was hired, to program templates and macros to do calculations and make things look pretty. I did, and after close to a year it was slick, speeding up work and reducing redundancies, but... Excel 3 and its macro language was buggy. 

I spent a lot of time on work-arounds instead of adding features, but it came together, except for one bug.

On other operating systems, letter spacing and word-wrap are first calculated assuming a 1000 point font, then adjusted for the actual pixel size of the screen. This ensures that word-wrap is done consistently. At the time, and perhaps to this day, Windows calculated letter spacing, which was then rounded-off, according to the display's pixel size, so word-wrap could be inconsistent. When told to resize cells to fit word-wrapped text - Excel could be inconsistent too.

It was rare, but with the right combination of screen size, zoom settings, and printer, a word that showed up on the screen would not show up on paper or vice versa. I saw it twice in 10 months.

With this one bug left to overcome, the legal department was briefed. They cancelled the project.

If one company saw the item "10 metres of drain pipe", while another company saw the same item as "10 metres of drain pipe installed", and if this changed a final bid by a penny, the local government could face a 20 million dollar suit. Because it was a bug in Excel, with no hope of us fixing it ourselves, the project was brought to a swift and unsuccessful end.

The project was doomed before it began. The ad mentioned Excel - I was hired because I was an Excel expert. So hey, it wasn't my fault.

The project failed, not because someone preferred some other spreadsheet product, and not because it wasn't received well in demos and trials, but because we were all app-centric. If I or anyone else had been turned on to data-centric thinking, there would be no problem to fix.

What we should have done was offer the same system with some minor tweaks - or at least they could have been minor if the project had started off right. We should have offered open format CSV files to the bidders, rather than Excel files. The same preparation would go into creating the Excel files, but on the File menu we could have hit Export to CSV instead of Save.

On the other side, companies could import the CSV into Excel. With copies of the templates and macros used in the preparation phase, with a few clicks they could have re-created the exact same Excel spreadsheet, as if we had handed Excel files to them instead of CSV.

The templates and macros I created could have been offered to the companies for free to get their buy-in. The CSV files could also be printed on paper, they just wouldn't look good for those few who cared. The only difference would be, the legal department could prepare a statement that the free macros were provided as is, with no warranty, and with a warning about Excel's bugs. Only the accuracy of the data itself would be an issue.

The difference would have been minimal. But actually, I think it would have led to a better overall design. The CSV format would have forced a decoupling of components which probably should have been decoupled, leading to cleaner and better design.

This simple realization after the fact turned me into a data-centrist. I didn't have a term for it, but do now since signing the data-centric manifesto. It isn't a specific technology - it is a mindset where a class of better solutions are constantly brought to mind. I realize now that I have come around to a data-centric mind-set since that great failure I was part of (but have proof it wasn't my fault)!