The Future of Software: A Series of Posts

I work for Cloud Services in the Enterprise Content Division of EMC.  We run a lot of software for our customers.  Software that runs our hardware.  Software that runs virtualization, networking, security, and monitoring.  And all of this to run software to help customers run their businesses.  We run software that is not multi-tenant safe and software that is multi-tenant safe.  Software that was designed 30 years ago and software that was just designed a month ago.  Some software that is exclusively designed for the cloud, and mostly software that was never designed to run in the cloud (but we do it anyway!).  We don't run every kind of software, but we sure do run a lot of it.  Many different variations from many different vendors and engineering teams.  We haven't seen it all.  But we sure have seen a lot.

Some of this software is very well designed.  It is easy to install, patch, upgrade, move, and scale up or down.  It was built with instrumentation, auditing, and the ability to quickly diagnose issues.  Unfortunately most of the software we run is not well designed.  The default install is easy for the person who developed the software, but no one else.  It can't be moved, won't scale vertically or horizontally without great effort, and diagnosing performance issues is almost impossible.  Upgrading requires extended down time, great risk, and almost certainly performance issues or outages until the vendor gets you patches.  When something finally works, we don't want to touch it.  We just want to leave it alone until the vendor absolutely refuses to support it any longer.

Security and privacy is of course another pervasive issue in our industry.  Some systems are very well designed to limit the scope of the data you can access, protect identities, encrypt sensitive data, and obfuscate and generalize data to protect privacy.  Other systems are wide open for attack and massive data theft.

When I got into the software business years ago, I had a foolish notion, instilled in my by my college professors and many mentors, that software would get better and better as time goes by.  People would collectively learn from their mistakes, build upon each other's ideas and inventions, and every year things would just get better and better.  Every year we would make great leaps in technology and never want to go back to how we did things a year ago.  And I have to admit, I've been around long enough to have enjoyed that for the first 5 years of my career.  

I quickly moved from assembly language to Pascal and C to Smalltalk and Java.  Three massive leaps in productivity and reliability.  I tried to move on to 4th generational languages and code generation, but unfortunately they didn't deliver on that massive productivity and reliability promise.  

This pattern continued for database technology, first working with flat files, then indexed sequential files, and on to relational databases.  But object and object-relational databases largely failed in their promises, and I'll get to my thoughts on NoSQL databases in a subsequent post.  

From where I stand, in the last 15 years I don't see a lot of improvement in the fundamental software technology that powers our systems.  Of course there is massive leaps in hardware technology. And through the tremendously hard work of millions of software developers around the world, we have billions of lines of code out there to leverage.  But I propose a fundamental thought for you to ponder: we really haven't seen a game changing software advance in the last 15 years.  That's a bold statement but I'd like you to think about it nonetheless.  I haven't seen anything in a long time that compares to the software advances we had in moving from assembly language to compilers, compilers to object languages with JIT compilation and virtual machines, and the transition from purpose-built databases to relational databases.

As a result of my experiences throughout my career, I thought you may be interested in a series of posts throughout the next year or so on the topic of the future of software.  I'm hoping to raise awareness on the issues we have as a software industry so that we can improve our craft and power the next generation of systems.  And perhaps we can help my friends in the cloud center operations team, some of the hardest working people I know.  Let's make their job easier, they deserve it!

I'd love to hear your thoughts on this topic and will use them to drive future posts.

Thanks!

   Todd Lauinger