The great application migration of 2003

In 2002 BP’s Digital Business Unit faced a major application migration in the subsurface exploration and production software infrastructure. As an available project manager at SAIC I thought this sounded like a lot of fun! The project was going to be very complex in terms of the moving parts that were going to be affected and essentially there was only one rule – do not impact the current infrastructure or any of its users. 

This project was segmented into 4 phases. Phase I was to envision and build the lab in which all of the testing could be done. Phase 2 was to perform all the tests to ensure that not only would the Landmark environment be stable but that all of the applications that interacted with it were also stable. Phase 3 was to deploy the result seamlessly over a weekend. Phase 4 was to deliver that same applications environment to each of the operating business units worldwide.  And all of this was to occur in such a way that operations would not be impacted with lost days fixing things. 

I had a great BP sponsor I will never forget – Donal Griffin. Without his assistance I am confident this project would not have happened so cleanly. I worked with Don to define what was needed physically in a test lab to insure that we would meet all the project criteria. This in itself was a significant undertaking. We needed to represent the entire cross section of desktop systems in use at the time of the migration. So we located an SGI deskside Onyx that was representative of the BP Hives, an SGI Octane and an SGI Indigo. In addition we installed 5 different Sun Microsystems that were essentially the pizza boxes each with a different graphics card. Yes, this was the pre-Linux days almost. The last system in the room was a high end PC that was purchased to begin coverage on the Linux platform.

We also installed the Landmark OpenWorks environment – one for each of the platforms. This was required since we needed to test standalone without affecting the rest of the network. As the environment was built out Fred Jenson was seconded to the project. Fred was a god send! The project also would never have finished without him. 

To insure that all the current workflows would continue unaffected we also had several PC Windows based systems used mostly by the engineering and petrophysical groups. The environment there was in fact one of the most complex elements of the entire lab. By the time we were done installing everything it almost seemed like a group of the applications were mutually exclusive because of the way they were configured to use X Windows on a PC Windows environment. 

In addition, the lab was also a hot bed for testing monitors. We had Sun monitors, Apple monitors, SGI monitors in dual and single head configurations. Of course the network was setup so that only the lab systems could find other lab systems. At this point we went to work installing all the applications on the platforms in a way that mimicked the rest of the BP environment.

Phase 2 was in full swing at this point. This environment was a great challenge because there were over 200 different applications installed. Each with its own script to start it and its own place on the application launchers. As testing progressed there were several times where some of the applications would simply not access the Landmark data stores. The testing team would identify these and track them on a big “bugs” spreadsheet. At times that list got fairly long! The vendors would resolve the issues and give us a new set of executables that we then installed and tested. 

One of the key challenges was creating and maintaining new scripts for each of the application suites. Each on its own was quite simple but keeping track of the sheer number of changes was time consuming. There were a couple of applications that were not in general use and required special knowledge to use – in those instances we brought in the actual end users to test to see if everything was working. 

Another critical aspect of the environment was to have a sampling of projects from BP that would represent the diversity of projects available in the actual business units. Most of our initial tests focused on getting a full range of data types, data volumes and test these for performance and workflow related issues. The BP environment was also quite complex in terms of the number of users and the number of underlying OpenWorks servers. 

Phase 2 lasted for about 6 months! Lots of apps to test! Lots of configurations! But Fred and I finally got through most of the testing. During that time, we also had great contributions from the Landmark team in terms of bug identification and bug fixes. We had lots of people through the lab. We also had great help from the SAIC data base management team to ensure that they understood the migration scenarios for the environment since they were key to the actual migration.

Now it was time for Phase 3. This involved insuring that the environment could be cloned and placed within the actual end user environment without falling over. For this we had great help from SAIC engineers who understood all the intricacies of the network and how to build a “clonable” disk drive suitable for delivery to each of the business units. We ended up buying a bunch of disk drives and putting together an outline of how to install once it reached each site. This was more complex than I had imagined but we finally figured out the right environmental variables and we were ready for prime time.

Phase 4 happened first in Houston. The gurus at SAIC put the new suite of applications into the current environment. Once installed each of the Landmark projects were migrated to the new environment from one OpenWorks system to the next. Then the user environments were migrated. All of this happened over a single weekend and when the BP staff came in the next Monday everyone had functional environments using the latest applications. I was really proud of the massive effort put in by all of SAIC staff and the BP infrastructure staff to ensure this happened seamlessly.  

Looking back on it I think ‘Wow’ – not only a great project but one that was so formidable that success had only one path!