AVOID IT PROJECT FAILURE BY HAVING PROPER TESTING PHASES

How we define either a project is successful or fail? is it enough that the implementation passed these four common acceptance tests such as Provisional Acceptance Test (PAT), User Acceptance Testing (UAT) and Final Acceptance Test (FAT)?

If you are new into IT project, you may say yes, but sorry to say, that’s not what it should be, reason being is that, if you are not lucky, you will only realise about the issue towards the end of the project completion, at that stage you will be in dilemma, if you issue stop work, the cost to repair might be expensive, but on the other hand, if you close one eye and get it accepted with some workarounds, you are risking the organization.

The first phase will be specification test, here you need to look at your purchase order documentation with the datasheet and compare it with the equipment specifications, a good purchase order or delivery note will list range of part numbers and the quantity, check it one by one. It is common that due to the long procurement process, you will not be given what you ordered, because of the item or their parts already End of Life (EOL), and the supplier will replace it with an alternative which should be similar or better. If everything is in order, power it up. There is a term Dead on Arrival (DOA) which means that the equipment you received is faulty, some vendor allows one to one replacement immediately but some will ask you to do a Return Merchandise Authorization (RMA) that will take some times either to be repaired or to be replaced. It is advisable that these conditions being raised in your procurement documents. There are some customers that will not only test the bootability of the equipment, but they do burn-in test, where the equipment will be left running with some software or environment for couple of days to see if it failed or not.

The second phase is after the equipment being loaded with all building blocks like operating system, software and configuration, while looking at the resource utilization, alert monitor and logs in finding any abnormalities. Here the technician must know various level of triggers by looking at some best practices, such as for a firewall, what would be a normal CPU and RAM utilization after enabling various modules before accepting traffics.

The third stage is by checking the interoperability with the adjacent chains equipment, forming an end to end system. The common monitored attribute will be in finding some bottleneck. In computer system parameters are observed for one second readings; I have seen a case where an organization purchased large hard disk for their storage, because of spindle’s limitation the whole system process degraded until they had to change their disk into smaller units with similar pool.

Forth test will be via CIA triad, confidentiality, integrity and availability of data from the origin until the destination. CIA triad is commonly used for information management in security domain, but here we should be looking at what are happening to the data, especially when the system requires some data cleansing on-the-fly or functions that involve Extract – Transfer – Load (ETL). We need to look at the data lifecycle at all three stages, data in use, data in transit and data at rest. Data behavior must be as what expected.

Once all those four phases tests done, then only we can proceed with fifth test, that involves Performance Testing, Load Testing and Stress Testing; as a reminder, prior to that, test environment, scenarios, cases and scripts must be defined and agreed with all parties involved, so that everyone knows what defined as pass, fail and the breaking points. Along with this, it is advisable to have a scenario planning, as what to do if the test failed. Some organization will do a segmental load testing, but it is a best practice to have a proper wholistic test to see any breaking or failing point.

The last level six, that normally most people never do is business objectives vs imperatives test. No matter how sophisticated or state of the art IT system that being put in place, if it cannot meet the business needs, then it simply a failed project. Here you can see that the implementation passed all the earlier five tests, but the system not used due to a reason that it is not user friendly or what so not; some people call this as white-elephant project.

Just to note that, those PAT, UAT and FAT normally done before the fifth test phase or some may do it within the phase as well.

Some may wonder these test methodology sounds like water-fall development lifecycle, and not agile, well to have agile in your testing, perform all these tests at modular basis before the iteration, then do one round after all the jigsaw puzzles assembled.