SW Test Levels

Here we are going to talk about SW testing levels from perspective of automotive SW , so the examples may be related more to Embedded SW testing

test levels are

1- unit testing which is testing the most fundamental unit of code in isolation e.g. a function in code

2- Module test which is usually a replacement to unit test , target is to test each module of code in isolation to test the module external interfaces which triggers the internal module state machine and exercise all the internal functions , also Model based testing may be included under same point here

3- SW integration test, test the SW of components after integration, and integration of SW component or stack with micro controller HW only

4- SW verification testing, test the complete SW

5- System test testing the complete product

We could say testing is a 4 legged creature

it walks on 4 legs :

- unit verification ( including Module testing & Model based testing )

- SW integration

- SW validation

- System test

losing one of them shall lead lose its ability to run

you could not improve your testing by focusing only on one test level and ignore the rest of test levels

testing techniques are divided into

1-black box, specification based e.g. boundary value analysis

2- white box , structure based tests , measures the structure coverage of the tests done, e.g. code coverage

3- experience based

usually we find a lot of colleagues in the industry name unit test as white box testing activities and system test as black-box testing activity , here black and white mean know the design and does not know the design, but does not mean that we are going to use only white box techniques or white box only, that is the misconception in unit testing we need to cover first the requirements on the function level then we check the structure coverage of our tests in system test we need also start by system requirement , also possible to measure the structure coverage for system tests, here you shall ask me how as we could not see the code any more and are running on real target not a simulator, the answer is special tools are used to instrument the sw then the instrumented sw is used which is designed to collect the structure coverage while executing , that is a feature in commercial unit testing tools called system test ok now you may ask why to calculate statement coverage for system test, the answer is simple if your system tests only exercise 30% of your code do you think your are preforming good enough testing for different scenarios?

so here we saw that unit test need to use the black box techniques and system test may need to use white box techniques as testing techniques are independent from testing levels

for white box testing techniques we need to start with requirements to write test cases , then after we cover the requirements we start to check the code coverage. then you need to check if your structure coverage is 100% or not

the point is to see if we have a scenario that we forgot to cover or a corner case missing. we could add tests to cover all corner cases that we forgot about , take care not to cover all cases without thinking first about if we had a case that development team had added code without a design for that code in requirements of component is that case this is treated as a bug until the design is updated and reviewed /approved,other case that we might have extra code added due to best practice or coding rules , e.g. misra c rules requires that each switch case have a default case even if not needed (if you already covered all cases for switch case ) in that case the coverage could not reach a 100% which could be justified and accepted .

common mistake of we focused on structural coverage and forget about requirements, state machine of component, interface tests, and component functionality.Module testing and Unit testing are key activities that is needed to secure the SW , same as other testing activities