General challenges in API testing, personal and documented perspective:
- Manual API testing do not usually test the edge cases ( mainly the happy path)
- Manual API testing do not cover all the possible combinations — task for automation but layered architecture will make always difficult to cover all relevant ( not possible !) test cases
- In manual API we can not measure code coverage
- The manual does not usually come after automation to validate the result ( so do not avoid the manual testing )
- Execution time in manual API Testing: manual testing is too often too time consuming
- Unit tests usually test individual components not multi services
- Mock services in a meaningful manner ( dependencies between services )
- Automatic CI/CD to enable short feedback cycles
- Automatic detection of data structure and each finding to lead to other findings
- Generating new inputs from existing inputs ( algorithms)?in the most structured way ( not randomly)
- Web applications crushes
- Recognize all dependencies
- Find code vulnerabilities ( infinite loops, exception handling )
- Not all automated testing approaches are equally effective or how to apply the right mix of technologies to rapidly create intelligent automation solutions
- Some automation test tools do not have any reporting capability by default or do not have their own IDE — integration is needed ( see selenium )
- The fastest way to automate is to use Black box testing for API but Black box testing does not take into consideration the code coverage
- In automatic Black box testing tools the amount of tested combinations is usually unknown
- White box testing challenge: modern fuzzers to be customized to scan for specific bug classes instead of automation scripts that are static
- Automation results to be enough to reproduce a bug
- Support dev teams to automate their security testing
- Start manual?tests runs that combine multiple testing software tools and manage all results centrally
- Adopt fuzzing as a practice in security testing
- Automated and parallel cross-browser testing
- Understanding how much maintenance for automation is required can be tricky
- Maintenance may increase as a result of application redesign or third parties integrations
- Test engineers to increase their knowledge by producing a repository of different tests
- The tests are dependent on UI stability. If the application UI changes frequently then we would need to change / update our automation scripts accordingly to reduce flakiness
- Handling captcha is another challenge in automation testing. There are some third-party tools to automate captcha but still, it not be?achieved 100% results
- Handling pop up windows and it is?beyond selenium’s capabilities
- Handling page load.?Sometimes some elements appear depends upon the previous action
- Handling dynamic Elements. If an element’s id is changing on every page load then handling these type of elements is bit tricky in the normal way
