How to Achieve Fast Paced Product Delivery by Automation
How to Achieve Fast Paced Product Delivery?
- Agile, Automation, DevOps & CI/CD
How to select right Automation tool? Questions you should ask
Why Automation? To Build High Quality Product in Less Time, Cost & Faster Release
- 25-35% of budget is spend on testing alone
- Automation is redefining the way engineers design and perform testing operations
- It helps in Enhance Efficiency, Effectiveness, Productivity and Reduce Testing cycle
- Ease Manual Effort, Save Time, Reduce Cost, to do Repetitive, to avoid Human error
- Test Execution, Test Verification & Regression Testing
- Improved Test Coverage & Quality, Fast Testing on multiple build/ environments
- Gives Confidence in Market/Sales Product, Edge in Competitive Market with Competitors, Allows Faster Product Release
- Used in different types of testing's
- Continuous Testing in CI/CD
- Test Reporting
Criteria for Automation Tool
Right Process, Tools, Technically Skilled Resource required for product testing need & challenges
- Scope, Compare, Analyze, Shortlist, Evaluate, PUGH Matrix
Analyze Automation Tool: Scope, Tool, Matrix
1) Scope of the project, Understand project requirement for automation testing
- Testing types: API, Unit, Functional, Regression, System
- Platform Supported: Desktop, Web, Mobile, Browser, OS
- Technology stack, Technical skill, & Expertise of Existing team (knowledge level on Programming)
2) Consider your existing tool as benchmark to evaluate with list of selected tools
- Analyze your current tool for stability, features, integration and adaptiveness
- Understand the Pros & Cons (UI features, Customer Support, Documentation, Browser Compatibility)
- One or many Automation tool used for all/different types of testing both functional and nonfunctional
(load, performance, security, penetration, accessibility)
- Compare Tools against the current tool for Pros & Cons and list the selected tools
- Shortlisted tools need to evaluate more on key criteria and other features
- Complexity of the tools, Ease of use, OS compatibility, Platform Support, Multiple Language, License, Test report
- Better scalability and access to other resources, extendable to meet the increasing demands
- System requirements, Integration with 3rd party
- License Cost & Budget
- Time and Deadlines
- Maintenance, Scale and Leverage
- Different Reports, Dashboard, Exports, Graphs & Charts
- Support
3) Matrix technique for analysis
- PUGH Matrix is a qualitative technique used to rank the multi-dimensional options of an option set
Measuring Automation? Metrics Measurement
Strategic Measurement - to align quality efforts to product objectives
DORA - DevOps Research and Assessment Metrics
Deployment frequency - is testing slows deployment, time needed to test, time spend on repetitive tasks
Lead Time for Code Changes - is testing cycle efficient, time needed to create and maintain tests
Change failure rates - team credibility, quality of work, no. of bugs in production
Time to Restore Service - structure test to easily identify failures, test and monitor integrations to prevent outages, efficient failover testing
Operational Metrics - to improve incrementally
Dashboard for tracking and views over time
Easy to check in and view the progress
Increase in automation tests
reduce in manual regression testing hour/efforts
Increased test coverage
Reduce Production Bugs
Increase Production Deployment
Testing Cycle Reduced
Other measurements
Reduce testing delays
Improve test coverage
Collaborate CI/CD
Improve product release frequency
Target product deployment
Automation regression ensure the existing product quality when there is a code change
Bugs can find earlier before production deploy
Test accurately and consistent
Repeatable process can be automated
Reduce production defects
Changes can be easily tested and implemented
Test on multiple platform and environments
Quantify time needed to test
Time spend on repetitive task get reduced
Measure the efficiency of testing cycle
Track time needed to create and maintain tests