Software Test Metrics With Continuous Testing
Continuous Integration and Continuous Delivery in an Agile driven project environment is meaningless without Continuous Testing. The ability to measure progress of a continuous testing process at each stage of the application delivery life cycle is vital to launch quality software with minimal business risks.
Below listed are 25 key metrics to monitor in a Continuous Testing project.
Unit testing
Unit testing is a process in which the smallest independent unit of a software application is tested for errors. It is usually an automated process although it can be done manually.
The following metrics are tracked during the unit testing:
1. Code coverage %
2. Cyclomatic complexity
E = Number of edges N = Number of nodes
3. Test Pass %
4. Build Quality
Developers perform Smoke testing before releasing the build for testing. Testers perform smoke testing to decide accepting the build for further testing.
Smoke Testing
Also known as build verification testing, smoke testing is used to ensure the most important functions. The results of smoke testing determines whether the build is stable enough to proceed with further testing. The term "smoke testing" originated from a type of hardware in which a device is said to pass the test if it didn't catch fire the very first time it was switched on.
Regression Testing
Often times when a code is modified, the new code adversely affects the old code. This is where regression testing comes in handy. Regression testing can be used to ensure that the new code does not have any side effects on the old code.
Below are the metrics tracked during regression testing process:
5. Test execution coverage %
6. Requirements coverage %
7. % defects by priority (Low, medium & high)
8. Defect rate % (No. of defects identified during testing)
9. Defect severity (type of fault)
Defect severity classification
10. Test duration
11. Test Execution Status
12. Defect Age
13. Mean Time to Detect (MTTD)
14. RTF Ratio
Exploratory Testing
As the name suggests, exploratory testing is about finding the capabilities of the software i.e. what is can and can't do. Unlike scripted testing which relies on test scenarios for guidance, in exploratory testing the software is tested without any plans.
The following are some of the metrics used in exploratory testing:
Pre-Production Deployment
In pre-production environment the software is subjected to rigorous testing to identify bug in the software prior to deployment.
The following metrics are tracked during the pre-production deployment:
20. Defect rejection ratio
21. Critical defects %
Load/Performance Testing
Load testing refers to increasing the demand on the software and measuring its response. This is done to determine how the system will react under peak load conditions. This in turn will help to determine the maximum capacity the system can handle before crashing.
Following are some of the metrics tracked during the Load/performance testing:
22. Latency time
Response time – Processing time
23. Throughput time
24. 90th percentile
n * 0.90
Sanity Testing
Once the software build has been tested and the issues have been resolved, sanity testing is done to check whether the bugs have been fixed and changes in the code haven't affected the old code i.e. regression is introduced. Therefore, it is a subset of regression testing. Sanity testing aims to check whether the predetermined functionality is working properly. The build is rejected upon failure.
The following is a metric tracked during the sanity testing process:
25. Defect Leakage
If you can't measure it, you cannot improve it. As the quote goes, metrics are vital for continued success in any project. Especially in Software Testing, where there are so many changes causing factors and interdependencies, metrics are important.
We believe that this check list is helpful for Software Testers. Whenever we come across additional metrics, we will keep this blog post updated.
Download Infographic
Source: https://www.softcrylic.com/blogs/top-25-metrics-measure-continuous-testing-process/
0 Response to "Software Test Metrics With Continuous Testing"
Post a Comment