Tips for Measuring Software Quality Assurance Progress

Last month, we discussed how to prepare for the software quality assurance phase, you can see that newsletter at http://www.pragmaticsw.com/newsletters/newsletter_2007_01.htm.

This month we will focus on key measurements to capture along the way, to ensure your QA effort is progressing as planned.

Tip 1 - Review Test Case Trending
To determine if your QA process is progressing, you must track how many test cases are awaiting run, how many have passed and how many have failed.  Below is a trend graph that can be produced using a spreadsheet or your test case management tool.  Notice that it is trending as we would hope:

If your test case management tool does not provide the graphs above, here is a spreadsheet you can use: http://www.pragmaticsw.com/newsletters/trending_testcases.xls.

Tip 2 - Review Defect Trending
To determine release quality, you need to determine how your defects have been trending over the past 2 weeks.  This alerts your team as to production readiness. Notice that it is trending as we would hope:

• The closed defects increased over time.  After 2 weeks of testing, we had only 8 defects awaiting work and we had 10 that were ready for QA.
• After the 2 week cycle, we fixed and closed 278 defects, and impressive feat for 2 weeks!

If your defect tracking tool does not provide the graph above, here is a spreadsheet you can use: http://www.pragmaticsw.com/newsletters/trending_defects.xls.

Conclusion
As you can see, graphs provide key indicators that allow your team to determine quality progression.  If things are not trending as desired, these items will quickly stand out on the graphs, and you can meet with your team to reprioritize their efforts and to add resources as needed.  The graphs above are simple to use and setup.  Once this is mastered, you can graph other things (like test cases and defects by severity, assignee, etc.) to aid in making decisions along the way.