How Much Is Your Offshore/Outsource Testing Defect Find/Fix Cycle Costing You?

Update:  Linda G. Hayes, who is the CTO of Worksoft, Inc., and the founder of three software companies including AutoTester, the first PC-based test automation tool, wrote an interesting article on this subject at StickyMinds.com.  She wrote about the "promise of the same value proposition" and outlined some key factors for success or failure with offshoring.  One of the more interesting, and relevant comments (at least as it pertains to this article), was on "Time to Value".  Linda said:

Offshore resources are never a quick fix, especially for testing. A 2005 report from AMR Research found that it took between fourteen months and three years before the offshore testers had sufficient familiarity to be effective in finding the root cause of problems. Others have found that the lack of domain knowledge resulted in spurious defects being reported that actually increased development overhead due to review and response times. In fact, one organization identified that as many as 33 percent of reported issues were traceable to tester error.

There are many reasons to involve external testing services. Sometimes companies need the expertise of an interoperability lab, other times they are attempting to reduce testing staff costs. Whatever the reason, one cost that should be calculated based on a known process is the Find/Fix Cycle Cost for defects. It's important to evaluate these costs not just based on the hourly rate of the offshore/outsource testing service provider, but on the total cost involved in finding and fixing defects. Ultimately, this cost involves all QA and developer resource costs.



The find/fix cycle represents the time required to find and fix one defect and you should know what that cost is to your company. In order to calculate this cost, you need to collect information along the way. I found a nice source for this calculation, as well as a number of other interesting testing topics, at Software Quality Consulting, Inc. Food For Thought. Steve Rakitin does a very nice job in describing this cost in the table below.

Estimate the find/fix cycle Cost
T1 is time required to run a test that finds a potential defect	T1
T2 is time required to report a defect	T2
T3 is time required to determine that this is a defect	T3
T4 is time required to determine if the defect should be fixed	T4
T5 is time required to fix the defect	T5
T6 is time required to include fix in a build	T6
T7 is time required to run regression tests to determine if defect is really fixed and close out defect report	T7
Total time required…	TT = (T1 + … + T7)
LC represents fully loaded labor cost ($/hour) for engineering time (development and QA)	LC
C represents cost to find and fix one defect	C = LC * TT
TC represents the total cost to find and fix the defects found and fixed in the last release (F1)	TC = C * F1

As you can see from this table, the "round trip" cost of a defect incorporates 7 different time elements that add up pretty quickly, especially if you have any inherent time delays such as dealing with remote test teams, external test staff turnovers, re-requests for defect characterization information, etc. And the final cost is not just that of the test team, but also incorporates the dev team costs.


My guess would be that T1 and T5 would be the largest time elements, while times T3 and T4, which require more verbal communication, could very well match T1 and T5, depending on the teams involved. It would be interesting, and most likely revealing, to measure these times and factor them into the costs associated with each project's QA and dev team composition.