Bugs per lines of code

Books that reports or web site that addresses the topic of bugs per line of code The book "Code Complete" by Steve McConnell has a brief section about error expectations. He basically says that the range of possibilities can be as follows 
(a) Industry Average: "about 15 - 50 errors per 1000 lines of delivered code." He further says this is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques. 

(b) Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC IS CALLED AS 1000 lines of code) in released product (Moore 1992)." He attributes this to a combination of code-reading techniques and independent testing (discussed further in another chapter of his book). 

(c) "Harlan Mills pioneered 'cleanroom development', a technique that has been able to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990). A few projects - for example, the space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing."

Vinnie Murdico Software with Brains, Inc. 
SWBTracker - Value-Priced Defect Management Software 
http://www.softwarewithbrains.com

9 comments:

Anonymous said...

Any chance you have also collected metrics on the Cost to achieve a specific kloc level?

The reasons this matters is that it relates to the number of Security Defects in code and the cost to control such events.

Knacho said...

In the real world of in-house testing, you need to make your own expected "bugs by code line" number. Whit this you can define SLA´s for the developers or factories you use.

Explore said...

i dont have any such metrics, but when the build is stable in functionality its better to initiate for security testing in parallel to the development. That can actually save time in terms of cost rather hen waiting completely for a finished product

pete keys said...

This is very interesting. The number of defects found is going to change through the stages, unit test, system integration test, acceptance test. Are there any metrics on the reduction through phases?
Also the number of defects is proportional to the test coverage /thoroughness. Are there any metrics on the number of tests per KLOC.
I am working on the first iteration of a new product with 430KLOC and expecting a defect density of less than 1. At more than 0.5 I would be disappointed and my exit criteria is 0.01: t
he defects rates quoted in this article seem very high. Perhaps the severity of a defect needs taking into account: a defect that causes a crash and data loss is different to a menu entry typO.

pete keys said...

This is very interesting. The number of defects found is going to change through the stages, unit test, system integration test, acceptance test. Are there any metrics on the reduction through phases?
Also the number of defects is proportional to the test coverage /thoroughness. Are there any metrics on the number of tests per KLOC.
I am working on the first iteration of a new product with 430KLOC and expecting a defect density of less than 1. At more than 0.5 I would be disappointed and my exit criteria is 0.01: t
he defects rates quoted in this article seem very high. Perhaps the severity of a defect needs taking into account: a defect that causes a crash and data loss is different to a menu entry typO.

Peter DeLorme said...

I, too, am looking for more depth in the analysis. For example, the defects per lines of code tested this month, so that I can trend the number and see that quality is improving toward benchmarks at say the mid-point and at the end of each test phase. Do you have any hints on where to find such metrics for .Net development?

Unknown said...

A computer bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from working correctly or produces an incorrect result. Bugs arise from mistakes and errors, made by people, in either a program’s source code or its design.
Defect Bug Tracking

Karthika Shree said...

Excellent and very cool idea and the subject at the top of magnificence and I am happy to this post..Interesting post! Thanks for writing it.What's wrong with this kind of post exactly? It follows your previous guideline for post length as well as clarity
Java Training in Chennai

Trending Technology said...

Thanks for sharing the blog. It's very helpful and informative. Thank you.

Azure Training in Chennai | Azure Certification Training in Chennai