Wednesday, September 17, 2014

Why ISO 29119 is bad for Software Testing?


ISO standard is a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose. We need them because they:
  • Act as repository of collective experience
  • Acts as shared knowledge base, understanding & definitions
  • Connect the world
  • Protect Suppliers & Customers from unfair competition

What is ISO 29119?
ISO/IEC/IEEE 29119 Software Testing is an internationally agreed set of standards for software testing that can be used within any software development life cycle or organization.







Why is ISO29119 bad for Software Testing?
To start with, I believe that ISO 29119 is irrelevant to all the testers. It is an over structured process model focused on documentation and processes with negligible content on actual testing, broadly, affecting three major areas:

  • Knowledge & Quality 
  • People & Economy 
  • Time & Efort

Knowledge & Quality
We all know that “knowledge” rules “Quality tests” of any software under test. Also, to point out, knowledge is dependent on the application & product under test. Knowledge required by a Banking tester, Medical device tester, or an ERP application tester is quite distinct and cannot be covered by any generic testing certification. Hands-On experience by a tester on the product can help in far better ways than to just go through set of guidelines/processes/certifications. If this standard is accepted, companies will start believing that testers without certification are not good which is not true at all. This will make companies dependent on these factors to access a tester’s knowledge & capability which is not the ONLY criteria. Companies that would otherwise have ignored the standard will feel compelled to comply in order to win business. If the contract requires compliance, then the whole development process could be shaped by a damaging testing standard. ISO 29119 could affect anyone involved in software development, and not just testers.
Additionally, things like “intuition” & “experience-based testing” will lose their meaning in the testing world by following set of standards making the whole exercise complex and time consuming.

People & Economy
Set of global/centralized standards should ideally reflect practices & consensus of all other related and established communities which is not the case here. There is a strong divergence within the academic community as well as within the practitioner community. Context-Driven School of thought, for example has been excluded/ignored- for their excellence in Context-driven tests and their inputs were not taken into consideration. Moreover, ISO standards are supposed to respond to a need in the market which is again not the case here.
On the economic front, few attributes of valid ISO standards that ISO 29119 violates are that an ISO standard is not supposed to:

  • Distort the market
  • Have adverse effects on fair competition

It is as good as saying that you have to have an ISO 29119 certification to work with regulated software or to get a government job/contract as a tester. This will also encourage the setup of unnecessary market for training, certification & consultancy in interpreting & applying the standard. [Rent-Seeking]


Time & Effort
Following ISO29119 standard is an expensive, time consuming & wasteful focus on exaggerated process models & excessive documentation. Any kind of work presents opportunity cost; when you’re doing something, most of the time it prevents you from doing something else. Every minute that a tester spends on wasteful documentation is a minute that the tester cannot fulfill the comprehensive mission of testing: learning about the product, with an emphasis on discovering important problems that threaten value or safety, so that our clients can make informed decisions about problems and risks.
It clearly ignores the development in Testing & Management thinking.
I am not objecting here to documentation, I am objecting to excessive and wasteful documentation. Ironically, the standard itself provides an example: the current version of ISO 29119-1 runs to 64 pages; 29119-2 has 68 pages; and 29119-3 has 138 pages.