Development testing is a software development process that involves synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs.
Depending on the organization's expectations for software development, development testing might include static code analysis, data flow analysis, metrics analysis, peer code reviews, unit testing, code coverage analysis, traceability, and other software verification practices.
Overview
editDevelopment testing is performed by the software developer or engineer during the construction phase of the software development lifecycle.[1]
Rather than replace traditional QA focuses, it augments it.[2] Development testing aims to eliminate construction errors before code is promoted to QA; this strategy is intended to increase the quality of the resulting software as well as the efficiency of the overall development and QA process.[3]
Purposes and benefits
editDevelopment testing is applied for the following main purposes:
- Quality assurance—To improve the overall development and test process by building quality and security into the software (rather than trying to test defects/vulnerabilities out).
- Industry or Regulatory Compliance—To achieve compliance with industry or regulatory compliance initiatives (e.g.., FDA, IEC 62304, DO-178B, DO-178C, ISO 26262, IEC 61508, etc.) that commonly require strict risk reduction as well as bidirectional requirements traceability (e.g., between requirements, tests, code reviews, source code, defects, tasks, etc.)[3]
VDC research reports that the standardized implementation of development testing processes within an overarching standardized process not only improves software quality (by aligning development activities with proven best practices) but also increases project predictability.[4] voke research reports that development testing makes software more predictable, traceable, visible, and transparent throughout the software development lifecycle.[2]
Key principles
editIn each of the above applications, development testing starts by defining policies that express the organization's expectations for reliability, security, performance, and regulatory compliance. Then, after the team is trained on these policies, development testing practices are implemented to align software development activities with these policies.[5] These development testing practices include:
- Practices that prevent as many defects as possible through a Deming-inspired approach that promotes reducing the opportunity for error via root cause analysis.
- Practices that expose defects immediately after they are introduced—when finding and fixing defects is fastest, easiest, and cheapest.[3][6]
The emphasis on applying a broad spectrum of defect prevention and defect detection practices is based on the premise that different development testing techniques are tuned to expose different types of defects at different points in the software development lifecycle, so applying multiple techniques in concert decreases the risk of defects slipping through the cracks.[3] The importance of applying broad set of practices is confirmed by Boehm and Basili in the often-referenced "Software Defect Reduction Top 10 List."[7]
Static analysis
editThe term "development testing" has occasionally been used to describe the application of static analysis tools. Numerous industry leaders have taken issue with this conflation because static analysis is not technically testing; even static analysis that "covers" every line of code is incapable of validating that the code does what it is supposed to do—or of exposing certain types of defects or security vulnerabilities that manifest themselves only as software is dynamically executed. Although many warn that static analysis alone should not be considered a silver bullet or panacea, most industry experts agree that static analysis is a proven method for eliminating many security, reliability, and performance defects. In other words, while static analysis is not the same as development testing, it is commonly considered a component of development testing.[8][9]
Additional activities
editIn addition to various implementations of static analysis, such as flow analysis, and unit testing, development testing also includes peer code review as a primary quality activity. Code review is widely considered one of the most effective defect detection and prevention methods in software development.[10]
See also
editReferences
edit- ^ McConnell, Steve (2004). Code Complete (2nd ed.). Microsoft Press. ISBN 0-7356-1967-0.
- ^ a b voke Market Mover Array Report: Testing Platforms by Theresa Lanowitz, Lisa Dronzek, voke, June 05, 2012
- ^ a b c d Kolawa, Adam; Huizinga, Dorota (2007). Automated Defect Prevention: Best Practices in Software Management. Wiley-IEEE Computer Society Press. ISBN 0-470-04212-5.
- ^ "Automated Defect Prevention for Embedded Software Quality" white paper by VDC Research
- ^ Great expectations for development—with policy automation by Wayne Ariola, SD Times, July 28, 2011
- ^ Rethinking Software Development, Testing and Inspection Archived 2013-05-07 at the Wayback Machine by Matthew Heusser , CIO, February 1, 2012
- ^ Software Defect Reduction Top 10 List by Barry Boehm and Victor R. Basili, Computer, January 2001
- ^ Static Analyzers in Software Engineering Archived 2012-10-15 at the Wayback Machine by Dr. Paul E. Black , CrossTalk: The Journal of Defense Software Engineering, March/April 2009
- ^ Top 3 Mistakes with Static Analysis for Embedded and Safety-Critical Development by Arthur Hicken, EE Catalog, September 25, 2012
- ^ Satisfying SIL Requirements: Ensuring Functional Safety of E/E/PE Safety-Related Systems Archived 2016-03-04 at the Wayback Machine article on DevelopmentTesting.com