«Software Fault Reporting Processes in Business-Critical Systems Jon Arvid Børretzen Doctoral Thesis Submitted for the partial fulfilment of the ...»
In this study, we first have collected defect reports from several large and medium software systems, which include both reusable software components and non-reusable software components. Second, we will classify defects in the defect reports using ODC (Orthogonal Defect Classification) . Then, we will compare the defect density and severity of different defect types of the reusable components vs. those non-reusable components. We expect to figure out the types of defects that are not related to reuse (i.e., their presence is the same for both reusable and non-reusable components). In addition, we hope to find out some defects that may be more probable in reusable components than non-reusable components. Finally, we will show the results of our defect density analysis to project members that building the reusable components and to those building the non-reusable component. By discussing and interviewing these project members, we expect to find out why making a component reusable has helped or not helped to reduce certain defect types. This paper is structured as follows: Section 2 introduces related work; Section 3 presents our research motivation and research questions; Section 4 illustrates detailed research design; Section 5 shows the current available data that are going to analyze; Section 6 concludes the paper.
2. Related work Several industrial empirical studies shown in Table 1 conclude that reuse reduces the defect density, and therefore helps to improve the quality (especially the reliability) of the system. However, most studies focus on quantity of defects, such as the number of defects, or the defect density, without considering the defect type.
Some studies have investigated why reusable components have better quality than nonreusable components. The study of Succi et al.  concludes that implementing a systematic reuse policy, such as the adoption of a domain specific library, improves the customer satisfaction. Results from study of Selby  show that software modules without revisions had the fewest faults. However, the results of these studies only show the connection between the reuse policy and the number of defects.
3. Research motivation and research questions Our motivation here is to investigate the issues related to defects, namely classification and severity, in relation to software reuse. In another word, we are interested in the connection between the reuse policy and different kinds of defects. We want to know whether system reuse will help to reduce all kinds of defects or it is only helpful to minimize certain kinds of defects. We also suspect that reuse may increase certain types of defects. For example, we expect that familiarity with a component might prevent testing a component thoroughly. The results of this study will help industrial practitioners to better understand the benefits of reuse. It can also help them to improve their reuse policy in order to get a better quality system. Our research questions are as
• RQ1: What are the more common defect types in the reusable components vs. those non-reusable components?
• RQ2: What are the severities of defects for the reusable components vs. the nonreusable components?
• RQ3: What are the reasons of reduced defects in the reusable components?
4. Detailed research design When abnormalities in the operation of a system are found, these are reported as failures. These failures are reported to developers through failure reports. A fault is an underlying problem in a software system that leads to a failure. Error is used to denote the execution of a passive fault which leads to incorrect behavior (with respect to the requirements) or system state , and also for any fault or failure resulting from human activity . In our study, defect is used in place of fault, error or failure, without distinguishing the origin or whether it’s active or passive.
To answer the first research question RQ1, we plan to use the Orthogonal Defect Classification (ODC) scheme defined by IBM  to classify the defects into different types. The goal of IBM’s ODC scheme is to categorize defects such that each defect type is associated with a specific stage of development. ODC has been used o evaluate and improve technology. For example, in order to investigate the value of automatic static analysis, the defects found by the static analysis and not found by static analysis can be classified .
Reuse is proposed as a mechanism to improve the efficiency and quality of software development. It is therefore reasonable to use ODC to analyze the quality improvement due to reuse. The attribute defect type in ODC captures the fix that was made to resolve the defect. For example, defects of type function are those that require a formal design change. Examples of the defect types are given in Table 2. Details of other defect types are in .
To answer RQ1, we will compare the density and distribution of different defect types of reusable components with those non-reusable components.
To answer RQ2, we will compare the distribution of defect severities, which are usually defined by testers or developers, of the reusable components with that of the nonreusable components.
After getting the results of the RQ1 and RQ2, we will do a causal analysis by comparing the development processes, quality assurances, change management, application domain, and other contexts of the projects building the reusable components with those building non-reusable components to answer RQ3. The causal analysis will be done by interviewing project managers with supplemental documentation analysis. The purpose is to figure out why the reusable components have less/more defect density/severity of certain defect types than the non-reusable components.
5. The available data We currently have data from two software systems in company A and B (the company names are not shown in the paper due to confidential reasons). More data in several other companies will be collected in the near future.
5.1. Available data from the Company A The company A is a large, Norwegian company, in the Oil & Gas industry. The central IT-department of the company is responsible for developing and delivering software, which is meant to give key business areas better flexibility in their operation. It is also responsible for the operation and support of IT-systems. This department consists of approximately 100 developers, located mainly in Norway. Since 2003, a central IT strategy of the O&S (Oil Sales, Trading and Supply) business area has been to explore the potential benefits of reusing software systematically. Company A has developed a customized framework of reusable components, which is based on J2EE (Java 2 Enterprise Edition). The reusable components have been developed for three releases with in total 56 KLOC. There are several applications using the function of the reusable components. One application is the document storage application and includes several components. The components in this application are defined as non-reusable components in our study. The application has also three releases with total of 67 KLOC.
In company A, the defects are recorded by the Rational ClearQuest tool. Each trouble report contains an ID, headline description, severity (that indicates how critical the problem is evaluated by developers, such as critical, high, medium, or low), classification (Error, Error in other system, Duplicate, Rejected and Postponed), estimated time to fix, remaining time to fix, subsystem location, as well as an updated action and timestamp record for each new state the defect enters in the workflow. There are 223 trouble reports for the reusable framework and 438 trouble reports for the nonreusable application.
5.2. Available data from the Company B
Company B is a large Nordic company, working in the IT industry. They specialize in applications for workflow and process support both for public and corporate purposes, as well as doing consultancy work for their customers. The company employs around 500 people. The project studied from company B is a combined web presentation system and task management system used in administration of public information and application processing.
The defect reports stem from three releases of the project occupying 6-7 developers.
The project from company B is developed using a framework with reusable components and generated code. This project is also based on J2EE. Major parts of the reused code were automatically generated by a code generation tool, and the company did not report on number of lines of code. The defects have been reported in the Atlassian Jira Bug Tracking tool. The trouble reports contains an ID (Key), a short summary, type, status (Resolved/Closed/Open), severity evaluated by the developers (Blocker, Critical, Major, Normal, Minor, Trivial), resolution (Fixed, Cannot Reproduce, Won't fix etc), which person has been assigned responsibility (Assignee), who reported the defect (Reporter), time created, time updated, version defect was found, version the defect should be fixed, and the subsystem location (Components). There are 379 trouble reports from company B, of which 286 trouble reports come from the reused parts, and 93 trouble reports come from the non-reused parts.
6 Conclusion and future work In this position paper, we present the research design of an on-going empirical study to investigate the benefit or cost of software reuse on software quality. By analyzing the defect reports of several software systems, which include both reusable and nonreusable components, we expect to deepen the understanding on why reuse improves the quality of software. The conclusions of this study will be given as guidelines on improving the reuse process to companies involved this study. In order to generalize our conclusion, the future work is to collect data from project with different contexts, such as application domains, technologies, and development processes, in order to find the common good practices and lessons learned of software reuse.
7 References  G. Sindre, R. Conradi, and E. Karlsson, “The REBOOT Approach to Software Reuse”, Journal of System Software, 30(3): 201–212, 1995.
 W. C. Lim, “Effect of Reuse on Quality, Productivity and Economics”, IEEE Software, 11(5): 23-30, Sept./Oct. 1994.
 P. Mohagheghi, R. Conradi, O. M. Killi, H. Schwarz, “An Empirical Study of Software Reuse vs. Defect Density and Stability”, Proc. 26th Int’l Conference on Software Engineering (ICSE’2004), 23-28 May 2004, Edinburgh, Scotland, pp. 282-291, IEEE-CS Press.
 A. Gupta, O. P. N. Slyngstad, R. Conradi, P. Mohagheghi, H. Rønneberg, and E. Landre:
“An Empirical Study of Defect-Density and Change-Density and their Progress over Time in Statoil ASA”, Proc. 11th European Conference on Software Maintenance and Reengineering (CSMR’07), 21-23 March 2007, Amsterdam, The Netherlands, p. 10.
 W.M. Thomas, A. Delis and V.R. Basili, “An analysis of Errors in a Reuse-Oriented Development Environment”, Journal of Systems and Software, 38(3): 211-224, September 2004.
 G. Succi, L. Benedicenti, and T. Vernazza, “Analysis of the Effects of Software Reuse on Customer Satisfaction in an RPG Environment”, IEEE Transactions on Software Engineering, 27(5): 473-479, May 2001.
 W. Selby, “Enabling Reuse-Based Software Development of Large-Scale Systems”, IEEE Transactions on Software Engineering, 31(6): 495-510, June 2005.
 R. Chillarege, I. S. Bhandari, J. K. Chaar; M. J. Halliday, D. S. Moebus, B. K. Ray, M.-Y.
Wong, “Orthogonal Defect Classification - a Concept for in-Process Measurements, IEEE Transactions on Software Engineering, 18 (1): 943-956, Nov. 1992.
 IEEE Standard Glossary of Software Engineering Terminology, IEEE Standard 610.12, 1990.
 A. Endres and D. Rombach, “A Handbook of Software and Systems Engineering:
Empirical Observations, Laws and Theories”, Addison-Wesley, 2004.
 J. Zheng, L. Williams, N. Nagappan, W. Snipes, J. P. Hudepohl, M. A. Vouk, “On the Value of Static Analysis for Fault Detection in Software”, IEEE Transactions on Software Engineering, 32 (4): 240-253, April, 2006.
 ODC defect type: http://www.research.ibm.com/softeng/ODC/DETODC.HTM#type
Abstract: In most software development projects, faults are unintentionally injected in the software, and are later found through inspection, testing or field use and reported in order to be fixed later. The associated fault reports can have uses that go beyond just fixing discovered faults. This paper presents the findings from interviews performed with representatives involved in fault reporting and correcting processes in different software projects. The main topics of the interviews were fault management and fault reporting processes. The objective was to present practitioners’ view on fault reporting, and in particular fault classification, as well as to expand and deepen the knowledge gained from a previous study on the same projects. Through interviews and use of Grounded Theory we wanted to find the potential weaknesses in a current fault reporting process and elicit improvement areas and their motivation. The results show that fault management could and should include steps to improve product quality. The interviews also supported our quantitative findings in previous studies on the same development projects, where much rework through fault fixing need to be done after testing because areas of work in early stages of projects have been neglected.
Keywords: Process improvement, Fault management, Fault classification, Software quality.