FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:     | 1 |   ...   | 17 | 18 || 20 | 21 |   ...   | 27 |

«Software Fault Reporting Processes in Business-Critical Systems Jon Arvid Børretzen Doctoral Thesis Submitted for the partial fulfilment of the ...»

-- [ Page 19 ] --

5.1 Implications of the results The results found in this study coincide with the results of the previous fault study we performed with different development organizations. In both studies the “function” faults have been the most numerous, both in general and among the faults rated as most severe. As “function” faults are mainly associated with the design process phase, as stated by Chillarege et al. in [13] and also by Zheng et al. in [20] as shown in Table 6, this indicates that a large number of faults had their origin in early phases of development. This is a sign that the design and specification process is not working as well as it should, making it the source of faults that are demanding and expensive to fix, as “function” faults will generally involve larger fixing efforts than pure code errors like “checking” and “assignment” types of faults. This means that we can recommend the developers in the projects that have been studied to increase the effort used during design in order to reduce the total number of effort demanding faults in their products.

This finding is also similar to the one from the study of Vinter and Lauesen [18], where “Requirements and Features” faults were the dominating fault type.

Table 6. ODC fault types and development process phase associations [20]

–  –  –

When looking at each fault type in Figure 3, we see which fault types that tend to produce the most severe faults. One observation here is that although “function” faults dominate the picture for critical severity faults in Figure 2, it is the “relationship” and “timing/serialization” fault types that consist of the most critical severity rated faults.

It can therefore be argued that the fault types “relationship” and “timing/serialization” fault types are important to prevent, as it is likely that these types of faults have greater consequences than those of for instance “GUI” and “data” type faults. “Function” faults show themselves to be important to focus on preventing due to the sheer number of them, both in general and for the “critical” severity rated faults. Although “function” faults do not stand out as a fault type where most faults are rated as “critical”, it is still the biggest contributor to “critical” severity rated faults.

When informing the organization involved of the results of this study, the feedback was anecdotal confirmation of our findings, as they informed us that they were indeed having issues with design and specification, even though their own fault statistics showed most faults to be coding faults. We would like to study this issue further in our future work on the subject.

In many cases, fault reporting is performed with one goal in mind, to fix faults that are uncovered through inspection and testing. Once the fault has been corrected, the fault report information is not used again. The available information can be employed in a useful fashion as long as future development projects are similar to, or based on previous projects. By reusing the information that has been accumulated during fault discovery through testing and during production, we are able to learn about possible faults for new similar projects and further development of current projects.

Measuring quality and effects on quality in a software system is not a trivial matter. As presented in Section 2, the opinion on how and if this can be done is divided. One of the means Avizienis et al. suggests for attaining better dependability in a system is fault removal in order to reduce the number and severity of faults [17]. By identifying common fault types, developers can reduce a larger number of faults by focusing their efforts on preventing these types of faults. Also, identifying the most severe fault types makes developers able to focus on preventing those faults that have the biggest detrimental impact on the system.

5.2 Further issues concerning fault reporting in this organization

In addition to our quantitative study results, we were able to identify some points of possible improvement in the studied organization's fault reporting. Two attributes that we found lacking, which should be possible to include in fault reporting are Fault Location and Fault Fixing Effort. The location of a fault should be readily known once a fault report has been dealt with, as fault fixing must have a target module or software part. This information would be very helpful if the organization wants to investigate which software modules produce the most serious faults, and they can then make a reasoned argument if these modules are of a particularly critical type (like infrastructure or server components), or if some modules are simply of a poorer quality than others.

Including fault fixing effort into the fault reports is also an issue that could be of great benefit when working to improve fault prevention processes. By recording such information, we can see which fault types that produce the most expensive faults in terms of effort when fixing them. These are issues that will be presented to the organization under study. Their current process of testing and registering faults in a centralized way hinders the testers and developers from including this valuable information from the fault reports. The testers who initially produce the fault reports do not necessarily know which software modules the fault is located in, and developers fixing the fault do not communicate the location it was found in after it has been found and fixed.

5.3 Threats to validity

When performing an empirical study on industrial projects, it is not possible to control the environment or data collected as we would do in an experiment. The following is a short presentation of what we see as the main validity threats.

Internal validity. An issue here might be factors affecting the distribution of fault types. When the fault data was collected the intention of use was solely for fault fixing, it was not intended to be studied in this way. The coarse classification given by the developers could have been biased. Such bias or other inconsistencies were hopefully reduced by us classifying the fault reports with new fault types.

External validity. The small number of projects under investigation is a threat to external validity. However, the results of this study support the findings of a previous similar study of fault reports from other software development organizations. The projects under study may also not necessarily be the most typical, but this is hard to verify in any way.

Conclusion validity. One possible threat here is the reliability of measures, as the categorization of faults into fault types is a subjective task. To prevent categorizing faults we were unsure of into the wrong category, we used a type “unknown” to filter out the faults we were not able to confidently categorize.

6. Conclusion and future work

In this paper we have described the results of a study of fault reports from five software projects from a company developing business-critical software. The fault reports have been categorized and analyzed according to our research questions. From the research questions we have found that "function" faults, closely followed by "GUI" faults are the fault types that occur most frequently in the projects. To reduce the number of faults introduced in the systems, the organization should focus on improving the processes which are most likely to contribute to these types of faults, namely the specification and design phases of development. Faults of the fault types "documentation", "relationship", "timing/serialization" and "interface" are the least frequent occurring fault types.

The fault types that are most often rated as most severe are "relationship" and "timing/serialization" faults, while the fault types "GUI" and "documentation" are considered the least severe. Although “function” faults are not rated as the most severe type of fault, this fault type still dominates when looking at the distribution of highly severe faults only.

In additions to these results, we observed that the organization’s fault reporting process could be improved by adding some information to the fault reports. This would facilitate more effective targeting of fault types and locations in order to better focus future efforts for improvement.

In terms of future work, we want to continue studying the projects explored in this paper, using qualitative methods to further explain our quantitative results. Feedback from the developers’ organization would aid us understand the source of these results, and help us suggest concrete measures for process improvement in the organization.

Acknowledgements The authors would like to thank Reidar Conradi for careful reviewing and valuable input. We also thank the organization involved for their participation and cooperation during the study.


1. R. Grady, Practical Software Metrics for Project Management and Process Improvement, Prentice Hall, 1992

2. J. A. Børretzen; T. Stålhane; T. Lauritsen; P. T. Myhrer, “Safety activities during early software project phases”. Proceedings, Norwegian Informatics Conference, 2004

3. J. A. Børretzen; R. Conradi, ”Results and Experiences From an Empirical Study of Fault Reports in Industrial Projects”. Proc. 7th International Conference on Product Focused Software Process Improvement (PROFES'2006), 12-14 June 2006, Amsterdam, Pages: 389P. Mohagheghi; R. Conradi; J.A. Børretzen, "Revisiting the Problem of Using Problem Reports for Quality Assessment", Proc. the 4th Workshop on Software Quality, held at ICSE'06, 21 May 2006, Shanghai, Pages: 45-50

5. ISO, ISO/IEC 9126 - Information technology - Software evaluation – Quality characteristics and guide-lines for their use, ISO, December 1991

6. J.-C. Laprie, “Dependable computing and fault tolerance: Concepts and terminology”, Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995, ' Highlights from Twenty-Five Years', June 27-30, 1995

7. B. Littlewood; L. Strigini, “Software reliability and dependability: a roadmap”, Proceedings

of the Conference on The Future of Software Engineering, Limerick, Ireland, 2000, Pages:

175 - 188

8. N. Leveson, Safeware: System safety and computers, Addison-Wesley, Boston, 1995

9. IEEE, IEEE Standard Classification for Software Anomalies, IEEE Std 1044-1993, December 2, 1993

10. K.A. Bassin; T. Kratschmer; P. Santhanam, “Evaluating software development objectively”, IEEE Software, 15(6): 66-74, Nov.-Dec. 1998

11. K. Bassin; P. Santhanam, “Managing the maintenance of ported, outsourced, and legacy software via orthogonal defect classification”, Proceedings. IEEE International Conference on Software Maintenance, 2001, 7-9 Nov. 2001

12. K. El Emam; I. Wieczorek, “The repeatability of code defect classifications”, Proceedings.

The Ninth International Symposium on Software Reliability Engineering, 1998, 4-7 Nov.

1998 Page(s):322 – 333

13. R. Chillarege; I.S. Bhandari; J.K. Chaar; M.J. Halliday; D.S. Moebus; B.K. Ray; M.-Y.

Wong, “Orthogonal defect classification-a concept for in-process measurements”, IEEE Transactions on Software Engineering, Volume 18, Issue 11, Nov. 1992 Page(s):943 - 956

14. R.R. Lutz; I.C. Mikulski, “Empirical analysis of safety-critical anomalies during operations”, IEEE Transactions on Software Engineering, 30(3):172-180, March 2004

15. R.A. Paul; F. Bastani; I-Ling Yen; V.U.B. Challagulla, “Defect-based reliability analysis for mission-critical software”, The 24th Annual International Computer Software and Applications Conference, 2000. COMPSAC 2000. 25-27 Oct. 2000 Page(s):439 - 444

16. D. Hamlet, “What is software reliability?”, Proceedings of the Ninth Annual Conference on Computer Assurance, 1994. COMPASS '94 'Safety, Reliability, Fault Tolerance, Concurrency and Real Time, Security', 27 June-1 July 1994 Page(s):169 – 170

17. A. Avizienis; J.-C. Laprie; B. Randell; and C. Landwehr, “Basic Concepts and Taxonomy of Dependable and Secure Computing”, IEEE Transactions on Dependable and Secure Computing, vol. 1, no. 1, January-March 2004

18. O. Vinter; S. Lauesen, “Analyzing Requirements Bugs”, Software Testing & Quality Engineering Magazine, Vol. 2-6, Nov/Dec 2000.

19. B. Beizer, Software Testing Techniques. Second Edition, Van Nostrand Reinhold, New York, 1990

20. J. Zheng; L. Williams; N. Nagappan; W. Snipes; J.P. Hudepohl; M.A. Vouk, “On the value of static analysis for fault detection in software”, IEEE Transactions on Software Engineering, Volume 32, Issue 4, April 2006 Page(s):240 – 253

–  –  –


The benefits of reusing software components have been studied for many years. Several previous studies have concluded that reused components have fewer defects in general than non-reusable components. However, few of these studies have gone a further step, i.e., investigating which type of defects has been reduced because of reuse. Thus, it is suspected that making a software component reusable will automatically improve its quality. This paper presents an on-going industrial empirical study on the quality benefits of reuse. We are going to compare the defects types, which are classified by ODC (Orthogonal Defect Classification), of the reusable component vs. the nonreusable components in several large and medium software systems. The intention is to figure out which defects have been reduced because of reuse and the reasons of the reduction.

1. Introduction Software reuse is a management strategy for software evolution, in terms of development for and with reuse. Development for reuse refers to the generalization of components towards reuse, while development with reuse has to do with the inclusion of these reusable components in new and future development [1]. Understanding the issues related to software reuse, involving its purpose and promises, has been a focus since the 1970s. The focuses have been on how to develop for/with reuse, technical/managerial/organizational aspects, measuring reuse in terms of quality and productivity, as well as reporting success and failures of reuse practices. Although some studies have found that reusable software components have less defect density than nonreusable components [2]-[7], few studies have studied why software defects have been reduced because of reuse.

Pages:     | 1 |   ...   | 17 | 18 || 20 | 21 |   ...   | 27 |

Similar works:

«Moving from Internal to External Services using Aspects Martin Henkel, Gustav Boström and Jaana Wäyrynen Department of Computer and Systems Science Stockholm University and Royal Institute of Technology Forum 100, SE-164 40 Kista, Sweden martinh@dsv.su.se, gusbo@kth.se, jaana@dsv.su.se Summary. Service oriented computing and web service technologies provide the means to structure an organisation’s internal IT resources into a highly integrated network of services. In e-business and business...»

«SEMIOTEXT(E) INTERVENTION SERIES © 2011 by Edizioni Casagrande, Bellinzona, Switzerland. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior permission of the publisher. Published by Semiotext(e) 2007 Wilshire Blvd., Suite 427, Los Angeles, CA 90057 www.semiotexte.com Thanks to Erik Morse. The translator wishes to thank Susanna Proietti for her...»

«Section XII Housing Assisted Living Demonstration Project Description: Assisted living is designed for people who want to live in a community setting and who need help with activities of daily living (ADLs), but who do not need nursing home care. It combines housing, supportive services, personalized assistance and health care to help individuals with daily activities. The Assisted Living Demonstration Project provides subsidized assisted living to persons who reside in specific affordable...»

«The Economic Club of New York The Honorable Agustin Carstens Governor, Bank of Mexico March 28, 2014 The New York Economic Club–The Honorable Agustin Carstens–March 28, 2014 Page 1 Introduction Roger Ferguson: Welcome and good morning to everyone. Good morning. I’m Roger Ferguson. I am the chair of the Economic Club of New York and I’m also the president and CEO of TIAA-CREF, a Fortune 100 financial services company. As is the practice at our breakfasts we will hear first from our...»

«S3 The Port of Dublin By A. General Manager, Dublin Port and Docks DENIS HEGARTY, Board. [Read before the Society on January 1th, 1955.) Introduction. My purpose this evening is to tell you something about the Port of Dublin. I shall endeavour to do this in three stages, first I shall touch briefly on its history, then I shall describe the services which are required of a large modern port, and I shall explain to you how the port is organised, controlled, operated and financed, and describe the...»

«CURRICULUM VITAE OF ROLAND CLAIRMONTE CRAIGWELL PERSONAL DETAILS HOME ADDRESS: Lot 15, Western Avenue Fort George Heights Upton, St. Michael BB 11102 Barbados, W.I. Tel: (246) 435-7010 (H)/417-4474 (W) Fax: (246) 435-7010 (H)/438-9104 (W) E-Mail: craigwell@caribsurf.com (H) roland.craigwell@cavehill.uwi.edu (W) WEB PAGE: http://www.cestagi.com/?Roland_Clairmonte_Craigwell0 DATE OF BIRTH: March 10, 1964 NATIONALITY: Barbadian MARITAL STATUS: Married (2 children) UNIVERSITY EDUCATION Ph.D....»

«What has the impact of the February Christchurch Earthquake been on fundraising, donor behaviour and charity decision-making? 12th May 2011 P O Box 1810 Palmerston North 493 Parnell Road Parnell, Auckland 0800 000 955 www.xponential.co.nz 1 On Tuesday 22 February 2011 at 12:51pm Christchurch City was rocked by a 6.3 magnitude earthquake. The devastation and loss of life has been catastrophic. The New Zealand public, business community, Government and those overseas with a special connection to...»

«WP/10/134 On the Optimal Adherence to Money Targets in a New-Keynesian Framework: An Application to Low-Income Countries Andrew Berg, Rafael Portillo, and D. Filiz Unsal © 2010 International Monetary Fund WP/10/134 IMF Working Paper Research Department On the Optimal Adherence to Money Targets in a New -Keynesian Framework: An Application to Low-Income Countries Prepared by Andrew Berg, Rafael Portillo, and D. Filiz Unsal June 2010 Abstract Many low-income countries continue to describe...»

«Commercial Law Developments 2015 Prepared by Professor Stephen L. Sepinuck updated 1/6/16 2015 Commercial Law Developments Page 1 SECURED TRANSACTIONS Scope Issues In re Montreal, Maine & Atlantic Railway, Ltd., 1. 799 F.3d 1 (1st Cir. 2015) A creditor’s security interest in the debtor’s accounts and payment intangibles did not extend to the debtor’s right to payment under its business interruption insurance policy because Article 9 does not apply to an interest in or a claim under an...»

«FROM POLITICAL INDEPENDENCE TO ECONOMIC EMANCIPATION: RESEEDING AFRICA WITH ENTREPRENEURSHIP AS A STRATEGY OF ECONOMIC DEVELOPMENT Joseph P. Eshun, Jr East Stroudsburg University Department of Business Management College of Business Management East Stroudsburg University 200 Prospect Street East Stroudsburg University, PA 18301 Phone: 570-422-3404 Fax: 570-422-2791 E-mail: jeshun@po-box.esu.edu ABSTRACT The resurgence of interest in Africa in recent decades has sparked an intense intellectual...»

«STATE OF INDIANA DEPARTMENT OF LOCAL GOVERNMENT FINANCE INDIANA GOVERNMENT CENTER NORTH 100 NORTH SENATE AVENUE N1058 INDIANAPOLIS, IN 46204 PHONE (317) 232-3773 FAX (317) 232-8779 TO: Solid Waste Districts FROM: Department of Local Government Finance SUBJECT: Budget Procedures DATE: June 2004 The following memorandum provides guidance on the 2005 Budget Process. Please read the memorandum in its entirety. DEPARTMENT OF LOCAL GOVERNMENT FINANCE JULY WORKSHOPS AND FALL BUDGET HEARINGS During...»

«The Center for Law and Economic Studies Columbia University School of Law 435 West 116th Street New York, NY 10027-7201 (212) 854-3739 Governing Interdependent Financial Systems: Lessons from the Vienna Initiative Katharina Pistor Working Paper No. 396 March 21, 2011 Do not quote or cite without author’s permission An index to the working papers in the Columbia Law School Working Paper Series is located at http://www.law.columbia.edu/lawec/ 1 Electronic copy available at:...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.