FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:     | 1 |   ...   | 23 | 24 || 26 | 27 |

«Software Fault Reporting Processes in Business-Critical Systems Jon Arvid Børretzen Doctoral Thesis Submitted for the partial fulfilment of the ...»

-- [ Page 25 ] --

Figure 5. Distribution of fault reports in the two DAIM fault report collections (%).

4.4 Comparing hazards and faults The method of employing hazard analysis and analysis of fault reports that is used here, could be described as triangulation of techniques to show how hazard analysis can be used to identify possible faults in software.

Combining the distributions from Figures 3 and 4 in one graph, we get Figure 6, which shows that the distribution of hazards and faults to be quite different to each other.

Hazards (%) Faults (%) 35,0 30,0 25,0 20,0 15,0 10,0 5,0

–  –  –

With only 6 of the 92 faults reported also being identified by the PHA analysis, there are many faults that was not identified by the PHA. With many of the faults reported being pure coding faults, like the faults of the “assignment” fault type, this was to be expected. A typical example of a fault report description in this fault type was “ThesisID missing for document”.

4.5 Efficiency of PHA for fault identification Looking at staff-hours spent per fault identified, we get two figures, one for the faults that were actually found in system testing and field use, and one for the total number of potential faults in the system (including the actual faults). Table 7 shows these figures, and compares them with some mean numbers on inspection efficiency from [7].

–  –  –

This result is based on four PHA sessions with five or six participants, but PHA can also be performed with as little as two participants, and still produce good results in the number of anomalies identified. This would certainly have reduced the ratio of staffhours per fault considerably.

5. Discussion

5.1 The results in terms of our Research Questions Our main findings related to RQ1 were that PHA was most useful in eliciting hazards that were related to “function” faults. These types of faults are related to specification and design, as shown in Table 3 and stated in [11, 13]. Hazards related to the “checking” and “algorithm” fault types were also common. Our reasoning about this result is that when performing a PHA, you are mostly basing your analysis on documentation and artefacts for the early stages of development. This means that you are more likely to be able to elicit possible hazards that are related to more general design and specification. Other types of hazards are found as well, but as the system details are unclear, it is more difficult in the PHA to specify exactly what can go wrong technically.

For RQ2, we did not find any correlation between hazards elicited through PHA and faults found in the fault analysis. As for finding direct matches between PHA findings and fault analysis, as stated in RQ3, there was a very low match rate. Of the 92 fault reports, only 6 of them could be said to have been specifically elicited as hazards in the PHA.

In this instance we believe that an important reason for the lack of match between elicited hazards and faults reported is the nature of the system under study. Compared to other systems we have performed fault analyses of, the DAIM system has a very different fault distribution profile. Earlier, we have performed similar analyses of fault reports, and these have had a distribution where “function” and “GUI” faults have been the most frequent. [15, 16].

5.2 Comparing fault distribution with previous studies When comparing the fault distribution for DAIM with fault distributions we have found in previous studies, we see that the distribution for DAIM seems atypical. As an example, we compare the fault distributions of DAIM and that of another fault report study where five industrial projects were analyzed from system testing and field use [16]. Figure 8 shows the difference between the distributions.

30,0 25,0 20,0

–  –  –

Figure 8. Comparison of DAIM fault distribution with previous fault study (%).

This is one example of why we think the DAIM system has an atypical fault distribution, and this is supported also by findings we made in [15] and also by Vinter et al. in [17]. As we see in Figure 8, the very numerous fault types “function” and “GUI” for the systems analyzed in [16] are not at all numerous in the fault reports for the DAIM system. This will of course have an impact on the ability to compare the fault reports in the DAIM system to the hazards found, where “function” and “GUI” were more numerous. It seems it would have been more appropriate to perform a “postmortem” hazard analysis on the systems studied in [16] as they had a fault profile more similar to the fault profile that hazard analyses are likely to result in.

5.3 Method evaluation: Improving specification and design inspection

The comparison of possible hazards identified during PHA sessions and the faults found after system testing and field use is a novel approach for exploring how the PHA method can be used for eliciting possible faults in a software system. PHA is a very light-weight and easy to learn method, which is suited for use in very early phases of development, as shown in Table 1. If we compare with the Perspective-Based Reading technique from [7], using PHA will result in inspections where the role of the readers will be with an emphasis on system safety.

Compared to the economics of other inspection methods, the results in terms of efficiency depend on using the number of faults actually found or the number of potential faults found. According to Wagner’s literature survey, the mean inspection efficiency is 1.06 staff-hours per defect found for requirements and 2.31 staff-hours per defect found for design [7]. Our study showed for actual faults found in fault reports an efficiency of 6.33 staff-hours per defect, and for potential faults found an efficiency of

1.46 staff-hours per defect, as shown in Table 6. As the DAIM system under study had not been injected with known faults, but was used because it was an accessible real life system with available documentation and fault reports, it is difficult to say how many actual faults the system had.

Also, it should be noted that the fault distribution of the DAIM system was very different from several other systems we have studied in previous studies. Another remark is that since PHA is a safety review technique, it will also catch potential safety hazards like the technique is originally meant to do. So when using the proposed method you could combine the results for safety and fault review, which would give another number on the efficiency, staff-hours per caught anomaly (hazards and faults).

5.4 Validity threats

The main validity threats in this study are:

Construct validity:

The main threat to construct validity is the difference between hazards and faults.

Hazard analysis and fault report analysis do not produce the same type of reports. By converting the hazards found to types of fault we were able to make a comparison between the two. As the results show, most of the hazards identified did not show up in the fault reports as actual faults. Still, a great deal of the hazards identified through the PHA would have manifested themselves as faults, if these hazards over time and diverse users had in fact occurred in the system. If these faults would have manifested themselves as observable failures in some future execution context is another matter.

Internal validity:

One threat to internal validity is that the fault categorization was performed by us.

Since such categorization is a subjective task, this can affect the reliability of measures.

By having two persons independently categorize and then compare results, we feel we reduced this threat.

Similarly, PHA sessions are also based on subjective views on the description of a system. This threat is not possible to circumvent as the PHA technique is based on personal ideas and collective “brainstorming”. The quality of the PHA results is dependant on the participants experience and knowledge.

Another threat is the issue of unconscious bias or “fishing”. By comparing the results from the PHA and the fault analysis, we were looking to find some connections between the results and this could have lead us to find ”weak” connections which others would not have found.

The group of hazards that were compared to the reported faults may cause a certain degree of validity threat in the form of selection. The PHA sessions were time limited, so only the most obvious hazards were taken into account. Also, the PHA sessions were performed over a period of time, so some maturation in the form of better understanding of the actual system may have occurred.

Another issue here is the time span of the study. It is possible that by studying too short a time span of fault report collection, some fault types were underreported in the collected fault reports.

External validity:

In our work we have analyzed data from only one and possibly atypical software system, DAIM, and this will conflict with the ability to generalize the results. Another issue is the size and simplicity of the software system studied, which may be smaller than many other web based projects of similar type. The development process of this system has also been rather small-scaled, with few people involved in design, implementation and testing. This may influence the distribution of faults found, as the developers have had a less complex system to develop. The reason thie DAIM system was chosen for study was that it was system developed close to us, which gave us a lot of freedom with respect to documentation accessibility and possibility for data collection and clarification with developers.

6. Conclusion and further work This paper has presented the description and an implementation of a novel method for identifying software faults using the PHA technique. Because of the nature of the system, the results did not turn out as clear as we had hoped. The fault reports were few and mostly limited to certain types. On the other hand, we did identify 6 faults that were actually found in the system as well as 20 potential faults that may be in the system. The hazard analysis also showed that there is a certain type of faults that analysis techniques such as PHA can help to uncover in an early process phase. Performing the PHA elicited many hazards that could have been found in the system as “function” faults.

That is, faults which originate from early phases of system development, and are related to the specification and design of the system. From this we conclude that PHA can be useful for identifying hazards that are related to faults introduced early in software development.

As for finding direct ties between hazards found in PHA and faults reported in fault reports, we were not very successful. This, we feel, is mainly due to the studied system’s particular fault type profile which was very different from fault distribution profiles we had found in earlier studies. Some weak links were found, but the data did not support any systematic links.

The method we have proposed in this paper should be validated by performing future similar studies. Because of the circumstances and type of system we analyzed here, interesting further work would be to execute a similar study on a larger system where the fault distribution would be more similar to other systems we have conducted fault report analyses of.

Acknowledgements The author wishes to thank Professor Reidar Conradi for valuable input and reviewing. I would also like to thank Jostein Dyre-Hansen, Professor Tor Stålhane, Kai Torgeir Dragland, Torgrim Lauritsen and Per Trygve Myhrer for their assistance during the execution of this study.

References [1] N. Leveson, Safeware: System safety and computers, Addison-Wesley, Boston, 1995.

[2] A. Avizienis, J.-C. Laprie, B. Randell, C. Landwehr: “Basic Concepts and Taxonomy of Dependable and Secure Computing”, IEEE Transactions on Dependable and Secure Computing, (1)1, January-March 2004.

[3] L. Huang, B. Boehm: “How Much Software Quality Investment Is Enough: A Value-Based Approach”, IEEE Software, (23)5, pp. 88-95, Sept.-Oct. 2006.

[4] S. Biffl, A.Aurum, B. Boehm, H. Erdogmus, P. Grünbacher: Value-Based Software Engineering, Springer, Berlin Heidelberg, 2006.

[5] Basili, V.R., Selby, R.W.: “Comparing the Effectiveness of Software Testing Strategies”, IEEE Transactions on Software Engineering, (13)12, pp. 1278 – 1296, Dec. 1987.

[6] Shull, F., Rus, I., Basili, V.: How perspective-based reading can improve requirements inspections, IEEE Computer, (33)7, pp. 73-79, July 2000.

[7] Stefan Wagner: “A literature survey of the quality economics of defect-detection techniques”, Proceedings of the 2006 ACM/IEEE international symposium on International symposium on empirical software engineering, Rio de Janeiro, Brazil, September 21-22, 2006.

[8] Ciolkowski, M, Laitenberger, O, Biffl, S.: “Software reviews, the state of the practice”, IEEE Software, (20)6, pp. 46-51, Nov.-Dec. 2003.

[9] M. Rausand: Risikoanalyse, Tapir Forlag, Trondheim, 1991.

[10] J. A. Børretzen; T. Stålhane; T. Lauritsen; P. T. Myhrer, “Safety activities during early software project phases,” Proceedings of the Norwegian Informatics Conference, pp. 180-191, Stavanger, Norway, 2004.

[11] R. Chillarege, I.S. Bhandari, J.K. Chaar, M.J. Halliday, D.S. Moebus, B.K. Ray, M.-Y.

Wong: “Orthogonal defect classification-a concept for in-process measurements”, IEEE Transactions on Software Engineering, (18)11, pp. 943-956, Nov. 1992.

[12] K. El Emam, I. Wieczorek: “The repeatability of code defect classifications”, Proceedings of The Ninth International Symposium on Software Reliability Engineering, pp. 322-333, 4-7 Nov. 1998.

[13] J. Zheng, L. Williams, N. Nagappan, W. Snipes, J.P. Hudepohl, M.A. Vouk: “On the value of static analysis for fault detection in software”, IEEE Transactions on Software Engineering, (32)4, pp. 240-253, April 2006.

[14] Oliver Laitenberger, Sira Vegas, Marcus Ciolkowski: “The State of the Practice of Review and Inspection Technologies in Germany”, Technical Report ViSEK/010/E, ViSEK, 2002.

Pages:     | 1 |   ...   | 23 | 24 || 26 | 27 |

Similar works:

«BIBLIOGRAFIA ADAMS, Roy D. e Masoud Moghaddam, The Risk Premia in Municipal Bond Yields: An Application of the ARCH-M Model., Journal of Macroeconomics 13 (4), Fall 1991, p. 725 731. AMSLER, Christine E., e Peter Schmidt, A Monte Carlo Investigation of the Accuracy of Multivariate CAPM Tests. Journal of Financial Economics 14, September 1985, p. 359 375. ARIEL, Robert A. “A Monthly Effect in Stock Returns.”, Journal of Finance 18, March 1987, p. 161 179. ATTANASIO, Orazio P., Risk,...»

«Interdisciplinary Journal of Research in Business Vol. 1, Issue. 7, July 2011(pp.53-80) Information Technology Adoption in Small and Medium-sized Enterprises; An Appraisal of Two Decades Literature Morteza Ghobakhloo (Corresponding author) Garmsar Branch, Islamic Azad University Garmsar, Iran E-mail: morteza_ghobakhloo@yahoo.com Mohammad Sadegh Sabouri Departments of Agriculture and Young Researchers Club, Garmsar Branch, Islamic Azad University, Garmsar, Iran E-mail:...»

«Report on the State Fiscal Year 2016-17 Enacted Budget May 2016 Message from the Comptroller May 2016 The State Fiscal Year (SFY) 2016-17 Enacted Budget includes a significant increase in school aid as well as a multibillion dollar reduction in personal income taxes to be phased in over nearly a decade. The Budget provides funding or commitments for new capital investments for the Metropolitan Transportation Authority and transportation projects throughout the State, as well as new funding for...»

«1 Business and Economics Journal, Volume 2010: BEJ-15 Empirical Study of the Effect of Conflict on Organizational Performance in Nigeria 1 O M Hotepo, 2A S S Asokere, *I A Abdul-Azeez1, 2S S A Ajemunigbohun 1 Department of Business Administration and Management Technology; 2Department of Accounting and Finance; Lagos State University, P.M.B. 0001, LASU Post Office Ojo, Lagos, Nigeria *Correspondence to: I A Abdul-Azeez, ibforson@yahoo.com Accepted: September 7, 2010; Published online: October...»

«OVERVIEW OF PRACTICAL CHALLENGES IN LOCAL SAVING MOBILIZATION BY ETHIOPIAN MICROFINANCE INSTITUTIONS (By Tony Storrow, Getaneh Gobezie and Anne Figge)1 Introduction For the poor, particularly those with low, irregular and unreliable income, saving is critical. Ethiopia’s poor do save. The poor tend to save fixed amounts of money regardless of income2, varying their consumption according to income, i.e, poor people do not save according to the equation, Savings = Income – Fixed Consumption...»

«The Per Curiam Opinion of Steel: Buckley v. Valeo as Superprecedent? Clues from Wisconsin and Vermont Allison R. Hayward* I. Introduction [W]e are talking about speech, money is speech, and speech is money, whether it be buying television or radio time or newspaper advertising, or even buying pencils and paper and microphones. That’s the—that’s certainly clear, isn’t it? Comments of Justice Potter Stewart during oral argument in Buckley v. Valeo, November 14, 1975.1 I think it was...»

«Budapest University of Technology and Economics PhD School in Psychology – Cognitive Science Gábor Péter Háden PROBING PERCEPTUAL CAPABILITIES UNDERLYING MUSIC PERCEPTION PhD thesis Supervisor: Dr. István Winkler Budapest, 2011 Contents Contents Acknowledgements Glossary of abbreviations Abstract Kivonat 1. Introduction 1.1. Questions about music and its origins 1.2. Music perception 1.2.1. Pitch and melody 1.2.2. Timbre 1.2.3. Rhythm, grouping and meter 1.3. The mismatch negativity 1.3.1...»

«Discuss this article at Journaltalk: http://journaltalk.net/articles/5887 ECON JOURNAL WATCH 12(2) May 2015: 242–259 The Endangered Classical Liberal Tradition in Lebanon: A General Description and Survey Results Patrick Mardini1 LINK TO ABSTRACT The Lebanese people believe that they live in a free market economy. However, Lebanon is ranked 96th in the Heritage Foundation’s 2014 Index of Economic Freedom and 60th in the Economic Freedom of the World Index. Compared to its Arab neighbors,...»

«Caterpillar, Inc. (CAT) – Short (67% upside): The Ultimate Value Trap (Would you rather be lucky or smart?) Mkt Cap: $56.35B EV: $92B Price Target: $28/shr Recommendation: Caterpillar (CAT) is an under-loved name that has lagged the S&P by 23% YTD, has a 2.5% dividend yield and to many appears to be at a cyclical trough. CAT has grown earnings from a low of $1.43 per share in 2009 to an impressive $8.48 last year, and looks to benefit from a positive North American construction cycle...»

«PATRICK LEE BROCKETT August 2014 HOME: 2902 Thousand Oaks Dr. OFFICE: Department of Information, Risk Austin, Texas 78746 and Operations Management (512) 293-1009 McCombs School of Business The University of Texas at Austin 1 University Station Austin, Texas 78712-1177 Phone: (512) 471-6816 (512) 471-3322 FAX: (512) 471-0587 E-mail: brockett@mail.utexas.edu CURRENT POSITIONS AT THE UNIVERSITY OF TEXAS AT AUSTIN Director, Risk Management and Insurance Program Gus S. Wortham Memorial Chair in...»

«INCOME AND BENEFITS POLICY CENTER RE S E AR CH RE P O R T The Growing Size and Incomes of the Upper Middle Class Stephen J. Rose June 2016 AB O U T T HE U R BA N I NS T I T U TE The nonprofit Urban Institute is dedicated to elevating the debate on social and economic policy. For nearly five decades, Urban scholars have conducted research and offered evidence-based solutions that improve lives and strengthen communities across a rapidly urbanizing world. Their objective research helps expand...»

«Wyrick Robbins Yates & Ponton LLP ATTORNEYS AT LAW 4101 Lake Boone Trail, Suite 300, Raleigh, NC 27607 PO Drawer 17803, Raleigh, NC 27619 wyrickrobbins P: 919. 781.4000 F: 919. 781.4865 www.wyrick.com January 28, 2016 Via E-Mail: shareholderproposals@sec.gov Office of Chief Counsel Division of Corporation Finance Securities and Exchange Commission 100 F Street, NE Washington, DC 20549 Re: First South Bancorp, Inc. Stockholder Proposal of Phillip A. Lewis Securities Exchange Act of 1934 -Rule...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.