FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:     | 1 |   ...   | 8 | 9 || 11 | 12 |   ...   | 27 |

«Software Fault Reporting Processes in Business-Critical Systems Jon Arvid Børretzen Doctoral Thesis Submitted for the partial fulfilment of the ...»

-- [ Page 10 ] --

5.1.1 Contributions related to BUCS goals The relation between our contributions and the BUCS goals as defined in Section 1.1

are now considered:

BG1 To obtain a better understanding of the problems encountered by Norwegian industry during development, operation and maintenance of business-critical software.

Regarding BG1, we have found that early development phases like specification and design are a source of a high number of faults in software. Lack of communication and adequate tools and processes for describing development difficulties in these phases seem to be the main problem. It is thought that the work in this thesis advances the state-of-the-art of software engineering of business-critical software as defined by our contributions C1-C3. Better understanding of problems encountered by Norwegian industry is achieved, as is reflected in contributions C1 and C3.

BG2 Study the effects of introducing safety-critical methods and techniques into the development of business-critical software, to reduce the number of system failures (increased reliability).

Our studies on fault reports suggest concrete measures to reduce the largest group of faults found in studies of business-critical software in our contribution C3. In addition, we have found that lightweight hazard analysis such as the PHA method is useful in eliciting hazards that could be avoided to reduce the number of faults originating in early development phases, from our contribution C1.

BG3 Provide adapted and annotated methods and processes for business-critical software.

Although the goal BG3 has not been an explicit focus of this thesis, we describe how fault report analysis and certain hazard analysis methods can be used to improve the development process, related to C1 BG4 Package and disseminate the effective methods into Norwegian software industry.

Most results are published, or planned on being published, and presented at interational and national conferences and workshops. During this thesis work, several Masters students have directly or indirectly been involved in activities, project work or Masters theses concerning business-critical systems and the BUCS project. Furthermore, the knowledge gained from our studies in commercial organizations have been disseminated back to them through reports and internal workshops. This relates to all contributions C1-C3.

5.2 Contribution of this thesis vs. literature In this section we present how our results and contributions compare with state-of-theart.

Looking at the wide perspective, our research on business-critical systems and software has shown not to be directly comparable with much of the literature on software engineering. This is something we were aware of from the start of the BUCS project.

The introduction of safety related methods into “regular” software engineering is not common, many of the methods are still regarded as resource-hungry and rigid methods, and this is difficult to combine with the emergence of agile and other lightweight methods [Beck99]. On the other hand, there are many types of systems that demand a more rigorous development process to ensure reliability and related qualities (e.g.

financial systems), and for these types of systems we have contributed both on a process level and with techniques that could be applicable.

In our work, we have proposed a novel method for doing fault inspections of specification and design documents [P6]. This adds to the existing literature on inspections, for instance that of Basili et. al concerning perspective-based reading [Basili00, Shull00].

The results of our quantitative studies on fault reports [P2][P4] show that in many systems, faults originating from specification and design phases constitute a major part of the total number of faults being found in testing. This is in line with the findings of Vinter et al. [Vinter00], but in contrast to findings by Eldh et al., where a common type of early process fault (function) was not very frequent [Eldh07]. Our fault report study of a small frame simple system in [P7] did, however, show that systems have different fault profiles.This may be as a result of both the type of system and the development method used when designing and implementing the system.

Further, we have discussed the need for improving fault reporting as a support tool for process improvement. Several sources present fault management processes as useful for such improvement in a software organization, among others [Grady92, Chillarege92].

We support this stance and suggest how to better utilize the available fault information [P3][P4][P6][P8].

5.3 Revisiting the Thesis Research Questions, RQ1-RQ4

In answering our four research questions, we have the following:

RQ1. What is the role of fault reporting in existing industrial software development?

a) Fault reporting seems to generally be underused and undervalued. Our experience is that the recorded data is often not of high quality, which not only makes any analysis hard, but also diminishes the usefulness of the fault reports for fixing faults.

b) All software developing organizations have a fault reporting system in operation, but its use differs substantially. The most basic fault reporting system is only used as a means to document faults that have been found and that are to be corrected, but more advanced use of the available data can easily be arranged.

c) Even where fault report data is thoroughly recorded and stored, it is not systematically used as a tool for software process improvement. A lot of detailed information is stored in the fault management systems of software organizations, but is never used beyond the simplest applications.

RQ2. How can we improve on existing fault reporting processes?

a) Developers should be more conscious about the potential for improvement by analyzing fault reports. Only through feedback on quality/fault data can an organization “learn from their mistakes”.

b) We need more formalized reporting schemes, and clearly defined procedures for reporting faults.

c) Introduce updated fault reporting schemes (fault type, severity, priority, effort, location etc) for the organization’s needs, so that the correct and complete information is reported. There is a need for a process looking at the requirements and possibilities in each organization.

RQ3. What are the most common and severe fault types, and how can we reduce them in number and severity?

a) P2 and P4 show that the most common fault type is the “function” fault type, i.e.

faults related to faults in the specification and design phases of development.

“GUI” faults are also numerous, and can in many cases also be related to specification and design phases.

b) Our studies on safety-critical analysis techniques have shown that the PHA technique is a useful tool for eliciting hazards that can be related to the fault types that are most common [P1] [P6].

RQ4. How can we use safety analysis techniques together with failure report analysis to improve the development process?

a) In P6, we have found that the PHA technique is useful for eliciting hazards that can be related to faults that are commonly introduced in early development phases.

5.4 Evaluation of validity For the validity of the work in this thesis, there are some overall issues to be discussed.

Initial validity concerns of the individual studies are discussed for each study in Section 3, as well as in each individual paper.

To improve validity of the studies seen as a whole, some possible actions can be


1. Replication of studies, both over time and in other organizations. This applies especially to the quantitative studies, in order to track development over and also to ensure that the results are generalizable. Example: our fault report studies on projects from five different organizations show very similar main results for most projects [P2, P4].

2. Using different research strategies to triangulate the research results. By using different research methods for the same study objects etc., we increase the validity of the results. Examples: Fault report study combined with interview sessions on the topic of fault report management [P4,P6], combining a qualitative study and a quantitative one in the DAIM study [P7].

Wohlin et al. define four main categories of validity threats [Wohlin00], which are further discussed in the next section, for different types of studies performed.

5.4.1 Quantitative studies: construct, internal, conclusion and external validity Studies 3, 4 and 6 used quantitative methods, and were mostly concerned with analyzing fault report data. These data were collected from existing fault report collections made by the organizations’ internal measures. Our contribution was the categorization of faults in the data where this had not been performed. Some threats to

the validity of quantitative studies and how this was handled is described here:

• Construct validity: In study 6, the main threat to construct validity is the conceptual difference between hazards and faults. We had to perform a conversion of the hazards found to potential fault types. It should be verified whether this type of hazard to fault type conversion is consequently correct, but during hazard analysis, there was a discussion of how each hazard could influence the system, and in many cases a software fault was proposed.

• Internal validity: In study 3, the greatest threat to internal validity is missing data in the fault reports. Many fault reports were not described well enough to be categorized and had to be left out. In certain fault reports, the fault had been classified by the developers, and they may have had a different opinion of the fault types than we had. In addition, with respect to severity of faults, it is not certain that the developers reporting the fault necessarily reported the true severity. In study 6, the hazard analysis sessions were time limited, so only the most obvious hazards were taken into account. Also, these sessions were performed over a period of time, so some maturation in the form of better understanding of the system being analyzed can have occurred.

• Conclusion validity: One possible threat to conclusion validity in study 3 and 4 is low reliability of measures, because of some missing and ambiguous data.

Because categorization of faults into fault types is a subjective task, it was important that the data we based the categorization on was correct and understandable. To prevent mistakes, we added an “unknown” type to filter out the faults we were not able to confidently categorize. The subjective nature of categorization is also a threat to conclusion validity.

• External validity: Especially in study 6 where we only studied one project, but also partly in studies 3 and 4 one threat to external validity is the relatively low number of projects studied. In study 6 we were not able to gain access to system documentation of more systems where we could also have fault report data. The projects under study may also not necessarily be the most typical businesscritical systems, but this is hard to verify in any way.

5.4.2 Qualitative studies: internal and external validity.

Studies 1, 2, 5 and partly 6 are qualitative studies, mostly explorative and descriptive in nature. The collected data is mainly from interviews and other subjective techniques (PHA) and are subject to interpretation. Here we have identified internal and external validity threats as the most serious.

• Internal validity. For Study 5, the main internal validity threat is that the same person performed interviewing, transcribing and information coding, which may introduce bias to how responses have been interpreted. By having workshops as feedback sessions after the interviews, we feel bias have been reduced.

• External validity. In Study 1, the main validity threat was the low number of organizations interviewed, and in Study 5 all interviews were performed with representatives from the same organization, although this is explained by us having to interview the people who had been involved in certain projects we had studied earlier.

5.5 Industrial relevance of results As many of our studies involved industrial data, our results were interesting not only to us, but also to the organizations the data was collected from. As such, we were able to present our results to the organizations and receive feedback both in terms of the results of the studies and how we should interpret the results.

In general, the organizations received general reports on the results, but also a specific report concerning the results from their organization. After Study 4, a workshop was held in order to convey our results to the organization as well as to receive more feedback.

5.6 Reflection: Research cooperation with industry Both the BUCS and the EVISOFT research projects are based on cooperation with industrial partners. Whereas EVISOFT had a number of industrial organizations involved from the project start, the BUCS project had no formal connections to any industrial partners as the project got under way. This meant that some effort had to be made in order to initiate contact and agreement with organizations in order to collect research data.

The hardest part of industrial cooperation was establishing contact and an agreement about what was going to be performed. In Study 3, the first fault report study, we initially contacted over 40 different organizations developing business-critical software.

Pages:     | 1 |   ...   | 8 | 9 || 11 | 12 |   ...   | 27 |

Similar works:

«Minutes – September 8th, 2015 – Council Meeting 90 MUNICIPALITY OF HURON EAST COUNCIL MEETING MINUTES HELD IN THE COUNCIL CHAMBERS, SEAFORTH, ONTARIO TUESDAY, SEPTEMBER 8th, 2015 – 7:00 P.M. Members Present: Mayor Bernie MacLellan, Deputy Mayor Joseph Steffler, Councillors Alvin McLellan, John Lowe, Dianne Diehl, Brenda Dalton, Robert Fisher, Nathan Marshall, Larry McGrath, Kevin Wilbee and Raymond Chartrand Members Absent: Councillor David Blaney Staff Present: CAO/Clerk, Brad Knight...»

«27 ConFerenCe SuMMary:  FinanCial inTerMediaTion and vulneraBiliTieS BANK OF CANADA REVIEW • WIntER 2012–2013 Conference Summary: Financial intermediation and vulnerabilities Jason Allen, James Chapman and Ian Christensen, Financial Stability Department The Bank of Canada’s annual economic conference, held in October 2012, brought together experts from around the world to discuss key issues concerning financial intermediation and vulnerabilities. The conference covered such topics as...»

«Carl Sargent AC/AM Edwina Hart MBE CStJ AC / AM Y Gweinidog Cyfoeth Gweinidog yr Economi, Naturiol Gwyddoniaeth a Thrafnidiaeth Minister for Natural Resources Minister for Economy, Science and Transport Alun Fred Jones Chair of the Environment and Sustainability Committee National Assembly for Wales Cardiff Bay Cardiff CF99 1NA committeebusiness@wales.gsi.gov.uk 27 January 2016 Dear Alun Ffred Thank you for a very interesting committee session on 13 January. We hope that the evidence we...»

«Urbanities, Vol. 3 · No 1 · May 2013 © 2013 Urbanities Molding of a Rite of Passage in Urban Japan: Historical and Anthropological Perspectives Melinda Papp (ELTE University, Budapest) pappmel@gmail.com This article focuses on the Japanese childhood ritual of Shichigosan and its changes over the last two centuries. The transformation of Japanese society has affected traditional customs to a great extent, including rites of passage. Social and economic processes such as urbanization, the rise...»

«資料1 Experts’ Meeting on International Finance and the Global Economy Ahead of the G7 Summit OECD Note for Prime Minister Shinzo Abe Wednesday, 13 April 2016 1 Prospects and Risks for the Global Economy A pickup in global growth remains elusive. World real GDP is projected to expand by only 3% in 2016, with a modest uptick expected in 2017. The recent weakening in momentum is broad-based across advanced and emerging market economies. While financial market volatility has receded from its...»

«PARENTING PLAN OPTION 1: This Parenting Plan Agreement is entered into by and between Petitioner * * * (Father/Mother) and Respondent *** (Father/Mother). The parties have [no.] child/ren to which this Agreement applies, * * * (the Child/ren).OPTION 2: This Parenting Plan applies to Petitioner * * * (Father/Mother), Respondent *** (Father/Mother), and their child/ren, * * * (the Child/ren). 1. CUSTODY. The parties agree that joint legal custody [and shared physical custody of their minor...»

«THE FAILURE OF NORTHERN ROCK: A MULTI-DIMENSIONAL CASE STUDY Edited By Franco Bruni and David T. Llewellyn Chapters by: Tim Congdon Charles A.E. Goodhart Robert A. Eisenbeis and George G. Kaufman Paul Hamalainen Rosa M. Lastra David T. Llewellyn David G. Mayes and Geoffrey Wood Alistair Milne Marco Onado Michael Taylor SUERF – The European Money and Finance Forum Vienna 2009 CIP THE FAILURE OF NORTHERN ROCK: A MULTI-DIMENSIONAL CASE STUDY Editors: Franco Bruni and David T. Llewellyn; Authors:...»

«The Collected Stories Of Katherine Mansfield The generation on them The Collected Stories of Katherine Mansfield feel you that the current month makes to do my store manager credit appraisal. Weak or graphic scenarios will surge the sure table to a property until this part or delays the placement of a play to do they recognized all the happiness. A in a wages can offer check your likely price The Collected Stories of Katherine Mansfield not higher and a would suffer you start a charitable real...»

«SPEARHEADING ICT4D KNOWLEDGE ICT for Development Working Paper Series Volume 4 Number 1 June 2014 SPEARHEADING ICT4D KNOWLEDGE ICT for Development Working Paper Series Volume 4 Number 1 June 2014 TABLE OF CONTENTS Editor’s Note 04 07 Utilization of Social Networking Sites of Government Agencies for Disaster Preparedness and Management Ma. Stephanie Joy A. Custodio, Michelle Mae Cabañero, Ave Jean P. Zubeldia and Christopher Jay M. Cortado Meron o Wala: A Study on the Usefulness of Twitter 25...»

«Regional and Global Liquidity Arrangements Ulrich Volz / Aldo Caliari (Editors) Regional and global liquidity arrangements Ulrich Volz / Aldo Caliari (eds.) Bonn 2010 German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE) The German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE) is a multidisciplinary research, consultancy and training institute for Germany’s bilateral and for multilateral development co-operation. On the basis of...»

«BIS RESEARCH PAPER NUMBER 266 Impact of Poor Basic Literacy and Numeracy on Employers FEBRUARY 2016 Impact of poor basic literacy and numeracy on English employers The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Business, Innovation and Skills. Department for Business, Innovation and Skills 1 Victoria Street London SW1H 0ET www.gov.uk/bis Research paper number 266 January 2016 Impact of poor basic literacy and numeracy on...»

«Food and Drink Medieval Period David Freidenreich Article Table of Contents 1. 1. Diet 2. 2. Dietary Laws 3. 3. Economic Aspects 4. 4. Literary Aspects 5. Bibliography Of the good things of this world the Muslims enjoy most sex; the Christians, money, the Persians, status; and the Jews, food.” So runs an Arabic maxim, perhaps from the early Islamic era, that S. D. Goitein quotes in the introduction to his indispensable discussion of evidence regarding food and drink from the Cairo Geniza...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.