FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:     | 1 |   ...   | 4 | 5 || 7 | 8 |   ...   | 27 |

«Software Fault Reporting Processes in Business-Critical Systems Jon Arvid Børretzen Doctoral Thesis Submitted for the partial fulfilment of the ...»

-- [ Page 6 ] --

According to Votta et al., the goal of empirical software engineering is to construct a “credible empirical base... that is of value to both professional developers and researchers” [Votta95]. They argue that empirical software engineering inherits most of the methodological approaches and techniques of social sciences, since its goal is to examine complex social settings, contexts where the human interaction is the most critical factor determining the quality and effectiveness of the results being produced. In particular, empirical work is accomplished through the execution of empirical studies.

This entails observations of specific settings, where the purpose is collecting and analysing/deriving useful information on their behaviour and attributes.

Empirical studies can be classified in three categories, according to the increasing

degree of rigor and confidence in the results of the study [Wohlin00]:

(1) Surveys (2) Case studies (3) Controlled experiments This is in line with the classification made by Votta et al. where the different types of investigations are said to be anecdotal studies, case studies and experiments [Votta95].

It can be argued that surveys and case studies should swap positions, as case studies do tend to be hard to replicate and very open ended, while surveys can be made very rigorous and are definitely suitable for replication.

Zelkowitz et al. describe different ways of studying technology [Zelkowitz98], and states that the empirical method is the following: “Empirical method: A statistical method is proposed as a means to validate a given hypothesis. Unlike the scientific method, there may not be a formal model or theory describing the hypothesis. Data is collected to verify the hypothesis.” Zelkowitz et al. describe 12 different ways of studying technology, in three dimensions: Observational, Historical and Controlled. The different ways are described in detail in [Zelkowitz98], and the the models they propose for validating technology are shown in table 2-3. In addition to these, there are a few other techniques commonly used in empirical software engineering. Firstly, as previously mentioned Wohlin et al. discusses surveys in [Wohlin00]. Secondly, the Action Research method is presented by Avison et al. This method involves researchers and practitioners acting together on a set of activities, including problem diagnosis, action intervention and reflective learning [Avison99]. Finally, a research method that is becoming more used in studies of software developing organizations is Grounded Theory which emphasizes generation of theory from data. This method originates from the sociologists Barney Glaser and Anselm Strauss and is presented in [Strauss98].

–  –  –

2.8.1 Research strategies in Empirical Research There are three main types of research strategies, each with distinct approaches to empirical studies, and they may all be used for Empirical Software Engineering [Wohlin00][Seaman99]:

• Quantitative approaches are mainly used to quantify a relationship or comparing groups, with the aim being to identify cause-effect relationships, verifying hypotheses or testing theories.

• Qualitative approaches are observational studies with the aim of interpreting a phenomenon based on information collected from various sources. This information is usually subjective and non-numeric.

• Mixed-method approaches are used to overcome limitations in the two strategies above, by triangulation of data and combining the advantages of the two former ones.

Table 2-4 gives an overview of empirical research approaches and examples of strategies for each. This is taken from [Moghaghegi04b] and [Creswell03].

The boundaries between these approaches are not sharp. For instance, case studies can combine qualitative and qualitative studies, and although case studies are often classed as qualitative in nature Yin states that case studies do not by their nature equal qualitative research [Yin03].

–  –  –

2.9 Main challenges in business-critical software engineering The following is a short list of some of the general challenges in software engineering today. They are sometimes cited as reasons for difficulties in software engineering, e.g.

in [Charette05]:

• Poor Requirements

• Rising Complexity

• Ongoing Change

• Failure to Pinpoint Causes In our work, we concentrate on issues dealing with reduction of product risk, improving requirement specifications, coping with complexity, and helping with pinpointing causes for failures. This adds up to a main challenge for business-critical software


• Developing methods that help reduce product risk, without increasing costs too much compared to the received benefit from these methods.

What this means is that we need to introduce low cost methods and techniques that focus on important areas, so as to spend just a little more effort to reduce the largest problems. This is in line with Boehm’s notion of value-based software engineering, where the main agenda is developing an overall framework to better focus effort where it is needed [Boehm03].

As far as empirical software engineering research is concerned, an important challenge is that of following up research and technology change proposals with continued observation and measurement in the field when practitioners put theory into practice.

Much of the research being performed concerns the software development process up to a point, but does not follow the eventual implementation of these results further.

3 Research Context and Design In this chapter, the project context is presented with more details. The overall research design is presented, and it combines quantitative and qualitative studies. Finally a more detailed description of each study is given.

3.1 BUCS Context In the last decade, computers have taken on a more important role in several areas of commercial business. As an effect of this, many of the functions required by industry and services depend on software and computer systems. Failures in such systems can have serious consequences for businesses that depend on these systems for their livelihood. As in many related areas, there will be substantial savings by discovering, reducing or removing these potential failures early in the system’s life-cycle. In fact, most potential failures should be possible to reduce very early in the process of the system’s development.

The main goal for the BUCS project is to better understand and to improve the software technologies and processes used for developing business-critical software. Much of the information about current practises and possible problems was collected at Norwegian IT companies. It was important that the relationship between the BUCS project and the involved companies and organizations was based on mutual profit for both parties.

The BUCS project have – through literature reviews, controlled experiments, historical data analysis and case studies – investigated methods from the area of safety-critical software. These methods include Preliminary Hazard Analysis (PHA), HazOp, Fault Tree Analysis (FTA), Cause-Consequence Diagrams (CCD), and Failure Mode and Effect Analysis (FMEA). We have also studied important standards in this area, such as IEC 61508 – a standard for functional safety. The effects of both methods and standards have been studied using controlled student experiments and through industrial case studies. All the above-mentioned methods are rather general. They can be applied to both local and distributed systems, they can be used on hardware, software and “wetware” (people). This is specially important when we are dealing with a problem as wide and diverse as business criticality.

Important techniques that we have sought to adapt from the development of safetycritical software are mainly PHA, HazOp, and FMEA.

As well as the industrial focus from the BUCS basic R&D project, some of the studies in this thesis also cooperated with organizations that were involved in the EVISOFT project, an industrially-driven research project [EVISOFT].

3.2 Research Focus When deciding the focus areas of this thesis, the input was the BUCS project context, less-exploited research areas, and available sources for research data. During our literature studies and after contact with Norwegian IT-companies some key areas to

focus our work in this thesis was decided:

• Business-criticality in terms of software faults

• Fault report analysis

• Fault reporting in software development

In terms of goals for the research, we formulated the following research questions:

• RQ1. What is the role of fault reporting in existing industrial software development?

• RQ2. How can we improve on existing fault reporting processes?

• RQ3. What are the most common and severe fault types, and how can we reduce them in number and severity?

• RQ4. How can we use safety analysis techniques together with failure report analysis to improve the development process?

To obtain answers to these research questions, we decided on common metrics for our studies. We started broadly, including attributes like structural fault location, functional fault location and fault repair effort. When we received actual data from industrial projects, we had to reduce the scope somewhat. This was due to lack of complete information in the data material, and great variation between organizations on what data they stored.

The main metrics we identified for fault report studies were:

• The number of detected faults is an indirect metric, attained simply by counting the number of faults of a certain type or for a certain system part etc.

• The metrics that are used directly from the data in the fault reports are the reported type, severity, priority, and release version of the fault.

The reasons why we decided to focus on software faults were several:

• The BUCS project is concerned with business-critical systems. A recurring theme in the definition of business-criticality is that the major threat to such systems is failures that stop or limit the use of the system. As described in section 2.4, “Faults are potential flaws in a software system, that later may be activated to produce an error. An error is the execution of a "passive fault", leading to a failure.” This means that by working to reduce the number and criticality of faults in the software, we would also reduce the number or frequency of failures.

• As Avizienis et al. suggests, one way of attaining better dependability in a system is fault removal to reduce the number and severity of faults [Avizienis04]. By working to identify critical or numerous fault types, developers can eliminate a larger number of faults by focusing on preventing such fault types.

• As far as industrial data, fault report data is abundant in most software developing organizations. Thus we had a wide array of potential industrial partners to collect data from.

As shown in Figure 1-1, the studies we have performed are all connected on the topic of business-critical software and fault report analysis, and have been performed in sequence. A short description of the studies is given in Table 3-1, and each study is elaborated in Section 3.3.

–  –  –

Study 2 Literature review: Software Criticality Techniques, Fault (2004) reporting and management literature.

Study 3 Historical data mining: Fault report analysis of industrial (2004-05) projects from four organizations.

Study 4 Historical data mining: Fault report analysis of industrial (2006-07) projects.

Study 5 Structured interviews: Exploring the results from study 4 further, (2007) regarding fault report analysis and fault reporting processes.

Study 6 Case study: Comparison of hazard analysis and fault report (2006-07) analysis in a practical setting.

Study 7 Lessons learned from our experiences with fault report studies.

(2004-07) 3.2.1 Data collection Before starting to plan and conduct our empirical studies, we decided on the goals of our studies, which types of studies we were going to perform, and which data sources we were going to need to complete them. Data collection was split up in three phases.

First, there was a pre-study phase of initial data collection and pre-analysis of this to narrow down research areas and questions. Second, we would focus the data collection on more deeper issues that seemed the most relevant. Finally, there would be an analysis phase to summarize and reflect and to collect lessons learned.

As the BUCS project is aimed at supporting business-critical systems, and part of the BUCS goals was close cooperation with Norwegian IT industry, we chose early on to focus on empirical studies of Norwegian commercial projects and organizations. This meant that we had to contact and select relevant organizations developing businesscritical applications. This raised the issue of which organizations we wanted to study.

As it turned out, the sampling of companies was mostly done out of convenience, because of apparent reluctance to disclose sensitive information about quality data and processes, as well as many organizations simply being “too busy”.

Another issue was whether data collection should be performed in pre-implementation phases or post implementation ones. Pre-implementation studies are better for working with possibilities for improvement initiatives, but a problem is knowing which data to collect. In this case the data would be more of a qualitative nature, and thus harder to analyse. Post-implementation studies, on the other hand, would be better for obtaining quantified data, but then there is the question about the data being relevant for investigation.

Once the studies to be performed were tentatively planned, the actual data collection depended on the available projects and their data, i.e. which companies we were able to cooperate with, and which processes those companies were willing to let us participate in. The employed data collection methods were interviews, surveys, and field/case studies, as well as historical data mining and analysis of reports or logs on relevant issues.

Pages:     | 1 |   ...   | 4 | 5 || 7 | 8 |   ...   | 27 |

Similar works:

«CHALLENGING THE RE-REGULATION OF LIBERALIZED ELECTRICITY PRICES UNDER INVESTMENT ARBITRATION Anatole Boute∗ Synopsis: According to the liberalized electricity markets paradigm, free market prices are expected to stimulate sufficient investment in production capacity to meet future electricity demand. This implies that electricity producers must be able to recover their operating and investment costs whilst achieving a reasonable return on their investment. They must be allowed to pass...»

«Journal of Economic Perspectives—Volume 30, Number 1—Winter 2016—Pages 185–206 Power Laws in Economics: An Introduction Xavier Gabaix P aul Samuelson (1969) was once asked by a physicist for a law in economics that was both nontrivial and true. This is a difficult challenge, as many (roughly) true results are in the end rather trivial (for example, demand curves slope down), while many nontrivial results in economics in fact require too much sophistication and rationality on the part of...»

«Maples and Calder Funds Update – Ireland Quarterly Update | April June 2015 Contents 1 Legal & Regulatory 1.1 AIFMD Update 1.2 Irish Collective Asset-Management Vehicle Act 2015 1.3 UCITS: Update 1.4 Companies Act 2014 1.5 Fund Management Company Boards 1.6 Money Market Funds: Update 1.7 EMIR Update 1.8 Corporate Governance Requirements for Investment Firms 1.9 MiFID II/MiFIR 1.10 PRIIPs KID Regulation 1.11 European Venture Capital Funds and Social Entrepreneurship Funds 1.12 CSDR 1.13 Market...»

«COMMONWEALTH CARIBBEAN BUSINESS LAW Second Edition Commonwealth Caribbean Business Law breaks away from the traditional English approach of treating business law primarily as the law of contract and agency. The book takes a panoramic view of the foundation of various legal systems with a subsequent examination of different areas of legal liability that may affect business activities. These areas include contract law, agency, tort law, criminal law and internet law as significant challenges...»

«CURRICULUM VITAE MOTY AMAR ACADEMIC POSITIONS Current position Assistant professor of Marketing, School of Business, Ono Academic College, Israel. 2009 – 2010 Post-Doctoral Fellow, Department of Marketing and JDM, Fuqua School of Business, Duke University, USA )iinvited by Dan Ariely(. EDUCATION 2004 2009 Ph.D., Department of Marketing. School of Business Administration, the Hebrew University of Jerusalem. Dissertation: “Generating a Placebo Effect through Marketing Actions”. Advisors:...»

«Working Paper/Document de travail 2012-25 Does the Buck Stop Here? A Comparison of Withdrawals from Money Market Mutual Funds with Floating and Constant Share Prices by Jonathan Witmer Bank of Canada Working Paper 2012-25 August 2012 Does the Buck Stop Here? A Comparison of Withdrawals from Money Market Mutual Funds with Floating and Constant Share Prices by Jonathan Witmer Financial Markets Department Bank of Canada Ottawa, Ontario, Canada K1A 0G9 jwitmer@bankofcanada.ca Bank of Canada working...»

«SUBCOMMITTEE OF THE STANDING COMMITTEE ON PUBLIC ADMINISTRATION AND FINANCE WATER SERVICES INQUIRY TRANSCRIPT OF EVIDENCE TAKEN AT BUNBURY THURSDAY, 29 JULY 2004 SESSION 2 Members Hon Barry House (Convenor) Hon John Fischer Hon Dee Margetts Hon Norman Moore (Participating Member) Hon Ken Travers Public Administration and Finance Subcommittee Session 2 Thursday, 29 July 2004 Page 1 [10.45 am] CALDER, MR GEOFF General Manager, Harvey Water, PO Box 456, Harvey, examined: Hon BARRY HOUSE: On behalf...»

«MONEY, MACROECONOMICS AND KEYNES This volume, along with its companion volume Methodology, Microeconomics and Keynes, is published in honour of Victoria Chick, inspired by her own contributions to knowledge in all of these areas and their interconnections. It represents both consolidation and the breaking of new ground in Keynesian monetary theory and macroeconomics by leading figures in these fields. The chapters have been contributed by some of the many who admire Chick’s work: C. Rogers,...»

«4. WHAT CAN YOU BE LIABLE FOR AND WHY? 4.1 Negligence Liability for negligence is a civil, not a criminal, matter. It is for the victim to prove that the defendant owed them a duty of care, that that duty was breached, and that they have sustained either foreseeable harm or economic loss as a consequence of the negligence alleged. If successful, the victim will be awarded damages assessed on the basis of the harm caused or loss sustained. In practice this means that a water chemist must avoid...»

«Volume 5, Number 2 2005 Allied Academies International Conference Las Vegas, Nevada October 12-15, 2005 Academy of Commercial Banking and Finance PROCEEDINGS Volume 5, Number 2 2005 page ii Allied Academies International Conference Las Vegas, 2005 Proceedings of the Academy of Commercial Banking and Finance, Volume 5, Number 2 Allied Academies International Conference page iii Table of Contents BUSINESS AND PERSONAL BANKRUPTCY.....................................»

«Proceedings of the 21st European Conference on Information Systems MODELING CONCEPTS FOR PROCESS AUDITS – AN EMPIRICALLY GROUNDED EXTENSION OF BPMN Mueller-Wickop, Niels, University of Hamburg, Max-Brauer-Allee 60, 22765 Hamburg, Germany, niels.mueller-wickop@wiso.uni-hamburg.de Schultz, Martin, University of Hamburg, Max-Brauer-Allee 60, 22765 Hamburg, Germany, martin.schultz@wiso.uni-hamburg.de Nuettgens, Markus, University of Hamburg, Max-Brauer-Allee 60, 22765 Hamburg, Germany,...»

«Agenda Monday 26 September 2011 Council Chambers, 126-138 Main Road Speers Point 7.00pm Apologies: For the non-attendance of Councillors Declaration of Interests: Confirmation of Minutes: Ordinary Council Meeting held on 12 September 2011 Presentations: Nil Public Access: Nil Notices of Motion: Awaba, Heaton and Olney State Forests 26 September 2011 Mayoral Minutes: Nil Recommendations of the: City Strategy Committee 19 September 2011 Community Development Committee 19 September 2011 General...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.