«This research has been partially sponsored by the Dutch Joint Academic and Commer- cial Quality Research & Development (Jacquard) program on Software ...»
• While our understanding of the management of architectural knowledge matured, we used the insight obtained to choose an alternative perspective towards the management of architectural knowledge in GSD. This alternative perspective focuses on managing architectural knowledge in GSD by capturing the architectural knowledge in the software product. With software product, we refer to the software architecture, the software itself, and its documentation, following (ISO/IEC, 2000). The software product can be a (possible intermediate) tailor-made solution, since we do not focus on commercial-off-the-shelf components in our research. Hence, we posed an additional question to identify whether or not the quality of software products developed using GSD, can be improved by architectural knowledge stored in the products. This question is answered in Chapter 5.
• As we have learned during the course of our research, the requirements engineering discipline is a well-discussed example of a discipline that becomes challenging in GSD (Damian, 2007; Hsieh, 2006). Furthermore, several solutions have been proposed and validated in the requirements engineering discipline. The similarity between the requirements and architecture and, more speciﬁcally, requirements and (architectural) design decisions has been pointed out by (van Vliet, 2008; de Boer and van Vliet, 2009). We have chosen the requirements perspective to identify whether architectural knowledge management can leverage the solutions that are available from the requirements engineering domain to address the challenges in GSD. As such, we identify proposed solutions using this alternative requirements perspective. We validated the usefulness of these proposed solutions at GSD projects that ran an industrial partner in our research and specifically looked into the number of sites involved in these GSD projects. The proposed solutions are described in Chapters 6 and 7.
Practitioners involved in managing architectural knowledge in GSD are only motivated to do so if they are supported adequately. As described in §1.2.6, several supporting solutions are proposed but not yet aimed at managing architectural knowledge. Hence, we identify possible supporting instruments (tools) for the practices that identiﬁed in RQ-2. The answer to this question is provided in Chapter 8.
RQ-3 How can architectural knowledge management (practices) in global software development be supported (by tools)?
To fully understand the applicability of the practices for architectural knowledge management in GSD, we investigate which of our deﬁned practices (see RQ-2) are actually used in GSD practice. The answer to this question is provided in Chapter 9.
RQ-4 Which practices for managing architectural knowledge are used in a global software development environment?
Fig. 1.2 illustates the research questions in context.
1.6 Research Methods and Studies This research was conducted at several industrial partners within the GRIFFIN and Stephenson projects.
All industrial partners where we performed studies are involved in global software development. Furthermore, the organizations were interested in obtaining research results on (how to) better utilize architectural knowledge in their GSD activities.
Although the GSD involvement and the interests of the industrial partners provides for the ability to use results obtained in a certain case study in subsequent studies, we do acknowledge that differences exist between the industrial partners in e.g., industry domain and size of the development organizations. To overcome these differences, we explicitly decided to not make an a priori assumption on the development process or
14 1.6. Research Methods and Studies
any methodology used by the industrial partners or any other organization that wants to implement architectural knowledge management in GSD.
In this section, we provide an overview of the research methods we applied in this research. To this end, Table 1.1 provides an overview of the studies performed, the research methods applied, the research questions addressed, and the chapter in this thesis that describes the results. The research methods applied are described in §1.6.1. Next, the outline and high-level approach of each of the individual studies is described in §1.6.2 through §1.6.7.
1.6.1 Research Methods Exploratory survey – (Pﬂeeger and Kitchenham, 2001) deﬁne a survey as “a comprehensive system for collecting information to describe, compare or explain knowledge, attitudes, and behavior”. Thus, a survey can be used to attempt to describe a phenomenon of interest (Kitchenham and Pﬂeeger, 2001-2002). An exploratory survey is a speciﬁc kind of survey aimed at performing an exploratory ﬁeld study in which there is no test of relationships between variables (Holz et al., 2006). We used an exploratory survey in our ﬁrst study to identify how architectural knowledge is used and in a descriptive case study on the perceived usefulness of the architectural knowledge management practices for GSD at Organization D, a large IT service provider.
Case studies – The main research method applied in for the research described this thesis is case study research. We use the deﬁnition of a case study provided by (Yin, 2003):
“a case study is an empirical inquiry that investigates a contemporary phenomenon
within its real-life context, expecially when the boundaries between phenomenon and context are not clearly evident”. As such, a case study is a research strategy which focuses on understanding the dynamics present within single settings (Eisenhardt, 1989).
Case study research emphasizes a detailed contextual analysis of a limited number of events or conditions and their relationships. According to (Yin, 2003), case studies may help in contributing to our knowledge of individual, group, organizational, social, political, and related phenomena. Case studies are the preferred strategy when ‘how’ or ‘why’ questions are being posed and when the researcher has little control over events.
Different types of case studies exist (Yin, 2003):
• Explanatory case studies – These studies are causal investigations and focus on the understanding of why a certain phenomenon occurs by studying potential factors that led to it.
• Exploratory case studies – Study patterns in collected data to identify a model that can be used to view or interpret the data.
• Descriptive case studies – Study aimed at validating an initial theory or model that may be the result of exploratory investigations.
We performed case studies where we conducted an empirical study at one of the industrial partners of our research; the case studies were primarily exploratory and descriptive in nature. Our aim was either to identify a model for interpretation or to test or validate the model developed initially. The case study performed at Organization A, a large developer of consumer electronics products, was exploratory in nature, the case studies performed at Organization D and Organization E (a Dutch producer of high-end printers for the business markets in high-volume printing, wide-format printing, and ofﬁce printing) were descriptive in nature.
Document analysis – Document analysis was performed on the architectural rules in the case study at Organization A and on the set of software product assessment reports to identify differences between GSD projects and collocated projects.
Critical analysis of the literature – This method concerns an appraisal of relevant published material based on careful analytical evaluation. We performed a critical analysis of the literature to identify architectural knowledge management practices based on the requirements engineering literature. We used a systematic literature review (Brereton et al., 2007) for our study to identify the key elements for architectural knowledge as part of our research on product practices for architectural knowledge management in GSD and on wiki support for the architectural knowledge management practices in GSD.
Prototype development – Prototype development is a technique that is used to validate parts of the theories developed for their suitable application in practice. The system that is built does not comprise all envisioned features; yet, it provides sufﬁcient functionality to convince the reader that the application is viable and can be effective. We have developed a prototype to show how selected use cases for architectural knowledge on which several of the architectural knowledge management practices for GSD build, can be implemented in practice.
16 1.6. Research Methods and Studies
Content analysis – Content analysis is a research method that uses a set of procedures to make valid inferences from text (Weber, 1990). Content analysis enables researchers to analyze large amounts of textual information and systematically identify its properties, e.g., the frequencies of most used keywords (conceptual analysis, according to (Weber, 1990)). This research method helps in e.g., revealing the focus of individuals or groups. We used this research method at Organization E to identify what architectural knowledge management practices for GSD are actually used in practice.
In several case studies, we use interviews as an information gathering technique. In an interview, questions are posed by an interviewer to an individual or group of individuals.
In our research, we have performed individual interviews using a question set. Conducting individual interviews allows for unbiased results and prevents group think. Group think is “a mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when the members’ striving for unanimity overrides their motivation to realistically appraise alternative courses of action” (Janis, 1972). A question set ensures that the same general areas of information are collected from each interviewee and hence allows us to compare and combine the results. We conducted interviews as part of all the case studies performed in this research; at Organization A, Organization B (a Dutch IT service provider that develops business information systems), Organization D, and Organization E.
1.6.2 1 – Survey: How is architectural knowledge shared?
Our ﬁrst study was conducted in 2006 and builds upon earlier work performed in the GRIFFIN project. As part of the initial activities in the GRIFFIN project, we have developed a series of so-called use cases for architectural knowledge (van der Ven et al., 2006a). These use cases cover a variety of reasons, activities, and motivations (i.e., possible uses) for using architectural knowledge2.
The intent of this study was to obtain an understanding of the applicability of these use cases in the architecting community in The Netherlands. We performed surveybased research at a broad selection of IT practitioners with architecting responsibilities in the Netherlands.
One of the main insights obtained in this study is that the intended beneﬁts of viewing the architecture as a set of architectural decisions (see §1.1.1) is not yet embedded within the mindset of the architects. Typically, the mindset of architects is rather positive and aimed at forward engineering by taking architectural decisions to address stakeholders’ concerns, rather than (re-)viewing the set of taken architectural decisions to perform impact or risk analyses.
2 Hence, the term use case differs from the deﬁnition given in (Jacobson, 1992) which focuses on a series of actions. Although the use cases for architectural knowledge are described as a series of steps, this ﬁrst study primarily focuses on the possible uses.
1.6.3 2 – Case study: Understanding practices for architectural knowledge management in global software development The ﬁrst case study at one of GRIFFIN’s industrial partners (Organization A) was conducted in 2006. The aim of this case study was to understand how architectural knowledge is managed in a typical organization involved in global software development. In this case study, we speciﬁcally focused on architectural knowledge that must be complied with at all times across all development sites. This architectural knowledge serves as architectural rules to the respective development sites.
We have researched the role of the architectural rules and the compliance practices that are implemented at a department involved in GSD at Organization A. During this research, we worked closely with the central architecture team. Our activities consisted of analyzing the use of the architectural rules and the role of the architecture team in securing compliance with these rules. We conducted questionnaires and follow-up interviews as data collection techniques and developed recommendations for improved compliance.
This case study exempliﬁed that, apart from a good structure of the architectural knowledge itself, additional process practices are needed to let the architectural knowledge sink in properly and ensure compliance with the architectural knowledge across sites: these practices included frequent communication on the architectural knowledge, documentation of possible deviations, and veriﬁcation of compliance with the architectural knowledge.
To further substantiate the results obtained in the study at Organization A, we compared Organization A with a second organization, Organization B, in 2007. We constructed a comparison model based on the challenges and issues involved in GSD. We used this model to compare how Organization A and Organization B each used architectural knowledge management practices to overcome these challenges and issues.
The results showed that whereas Organization A has a focus towards using rules related to compliance, Organization B more reverts to measures and practices in the process to overcome the challenges and issues involved in GSD.