«This research has been partially sponsored by the Dutch Joint Academic and Commer- cial Quality Research & Development (Jacquard) program on Software ...»
Further, we have to take into account the differences between the two organizations, such as differences in complexity and size (Perry and Kaiser, 1991). Nevertheless, we feel that a number of light-weight practices in place at Organization B can be transferred successfully to Organization A. For example, two-way traveling greatly helps in building a team and, consequently, in having development sites regard each other fully as ‘peers’. In addition, to build a collective sense of urgency, the proportion of architect’s work and developer’s work (regardless of what development site is their stand) could be distributed more uniformly across development sites. The other way round, Organization B may need to place more emphasis on architectural rules in addition to process measures when development scales up. We conclude that rules regulating a combination of both the architectural rules and measures pertaining to the architecture process proves valuable in handling GSD challenges.
72 5 Product Practices for Architectural Knowledge Management in Global Software Development Chapters 3 and 4 have provided insight into how compliance with architectural rules can be secured in global software development organizations. The main ﬁndings obtained in this research indicate that process measures seem prevalent over measures that can be taken in the product. In this study, we determine whether practices related to the product can help in capturing architectural knowledge and serve as measures to increase the quality of the software product.
5.1 Introduction In the discipline of software architecture, increasing attention is paid to capturing architectural decisions and underlying rationale as important means to describe software architectures. In this way, architectural knowledge serves as an enabler for high quality software products: an increased understanding of the ‘why’ behind certain decisions helps to reduce the implementation and maintenance effort of the resulting system (Bosch, 2004; Lago et al., 2010).
The increasing trend towards global software development (Aspray et al., 2006) poses an additional burden towards developing high quality software products, since additional challenges come into play. These challenges include temporal, geographical, and socio-cultural challenges (Holmstr¨ m et al., 2006).
o It has generally been acknowledged that measures in the realm of communication ˚ and coordination strategies can address the challenges involved in GSD (Agerfalk et al., 2005; Clerc et al., 2007b). These measures include speciﬁc practices to share architectural knowledge in a global context, see e.g., Chapter 3. Aside from these measures that focus on the process, measures can be taken in the software product as well; taken measures can serve as a proxy for the quality of the software product. However, we are currently lacking insight into what product-based architectural knowledge measures exactly contribute to the quality of the software product.
735. Product Practices for AKM in GSD
To address this lack of insight, we study relevant literature to identify what key elements constitute architectural knowledge. Using these key elements, we study a series of software product assessments performed by Organization C, an IT consultancy, training, and audit organization, and determine how architectural knowledge measures have been taken in these software products. In addition, we contrast the use of the key elements of architectural knowledge in a series of software product assessments, developed in both collocated and global projects to identify product recommendations for GSD.
Our comparative study did not provide us with very signiﬁcant differences. Yet, we observe that rationale still is not commonly captured in architectural descriptions. In addition, GSD products tend to lack view-based architectural descriptions and do not cover an important quality criterion as performance.
The remainder of this chapter is organized as follows. In §5.2 we list related work in the ﬁelds of architectural knowledge, global software development, and software product assessments. In §5.3 we describe the approach for this research including the key elements of architectural knowledge. Next, we list our results in §5.4. We conclude this chapter with a discussion of the results in §5.5.
5.2 Related Work
Recently, a growing trend towards capturing and managing architectural knowledge to represent the architecture of a software system can be observed in the literature (Bosch, 2004; Avgeriou et al., 2007; Kruchten et al., 2006; Babar et al., 2007). Even more recently, work has been performed to deﬁne what architectural knowledge exactly entails (de Boer and Farenhorst, 2008). When we regard architectural knowledge in the realm of global software development, we observe contrasting views on the amount ˚ of knowledge to be made explicit in documentation, e.g., in (Agerfalk and Fitzgerald, 2006; Ramesh et al., 2006). This contrast is further exempliﬁed by our previous work (Clerc et al., 2007b; Lago et al., 2010). The work reported so far primarily seems to focus on measures related to the development process. Bass et al. (2007a) have identiﬁed a best practice template to describe these process measures. In Chapter 4, we compared two organizations involved in GSD, and found that one organization was very successful by primarily applying process measures.
Existing literature does not so much include measures that can be taken in the software product1. For identiﬁcation of measures present in software products, one may use an auditor’s approach using an explicit frame of reference for an audit, such as the one described in (ISO/IEC, 2000).
1 A software product is deﬁned as a “set of computer programs, procedures, and possibly associated documentation and data” (ISO/IEC, 2000). For this study, we focus on (possible intermediate) tailor-made solutions, since we do not include commercial-off-the-shelf components in this study.
5.3 Context and Approach In this section, we provide the context for this research by describing the tool used by the IT consulting ﬁrm to perform software product assessments. Furthermore, we describe the research approach.
5.3.1 Software quality evaluation framework We have performed research at a Dutch IT consultancy ﬁrm, Organization C, that regularly performs independent software product assessments for its clients. These clients are e.g., banks, IT departments of ministries, and non-governmental organizations. Organization C has laid down its experience in performing over 20 of these assessments in a reusable framework containing evaluation criteria (i.e., a software quality evaluation framework). These evaluation criteria are backed by literature as to positively impact one or more quality attributes (as e.g., speciﬁed in (ISO/IEC, 2001)) and hence serve as measures that can be taken in the software product to increase its quality.
The software quality evaluation framework contains 93 evaluation criteria. Each criterion is described by its name, a description, a motivation, an assessment method for the criterion, and inferred consequences of applying the criterion. In addition, the framework deﬁnes the scope of the criterion (e.g., “does the criterion apply to the software product as a whole, or to subsets, such as documentation or source code?”). Furthermore, the framework relates each evaluation criterion to certain quality attributes. We
provide an abstracted example of a evaluation criterion used by Organization C below:
Name – No code duplication.
Description – Software code should not contain unnecessary duplication.
Motivation – Code duplication increases the change effort of the software product;
changes need to be made at multiple locations in the source code.
Evaluation method – Use automatic code analyzers to detect code duplication.
Consequences – Removing code duplication may lead to refactoring.
Quality attributes affected – Analysability, Changeability, Stability, Testability.
5.3.2 Research approach Since we are primarily interested in how architectural knowledge can help to improve the quality of software products, we focus on the evaluation criteria that speciﬁcally deal with the topic of architectural knowledge. We deﬁne key elements of architectural knowledge by researching deﬁnitions of architectural knowledge posed by relevant literature (see §5.3.3). Using these key elements of architectural knowledge, we study the set of evaluation criteria in the software quality evaluation framework and select those criteria that, when applied, make explicit one or more key elements of architectural knowledge (see §5.3.4).
Using the subset of the resulting so-called architectural knowledge evaluation criteria, we analyze eight pre-selected software product assessment reports to determine
what key elements of architectural knowledge are applied in the products. These assessments concern four products that have been developed using GSD and four products that have been developed on a single site. The result of this analysis is presented in §5.4.
5.3.3 Key elements of architectural knowledge To identify what key elements constitute architectural knowledge, we performed a thorough analysis of deﬁnitions listed in relevant literature. We identiﬁed relevant literature by focusing on the reports produced in the SHARK workshop and the WICSA conference series, both since 2006. We searched these articles for deﬁnitions of architectural knowledge. Furthermore, we included recent work that describes a systematic review on how architectural knowledge is deﬁned and how different deﬁnitions are related (de Boer and Farenhorst, 2008). In their paper, De Boer and Farenhorst list deﬁnitions of architectural knowledge from fourteen papers, collected after synthesis of a total set of 751 found initially. While analyzing all collected deﬁnitions, we identiﬁed similar parts in these deﬁnitions. These similar parts included synonyms like “architectural decisions” and “architectural design decisions”, or “design” and “architectural design”.
We mapped the synonyms identiﬁed onto a single key element of architectural knowledge. The resulting list of key elements of architectural knowledge is shown in Table 5.1.
5.3.4 Architectural knowledge evaluation criteria Following the approach described in §5.3.2, we selected those evaluation criteria from the software quality evaluation framework that concern architectural knowledge. This resulted in fourteen out of 93 evaluation criteria to be selected. Each of these fourteen evaluation criteria make explicit at least one key element of architectural knowledge.
For reasons of brevity and conﬁdentiality we do not provide an exhaustive description of the evaluation criteria. Rather, we list the names of the selected architectural knowledge evaluation criteria in Table 5.2.
Table 5.2: Architectural knowledge evaluation criteria
The selected evaluation criteria focus on either building blocks of architecture (e.g., Des, Env, and Dom in Table 5.1) or on the decision-making process and its results (e.g., Ass, Dec, Dep, Dom, Pat, Alt, Rat). In addition, we observe that the quality attributes affected most by the selected architectural knowledge evaluation criteria are ‘analysability’, ‘changeability’, and ‘manageability’. Each of these quality attributes are positively impacted by nine of the fourteen selected architectural knowledge evaluation criteria.
5.4 Results This section lists the results of our research. First, we provide a brief characterization of the context in which the selected software products were developed in §5.4.1. Next, we provide the results of our evaluation in §5.4.2 by providing a scale and contrasting the products that were developed using global software development with products that were developed using collocated development.
5.4.1 Characterization of the selected software development projects We have included eight software development projects as a basis for our study. These projects focused on the development of business administration systems and not on e.g., commercial-off-the-shelf products. The evaluations of these projects were performed in the period late 2004 until late 2007. Four of these projects took place using multiple,
775. Product Practices for AKM in GSD
globally distributed development sites. The remaining four were developed either at the customer’s location or at a single location of the IT service provider developing the product close to the customer. The global software development projects included 12 project members on average (range 3–23); the others included 5 project members on average (range 3–12). Programming languages and technologies in use included.Net (C#), Java, Oracle, and various workﬂow engines.
5.4.2 Evaluation results We studied the assessment reports of the eight selected software product assessments to determine to what extent the evaluation criteria are complied with in the software products. Since the assessment reports did not provide us with a single, uniform scale, we devised an ordinal scale to identify the degree of implementation, see Table 5.3. This scale does not only focus on whether a criterion has been applied, but also on the level of description of the application, since that information is key to transferring knowledge to other project members – especially in the context of global software development.
Consequently, this scale serves as re-alignment of the auditors’ judgment over time.
Regarding the scale: several reasons may exist for rating the implementation of an evaluation criterion with ‘+’. For example, the rate ‘+’ is given if either the description of the application of the criterion is incomplete, or if the description is unclear in the perception of the auditors.
Table 5.4 lists the results of the software product assessments that have been evaluated.