«This research has been partially sponsored by the Dutch Joint Academic and Commer- cial Quality Research & Development (Jacquard) program on Software ...»
To get the most out of the architectural knowledge of information systems in general, we need to determine how different stakeholders use architectural knowledge. We term these typical uses use cases for architectural knowledge1. Some of these use cases may depend on the roles that stakeholders fulﬁll or the architecture level stakeholders are engaged in. Architects may favour other use cases than designers or technical specialists, and enterprise architecture practitioners may give priority to other use cases than software architecture practitioners. Currently, we determine what information is particularly important for certain stakeholders, by using approaches and standards such 1 As indicated in §1.6.2, this deﬁnition differs from the deﬁnition for the use case technique as part of OMG’s Uniﬁed Modeling Language standard.
as (Clements et al., 2003; IEEE, 2000). Yet, we do lack insight into how these stakeholders use architectural knowledge.
We conducted a survey-based study to address the lack of insight into the importance of architectural knowledge. We designed a survey which includes use cases for architectural knowledge. These use cases are based on earlier work (Kruchten et al., 2006; van der Ven et al., 2006a), experiences in industry, and our own experience.
This chapter reports on the results of our study. We provide insight into the way practitioners in the Netherlands view and use architectural knowledge. In doing this, we reveal the mindset of practitioners with respect to the use of architectural knowledge by listing what uses are important for what roles and on what architecture levels.
Based on the survey results, we make the following observations. Architects regard the architecting process as a forward engineering discipline and do not see clear beneﬁts of reﬂection, assessment, and change of the architecture. Yet, literature argues that these are precisely the intended beneﬁts of architecture (Bass et al., 2003). Apparently, these intended beneﬁts of architecture have not yet been ﬁrmly established in the mindset of architects nor transferred to practice. Furthermore, a forward decision-making process is reﬂected by the mindset of architects, but the value of managing the set of decisions (so, architectural knowledge management) is not yet clear. Finally, the importance of stakeholder communication of the architecture is generally recognized.
The results of this research call for further knowledge transfer on the more innovative concept of viewing architecture as a set of architectural decisions. Furthermore, it is important to quantify the beneﬁts of this concept. At the same time, further research is needed into the foundation for the mindset to identify the activities needed to further establish the concept of architectural knowledge in the architect’s mindset.
The remainder of this chapter is organized as follows. In §2.2 we discuss related work in the ﬁeld of architectural knowledge, design rationale, and architectural roles.
Next, §2.3 describes the design of the research. §2.4 and §2.5 present the ﬁndings and a discussion of their limitations. In §2.6, we reﬂect on the results and provide conclusions.
Finally, §2.7 provides directions for future work.
2.2 Related Work Tang et al. (2006), have analyzed the use and documentation of architecture design rationale. The survey reveals the view of the participants on several generic uses of design rationale. The study shows that although participants regard design rationale as important, they do not capture the rationale. The main reason for this is a lack of appropriate tools to support the architects. Furthermore, the study shows that architects tend to focus on the positive aspects of architectural decisions and design rationale instead of looking for problems in a speciﬁc architecture. We view design rationale as a speciﬁc subset of architectural knowledge (Kruchten et al., 2006). In this chapter, we revert to the use cases for architectural knowledge, initially described in (Kruchten et al., 2006) and elaborated on in (van der Ven et al., 2006a), and provide a detailed view on possible uses of architectural knowledge.
A template for architectural decisions is provided by (Tyree and Akerman, 2005).
24 2.3. Research Design
This template describes a decision, its underlying assumptions, related requirements, and implications. The template is useful for organizing architectural knowledge but does not provide insight into the use of architectural knowledge. Multiple templates or tailoring of existing templates may be necessary to fully support architects in their use of architectural knowledge. Our work can provide input for this. Zimmermann et al. report on a framework that can be useful in identifying, making, and enforcing architectural decisions during the architecting process (Zimmermann et al., 2007).
Smolander (2002) extensively describes the meaning of architecture in practice.
Smolander describes four metaphors for architecture: ‘architecture as blueprint’, ‘architecture as literature’, ‘architecture as language’, and ‘architecture as decision’. The metaphors explain the meaning of architecture in practice. This meaning may differ depending on the role practitioners fulﬁll or the architectural levels they are engaged in. We provide insight into the importance of use cases for architectural knowledge and relate this insight to the metaphors from (Smolander, 2002). Thus, we show what metaphors are accepted and in use by practitioners.
Hofmeister et al. (2000) provides a good description of possible roles and activities of an architect. Examples include a visionary, technical consultant, decision maker, and coach. These different activities could be supported by appropriate use cases for architectural knowledge. Use cases for architectural knowledge that are deemed important for aforementioned roles enable the architect to effectively capture and reason about architectural knowledge. The other way around, the relevance of use cases for architectural knowledge for the practitioners could indicate to what extent the possible roles and activities from (Hofmeister et al., 2000) in fact are established.
Clements et al. (2007) performed a study on publicly available resources to identify the duties, skills, and knowledge of software architects. They show that the role of an architect implies more than only making technical decisions. Rather than focusing on the competences of architects, our study focuses on the use of architectural knowledge to support the architect in achieving and maintaining his competences.
2.3 Research Design We aim to ﬁnd out how practitioners engaged in architecture in the Netherlands view architectural knowledge. This helps us to construct the mindset of architects with respect to architectural knowledge. In order to reveal this mindset, we use a survey instrument. Survey instruments are widely used in software engineering research (Kitchenham and Pﬂeeger, 2001-2002). In survey-based research, a number of factors should be taken into account: the design of the survey, the selection of participants, and how to control the response rate of the survey. The remainder of this section describes how we have addressed these factors.
2.3.1 Survey design The ultimate objective of the survey is to identify the way practitioners view and use architectural knowledge. We took the typical uses distilled from experienced practitioners
within the GRIFFIN research project (Kruchten et al., 2006; van der Ven et al., 2006a) as a starting point. We validated and augmented the list of use cases with the industrial partners in our research project and with our own experience. Next, we reformulated each use case into a clear, self-explaining statement on architectural knowledge. We allowed the survey participants to indicate the importance of each use case using a 5-point Likert scale (Likert, 1932) ranging from ‘not important’ to ‘very important’.
We hypothesized that the importance of certain use cases may depend on the roles practitioners fulﬁll and on the level of architecture that practitioners are engaged in.
We posed some demographic questions and speciﬁcally asked the participants to a) indicate the architectural roles they typically fulﬁll and b) indicate the relative amount of time they spend on certain architecture levels. Using the information on the roles and architecture levels, we constructed two different perspectives to analyze the way respondents perceive architectural knowledge.
As a ﬁrst step, we conducted a pilot with the survey on a focused group consisting of selected employees of one of our industrial partners. We then developed a web-based survey for administration of the complete population.
2.3.2 Selection of participants of the survey We needed to construct a representative subset of Dutch practitioners that play a key role in architecturing activities. To come to this subset, we identiﬁed three dimensions by which we selected participating organizations: domain (such as banking, telecommunications, insurance, governmental), type (IT service providers, software development organizations), and market (commercial, non-commercial). Next, we selected organizations or platforms (such as communities of enterprise architects and embedded architects) in each of these dimensions and identiﬁed practitioners that play a key role in architecture at these organizations. This gives us conﬁdence that we have selected a representative subset of Dutch practitioners to give us feedback on the use of architectural knowledge.
2.3.3 Control of the survey In order to keep control on the response rate, we directly contacted key practitioners at the organizations involved. We enquired their willingness to act as on-site representative for that organization. We sent these representatives the hyperlink to the online survey. The on-site representatives forwarded the hyperlink to knowledgeable colleagues and notiﬁed us of the total number of colleagues involved (snowball sampling (Kitchenham and Pﬂeeger, 2001-2002)).
2.4 Findings This section describes our ﬁndings. We ﬁrst provide demographic information in §2.4.1, after which we discuss the two different perspectives for presenting the results as described in §2.3.1. Next, §2.4.4 describes how we clustered the use cases for architectural
knowledge. After that, we present our main results in §2.4.5: how practitioners view and use architectural knowledge.
2.4.1 Demographic information We sent out the survey to 36 persons acting as respondents and on-site representatives.
They forwarded the survey to 348 practitioners. So, in total, 384 practitioners were reached. We collected 143 responses, of which 107 were complete. This corresponds to a response rate of 27.86 %. We took these as a basis for our survey.
Of the total population, 213 practitioners are employed at one of the large IT service providers included in our survey. We discuss the overrepresentation of practitioners employed at IT service providers in §2.5. We received 75 responses from these practitioners, corresponding to a response rate of 35.21 %. The remaining 171 practitioners are employed at smaller IT consultancy ﬁrms or e.g., IT departments of banks, insurance organizations or governmental organizations. We received 32 responses, corresponding to a response rate of 18.71 %.
2.4.2 Architecture levels Various different deﬁnitions of ‘architecture’ exist (SEI, 2011). Examples include a systems-oriented view and a view focusing on the information ﬂow in or surrounding a software system. In our survey we used concise deﬁnitions from (Florijn et al., 2003)
for the so-called architecture levels:
Software architecture – The structure and relations of a software system.
Systems architecture – The architecture of a single system, taking into account both software and hardware.
Information architecture – The information needs and ﬂows of business functions as they are identiﬁed.
Enterprise architecture – Architecture at the level of an organizational unit or an organization as a whole.
Process architecture – A description of the processes running in or surrounding a software system.
Each practitioner indicated the amount of time spent on a certain level of architecture.
To be able to compare the data collected, we normalized the total amount of time spent by each practitioner to 100 %. The relative amount of time spent on each level of architecture for all respondents is depicted in Table 2.1. We observed that a concrete architecture of a single system (i.e., ‘software architecture’ and ‘systems architecture’) receives more attention than company-wide architectures (i.e., ‘information architecture’, ‘enterprise architecture’, and ‘process architecture’). Other architecture levels that are less often practitioned are grouped into the ‘other’ category in Table 2.1.
Since the population is relatively large, we grouped the respondents into clusters and based our analysis on the clusters instead of on the individual responses. Of course,
architects may work on different architecture levels simultaneously. We wanted to see if these architecture levels are related based on the responses of the practitioners. For example, if architects often work on two architecture levels simultaneously, we group these two levels. Moreover, this enables us to group architecture levels that have different names, but in fact appear to be closely related. In order to group similar or closely related architecture levels, we calculated the correlation between each of the architecture levels. This resulted in an n-dimensional space, n being the number of architecture levels. In order to plot the relative distances between the architecture levels, we reduced the number of dimensions to two using classic multi-dimensional scaling (Borg and Groenen, 2005). Multidimensional scaling refers to representing the architecture levels as points, in such a way that the interpoint distances approximate the speciﬁed dissimilarities. In order to assess the accuracy of the distance, we applied kmeans clustering (MacQueen, 1967) to cluster architecture levels with a small distance.