«Abstract This paper revisits the debate on the units of Content Analysis (CA) for the purposes of Corporate Social Reporting (CSR) research and also ...»
It has been argued, however, that the focus on the employment of Annual Reports may result in an incomplete representation of the quantum of CSR (Zéghal and Ahmed, 1990) since these are not the only medium through which companies report their socially responsible behaviour, a view empirically supported by Unerman (2000), Frost et al. (2005) and Guthrie et al. (2008). Erusalimsky et al’s (2006) affirmation that “in the years since 1990… the rise in the number of voluntary 8 But note that “the exclusive focus on the corporate annual report has been accused of being selfsustaining given that its exclusive choice is often justified in the literature on the grounds that this is what most other studies have done” (O‟Dwyer, 1999, p. 229, a justification supported by Unerman, 1998).
- 11 standalone reports has been astonishing” (p. 12) and Guthrie et al‟s (2008) finding that “companies may use the annual report and corporate website for reporting different types of information” (p. 40) gives further support to Unerman‟s (2000) conclusion that “future studies focusing exclusively on annual reports might not produce particularly relevant results” (p. 674).
As a consequence, a number of CA studies now increasingly employ CSR sources in addition to or other than the Annual Reports as sampling unit. Quite frequently standalone reports are employed (see e.g. Laine, 2005; Jones, 2006; Turner et al., 2006; Bebbington and Larrinaga, 2007), although concerns have been expressed for the use in their analysis of the same protocols employed in the Annual Reports (Erusalimsky et al., 2006); and a number of studies also employ information from the internet (e.g. Esrock and Leichty, 1998; Adams and Frost, 2004; Patten and Crampton, 2004; Unerman and Bennett, 2004; Turner et al., 2006; Guthrie et al.
2008), a data source easier to be captured by the index rather than the volumetric approaches to CA, given its volatile nature (but note that Patten and Crampton, 2004, printed out all the environmental disclosures identified and also applied a volumetric approach). Notably Zéghal and Ahmed (1990), Tilt (1994) and Unerman (2000) have employed a much wider array of CSR sources as sampling unit, including e.g.
brochures and advertisements and several internally circulated bulletins and other ad hoc documents published in each year (see also Lewis et al., 1984, for a study considering publications to employees).
Nevertheless, as Unerman (2000) points out, “a limit must be set to the range of documents included in any research study… [due to the risk of] a researcher being overwhelmed by the number of documents… [and of not being] possible to ensure completeness of data” (p. 671. See also Campbell, 2004, for an account of feasibility factors that might also be at play). Further, it is almost impossible to identify all corporate communications that could possibly contain CSR information (O‟Dwyer, 1999; Guthrie et al., 2004) and it thus seems similarly impossible to identify all the CSR activities of the examined organisations. It, therefore, seems justifiable for studies to employ Annual and standalone reports as the sampling unit as these should contain the bulk of the disclosed CSR information. Ultimately, the sampling unit for each study depends on the set research objectives (Unerman, 2000) and also on the
Context units Context units are defined by Berelson (1952) as “the largest body of content that may be examined in characterizing a recording unit” (p. 135). As Krippendorff (2004)
“the meaning of a word typically depends on its syntactical role within a sentence. To identify which meaning applies to a word from a list of dictionary entries, one must examine the sentence in which the word occurs.
How else would one know, for example, whether the word go is meant to denote a game, an action, or a command? Here, the sentence is the context unit and the word is the recording unit” (p. 101).
Unlike sampling units, which appear to be a widely accepted and undisputed term, “much of the discussion on the „unit of analysis‟ confuses the issues of what should form the basis for coding with what should form the basis for measuring or counting the amount of disclosure” (Milne and Adler, 1999, p. 243, see e.g. Gray et al.‟s, 1995b, assertion that “Sentences are to be preferred if one is seeking to infer meaning… [but] Pragmatically, pages are… easier… unit to measure by hand”, p. 84, also Guthrie et al., 2008 for similar arguments). Part of this confusion may be generated from the use of different and at times opposite terms from researchers (e.g.
Milne and Adler‟s (1999) coding units are what Krippendorff (2004) describes as context units and Neuendorf (2002) as analysis units, whilst Milne and Adler‟s measurement units are Krippendorff‟s recording/coding units and Neuendorf‟s (2002) data collection units, see also Walden and Schwartz ; Unerman [1998, as cited by O‟Dwyer, 1999, pp. 229-232]; O‟Dwyer , for alternative interpretations). In this paper, Krippendorff‟s (2004) terminology is adopted, in accordance with a number of other prominent CA theorists (Grey et al. 1949; Berelson, 1952; Osgood, 1959; Stone et al., 1966; Weber, 1985; 1990).
- 13 Most of the CA studies refer to a single unit of analysis, that being most frequently the recording unit for the volumetric studies and the context unit for the index ones.
This is not surprising given that in the index studies where most frequently the recording unit is the presence or absence of specific information, the unit that usually needs attention is the context one (see e.g. Patten and Crampton, 2004, pp. 39-40, for an illustrative account on how sentences were used as a basis for coding for their index). In contrast, for volumetric studies the recording units seem to be of more importance, whilst for the context units, since unlike sampling and recording ones they “are not counted, need not be independent of each other, can overlap, and may be consulted in the description of several recording units” (Krippendorff, 2004, p. 101) researchers do not often explicitly discuss their choices (but see Zéghal and Ahmed,
1990) and meaning is coded perhaps by section heading, phrase or sentence (Buhr, 1994; Campbell, 2004).
There seems to be no logical limit for the size of the context units: As Krippendorff (2004) notes “sentences are the minimal context units for individual words, but sentences may not be enough” (p. 101) and at times, when e.g. decisions on a positive or negative context of a commentary are made, “analysts might need to examine even larger context units, such as a paragraph or a whole speech.... The best content analyses define their context units as large as is meaningful (adding to their validity) and as small as feasible (adding to their reliability)” (ibid., pp. 101-102). With regards to this, Milne and Adler (1999) demonstrate that “sentences are far more reliable than any other unit of analysis” (p. 243) and further assert that “Most social and environmental content analyses in fact use sentences as the basis for coding decisions” (ibid.).
Naturally, researchers need to clearly define their context units before commencing recording them. Abbot and Monsen (1979) seem to consider a prominent type of error in CA to be “the formulation of categories that do not reflect all the issues actually contained in the report that are of policy interest” (p. 506). As Holsti (1969a) points out, “categories should reflect the purposes of the research, be exhaustive, be mutually exclusive, independent, and be derived from a single classification principle” (p. 95, emphasis in original). It seems particularly useful in CSR research to establish
- 14 clear rules as to what consists CSR and what does not, a problem largely stemming from the variety of definitions of the field, that are “generally too exclusive… or too all-embracing” (Gray et al., 1995b, fn4, p. 89)9. As Milne and Adler (1999) empirically attest for inter-rater reliability, “by far the greatest proportion of disagreements concerned whether a particular sentence was or was not a social disclosure regardless of the coder. If coders had agreed to a sentence being a social disclosure (regardless of which theme) they were relatively unlikely to disagree over which theme, what sort of evidence and what type of news characteristics the sentence contained” (p. 252).
In the literature four major „themes‟ (what Holsti, 1969a refers to as manifest subject of information, not inductively generated) for CSR are employed: marketplace (consumers, creditors), workplace (employees), community, and environment, but there will always be a need for a development of an „other‟ category (Gray et al., 1995b). This classification was employed in the coding spreadsheet of the BA study, which for illustration purposes is depicted in Figure 1. This spreadsheet was based, among others, on Ernst and Ernst (1978) and Gray et al. (1995b) CSD classifications, as well as those of the GRI (2002), which Erusalimsky et al. (2006) consider, along with AA1000, to “perhaps represent the very base, the entry-level for analysing reporting in an even vaguely serious manner” (p. 19)10.
9 This variety of definitions is also reflected in the number of alternative terms that at times have been offered to even term the field, including: Corporate Social Reporting (Gray et al., 1988; 1995a); Social Accounting (Gray, 2002; Gray et al., 1997); Social and Environmental Accounting (Mathews, 1997;
Gray, 2006); Social and Environmental Accounting and Auditing (Owen, 2004); Social and Environmental Accountability (Parker, 2005); Social Responsibility Disclosure (Trotman, 1979; Neu et al., 1998); Corporate Social and Ethical Reporting (Adams, 2002); Ethical Reporting (Adams, 2004);
Despite that these, as Epstein (2004) notes, “are all terms used to describe the measurement and reporting of an organization‟s social, environmental, and economic impacts, as well as society‟s impacts on that organization, including both positive and negative impacts”, and that the adoption of a specific term for CSR may sufficiently serve the short-term needs of a „pragmatically–oriented‟ research paper, it should be noted that this also adds to the long-term confusion over the terminology and the (much sought [Parker, 2005]) conceptual framework of the area. Naturally, the adoption of different definitions and resulted context categories further prohibits meaningful comparative analyses of the findings of the papers.
10 Note that this was a multiple classification protocol, in that each coding unit was classified into more than one category, based on the condition that each entry may have more than one attribute. It should be further noted though that the gain from the adoption of this multiple classification in semantic precision does not outweigh the potential losses from logical distinctiveness and exclusiveness since “not all entries need have the same attribute to the same extent” (Weber, 1990). This is particularly the case when latent rather than manifest classification schemes are examined, as further discussed in section 6.
- 15 Further, it seems reasonable to take a more „pragmatic‟ rather than „classical‟ approach and suggest that the categories of each study should also reflect the set research objectives: in the BA study the categories reflect the focus on the CSD effects of aviation accidents and subsequently the Health and Safety disclosures, whilst for Deegan et al. (2002) the categories reflect their focus on Human Resources.
Holsti (1969a) considers “reflect[ing] the investigator‟s research question… the most important requirement of categories” (p. 95) and warns that “Unless theory and technique are intimately related, even the minimal requirements of validity cannot be met” (p. 94). This however should not necessarily be the case for general CSR surveys, where the adoption of a more „standardised‟ approach may increase comparability and cumulative research (Berelson, 1952).
Take in Figure 1
Further consideration of the CSR subject categories is beyond the scope of this study.
However, as indicated in Figure 1, when reviewing and deciding on the context units all the alternative ways to define CSD need to be set and clearly defined. For the BA study further distinctions on the type of disclosure (positive or negative) and the possible underlying corporate strategy (3 substantive and 3 symbolic) as well as on whether CSD was mandatory or voluntary and narrative or non-narrative were included. This resulted in overall (19 choices of theme) x (7 choices of strategy) x (3 choices of type) x (2 mandatory/voluntary) x (2 narrative/non-narrative) = 1,596 possible choices for coding each individual CSD. These distinctions will be discussed more extensively in the related findings section.
Recording/coding units Recording/coding units are defined by Holsti (1969a) as “the specific segment of content that is characterized by placing it in a given category” (p. 116). As
Krippendorff (2004) elaborates:
Whilst for the basis for coding researchers seem to concede that “Sentences are to be preferred if one is seeking to infer meaning” (Gray et al., 1995b, p. 84, see also Milne and Adler, 1999; Unerman, 2000), with regards to the basis for measuring, “once the content has been coded… quantification may be done in a number of ways” (Milne and Adler, 1999, p. 243). Indeed, researchers have at times employed a variety of different approaches to measurement, often justifying their choice on the empirical evidence (Grey et al., 1949; Patten, 1992; Deegan and Gordon, 1996; Deegan and Rankin, 1996; Hackston and Milne, 1996; Williams, 1999; Campbell, 2000) which gives support to the suggestion “that measurement error between various quantification techniques is likely to be quite negligible” (Milne and Adler, 1999, p.
243). However, as now illustrated, each unit has its distinct advantages and disadvantages which need to be considered when selecting units and interpreting results.
The four types of recording units considered here are words, sentences, proportion of pages and page size data11. A summary of a number of issues of concern drawn from the literature regarding these units is provided in Table 3.