«Citation: Dalkin, Sonia (2014) The Realist Evaluation of a Palliative Integrated Care Pathway in Primary Care: What Works, For Whom and in What ...»
The explanatory mechanism is the most characteristic tool of realist evaluation and is often referred to as underlying mechanism or generative mechanism (Pawson and Tilley 1997, Pawson 2006). The term ‘mechanism’ relates to causation. Processes that produce events, or patterns of events, can be seen as causal mechanisms. Mechanisms explain causal relations by describing the powers built in to a system, including the reasoning of stakeholders (such as health care professionals, patients, carers or bereaved relatives), and resources of the social programme (such as ACP, palliative care registration, peer support from MDT meetings) (Pawson 2006). Yet in all cases, it is something about context and a combination of resource and reasoning which form a mechanism with explanatory potential for the observed outcome. Therefore the mechanism explains what it is about the system that makes things change (Pawson 2006). Mechanisms can often not be directly observed, and the evaluator must hypothesise which mechanism is likely to have ‘fired’ and then test this theory with data. For example, if it is thought that the number of patients who have anticipatory medication is higher (outcome) in practices (context) where the ICP is more embedded and adopted as routine practice (mechanism), then the degree of how embedded the ICP is must be investigated by the evaluator, despite embeddedness not being an explicitly measurable factor.
It is also important to note that social programmes offer resources, however it is the reasoning of the subjects in combination with the resources provided by the social programme that result in outcomes (Pawson and Tilley 1997, Pawson 2006). Social programmes only work if the people involved choose to make them work by adhering to the programme theory (reasoning) and using the resources as intended (Pawson 2006).
The acknowledgement of reasoning is one of the key strengths of using realist evaluation (together with the importance of context), as most other evaluations assume a relation of straightforward causality between the resources introduced by an intervention and the outcomes observed. However, as with most practice development efforts, interpersonal relationships between health care professionals and patients embody the intervention. They are the resource that is intended to bring about change (Entwistle and Cribb 2013). Thus, in order to help understand the mechanisms of the ICP in detail they will be unpacked in terms of reasoning and resources throughout the thesis. This is represented in Figure 1 below, as well as throughout the findings. Taking the example about how embedded the ICP is, we can now ask what are the resources and reasoning of this mechanism? Resources could be informed practitioners who have access to anticipatory medication. The reasoning is the decision to provide the anticipatory medication in advance of a decline in health requiring the medication, and the outcome would be an increase in the use of anticipatory medication.
In realist evaluation outcomes are not a sufficient base for establishing causality, yet they are important in science (Pawson 2006). In social science, strict regularities are not always viable, as they are in engineering or chemistry where total control of variables is the objective. Therefore, evidence based policy would aim to choose an intervention that has a high chance of repeating the positive outcomes achieved elsewhere. To do this, outcome patterns must be sought rather than outcome regularities. Significant outcome patterns are embedded and dependent on the introduction of not only suitable ideas and interventions (mechanisms) but also the appropriate existing social and cultural conditions (contexts). In metaphoric terms, causality is thus attributed to the right substance being activated in the right conditions (Pawson and Tilley 1997).
How is a social programme evaluated?
Social programmes provide resources (such as ACP, the traffic light system of wellness, the new palliative care unit), which activate people’s reasoning (M). However, the activation of the mechanism is dependent on the characteristics and circumstances of subjects, situations and societies (C), resulting in a varied pattern of impact (O) (Pawson 2006), as detailed in Figure 1. These three concepts are the crucial sources of evidence in realist evaluation. Realist evaluation does not ask if a social programme works, instead it focuses on the fundamental question, “what works for whom, in what contexts, in what respects and how”? Thus, in order to evaluate social programmes, the theories within a programme must be made explicit, by developing clear hypotheses about how, and for whom, programmes might work. This is done by identifying context (C), mechanism (M) and outcome (O) configurations (CMOC) because causal outcomes following from mechanisms acting in contexts is the base upon which all realist explanation builds (Pawson and Tilley 1997). A CMOC, as depicted in Figure 1, is a suggestion that states what it is about a social programme which might work, for whom, in what circumstances. A programme theory or initial CMOC is the starting point for evaluation, and refinement of this CMOC through data analysis leads to the concluding finding of an evaluation (Pawson and Tilley 1997). This is an iterative process. In order to construct and refine CMOCs, evaluators need to engage with policy makers, practitioners and participants (Pawson and Tilley 1997). This is especially important when refining CMOC. Since data collection needs to provide evidence for CMOCs and engage with several different forms of participants, it must be wide reaching and varied in content and pitch. It will not only concentrate on impacts but also on the process of implementation, context and underlying mechanisms that may lead to changes. This is why integration of the researcher into the locality was so important in this project.
Knowledge of the locality and different GP practices involved with the ICP was essential in understanding and refining the initial programme theories.
Although the findings from this study will be specific to the locality, they will have translational potential. They will offer explanatory potential to the broadest of palliative and end-of-life care scenarios, thus enhancing transferability. The study findings will enhance understanding of the crucial mechanisms within palliative and end-of-life care and how resources put into specific contexts lead to specific outcomes, both intended and unintended.
Figure 1: Context, Mechanism and Outcome Configurations in realist evaluation Traditional research formats aim to demonstrate a relationship of causality between resources inputted by a social programme and observed or measured outcomes. As exposed in ‘Mechanism’ above (pg.83) realist thinking posits that this causality is mediated by the context within which the resources are implemented and the reasoning this triggers in key stakeholders, which results in them acting in a different way and leads to observable outcomes. Causality is thus more complex, and reliant on often unobservable and unmeasurable parameters, which the realist evaluator endeavours to shed light on.
In order to make explicit the distinction between the intervention components (visible and known to practitioners) and the kind of reasoning this triggered in certain contexts (assumed or implicit), it was decided to disaggregate resources and reasoning throughout the thesis, as demonstrated in Figure 1. This is in order that readers from all backgrounds and prior knowledge of realist thinking can engage with the concept of ‘causal mechanisms’. Figure 1 depicts how resources are introduced into pre-existing contexts which in collaboration result in reasoning of an individual which leads to an outcome. The reasoning border has softer lines to identify its psychological nature. The outcome box has a thick box surrounding it to depict its observable nature.
In the description of CMOC throughout the results chapters they will be described in the following format: outcome, mechanism then context. This is because the inquiry usually started from an outcome. From here, relevant mechanisms and contexts that might have led to the outcome were identified. The only exception to this is Chapter 7 (p.231) where the presentation is context, mechanism, then outcome. This is because inquiry in this part of the thesis began with context.
Research questions and programme theories In terms of the ICP, what works, for whom, in what circumstances?
The main research question stated above was very broad to allow the formulation of programme theories. Programme theories are the underlying assumptions about how an intervention is meant to work and what impacts it is expected to have (Pawson, Greenhalgh et al. 2005). There is usually more than one programme theory per intervention and they differ from normal theory in that they include the two additional key components from realist methodology, context and mechanism. The programme theories in this study were prompted by outcomes of the ICP, the palliative care ICP literature, integration in the locality and consultation with practitioners implementing the ICP. Data available from the GP practice systems (MIQUEST and Death Audit) guided the formulation of the programme theories, as it provided crucial outcomes. The literature identified gaps in knowledge and highlighted why a realist evaluation was appropriate to address these, as exposed in Chapter 2 (p.40). It also contributed towards the initial programme theories. Furthermore, there was a lot of learning that occurred as a result of integration into the locality; in order to develop programme theories for the ICP immersion in the field was essential. This was achieved through observing consultations between GPs and palliative care patients, attending Palliative Care Partnership meetings, local research and education meetings, locality organised educational workshops, PCQVs, spending time at the locality palliative care unit and reading the ICP business cases and reports. Additionally, close contact with the ICP founder was established. This learning allowed my knowledge of the ICP and the locality to increase and formulate testable programme theories. Five programme theories were developed for the ICP. They are stated at the beginning of each Findings chapter and refined using the data collected; the refined programme theory is then stated at the end of each chapter. The five programme theories are stated below and
accompanied by subsidiary questions:
Programme theory 1 Integration into the locality has resulted in a basic knowledge of how the fourteen different GP practices work, the ethos they have in relation to the ICP and the outcomes they achieve. From this knowledge and literature surrounding communication, coordination and team working in palliative care ICPs (Smith 2012), the following programme theory was developed.
The number of people who die in their chosen location (outcome) will depend on the GP practice (context) they are registered with and how embedded the ICP is as indicated by the number of interventions used per patient (outcome). Thus, this is a programme theory about the process of implementation, considering implementation as an intermediary outcome.
- Does the use of more interventions result in better outcomes?
- Do CQI initiatives increase intervention use?
What characteristics do ‘high performing’ GP practices have?
Programme theory 2 The GSF states that palliative care patients should be identified early (The National Gold Standards Framework Centre 2009). Bower, Roderick et al. (2010) highlighted that using the GSF empowered health care professionals to identify patients with any illness early in their trajectory. From this knowledge and integration into the locality the
following programme theory was created:
Palliative care registrations should increase (outcome) due to a focus on identifying patients early using the palliative care register (mechanism) in a health care domain that appreciates the palliative care needs of patients (context).
- Are palliative care registrations increasing in the locality and if so why?
- Are both cancer and non-cancer patients appropriately put onto the palliative care register?
Programme theory 3 The ICP promotes the use of preference discussions with patients and ACP. ACP is also explicitly advocated by UK policy in The End of Life Care Strategy (Department of Health 2008, Addicott and Ross 2010) and has been used in previous studies that have produced positive outcomes for patients (Hockley, Watson et al. 2010, Hall, Goddard et al. 2011, Reymond, Israel et al. 2011). Early integration into the field highlighted an increased awareness of the importance of documenting preferences. This prompted the
following programme theory:
There will be an increase in the use of preference discussions and ACP (outcome) as health care professionals become more confident with broaching the subject of death and dying with patients (mechanism) and aware of the importance of having and documenting preference discussions, which has been highlighted by recent policy (context).
- Are preference discussions increasing and if so why?
Are preference discussions occurring earlier in a patient’s illness trajectory green traffic light phase)?
- Is the number of locality advance care plans carried out with patients increasing and if so why?
- Do preference discussions predict the use of advance care plans?