«A rapid evaluation of SHREWD: the Single Health Resilience Early Warning Database Erica Wirrmann Gadsby Linda Jenkins Stephen Peckham Centre for ...»
Monitoring the situation There is a general consensus that SHREWD is giving a better overview of what is happening, when information is uploaded reliably and frequently. This is achieved without having to ask administrative staff to chase around for the figures, and by having the key information in one location. Great importance is placed, however, on the timeliness of the information. SHREWD appears to allow some people (higher level managers) to monitor the situation better, but those at operational level stressed that the information on the dashboard is only a snapshot of a particular day. This snapshot is not always an indication of how the rest of the day will go. One interviewee felt that the dashboard would need updating every four hours or so to really reflect the current situation. The implementation of automatic data upload is likely to be of significant benefit.
Interviewees felt that it is possible to look at SHREWD to see whether pressure is building, and most explained that they will look at the dashboard most mornings (although most only during winter), to get a brief overview, and to see if there are any pressure points. This has been encouraged by senior level leadership, focusing attention on the use of SHREWD, and getting people into the habit of checking it. It was felt that the simple colour coding system is working to a certain level – if someone is in red, it prompts questions: why are you in red? Do you need any assistance to manage the pressures? And so on.
It was felt that SHREWD helped with monitoring the situation last winter, particularly through the cross-working and teleconferencing. “However, the system is not yet trusted enough and organisations do not want it forced on them” (strategic level interviewee).
Making strategic decisions
The implementation of SHREWD so far has focused on its role in winter planning, rather than during emergency situations. However, its role in preparing for and responding to emergencies has clearly been foreseen by its developers. Several interviewees felt it has the potential to help make decisions, but it would be wholly dependent on having timely and trustworthy information. Several interviewees at operational and tactical levels felt that there would not be time to continuously update SHREWD during an emergency. One raised a question about whether the system had been ‘stress-tested’, to ensure it could cope with this continuous updating by a large number of users at the same time. It was generally felt that the system would need to be used in several exercises before “being used in anger”.
Whilst SHREWD has not yet been used in an emergency, several interviewees talked about its role in preventing a pressured situation escalating to an emergency, by improving access to and sharing of information. There was a feeling amongst several interviewees at strategic level that SHREWD has helped people to make decisions when things are coming under pressure.
However, no concrete examples were provided, and these discussions were largely theoretical, rather than based on actual experience.
It was felt by some interviewees that SHREWD helps strategic decision making on a number of levels. One interviewee in primary care felt that using SHREWD has made him more aware that
he is working in a system:
“At the end of the day it’s all about patient flows from primary care to secondary care, so it does make you more aware of what is going on, and the difficulties that they are having” (tactical level, primary care).
He felt that this greater awareness of the whole system, and the patient flows, could enable better, more informed decisions to be made. A community trust interviewee felt that SHREWD had stimulated new ways of working more effectively across organisations – she was getting calls from other organisations that wouldn’t have happened if they were not able to view and respond to SHREWD data.
One interviewee in commissioning felt that, as long as the information in the database could be regarded as ‘true’, SHREWD definitely helps them to think more strategically about managing system pressures. He explained the example of delayed transfers of care, and how accurate information in the database would allow them to see how many patients are where, and to see where the blocks are. He explained (hypothetically) that this would enable them to investigate why the blocks are there, so they could then exert the appropriate pressures to help change things. This sort of response could be seen in action during the winter planning exercise, and was also observed to a limited extent during the observed teleconference calls. In Medway, where the minutes and actions function of SHREWD is more consistently used, examples of these decision making processes can also be seen relating to dealing with winter pressures in 2011/12, where increased pressures in one part of the system prompted actions in other parts of the system, either to further examine why the pressure was building, or to help alleviate the pressure by altering patient pathways or increasing/moving resources.
It was clear that different organisations would see more or less benefit from using SHREWD. An interviewee in the council, who felt that his organisation was providing information purely to reassure his NHS partners that they are still able to “service the machine”, felt that whilst it is not difficult to manage pressures within their own organisation without having the dashboard and the teleconference, it makes good sense for big organisations like ambulance and acute trusts.
When looking at the health economy as a whole, there was broad agreement that the benefits provided by SHREWD would enable more informed decisions to be made at a strategic level.
However, it was felt that the system (or the users of the system) had not achieved this yet. One interviewee felt that this might be because the right people (accountable officers and chief operating officers in CCGs) are not particularly engaged with it. Another felt that this objective can’t be achieved until there is sufficient data and a track record of the system’s use, so that it is believed and relied upon.
Two interviewees felt that it was too early to say whether SHREWD has yet been useful in strategic decision making, but felt that real lessons will be learned when they are able to look back over previous years’ data, to look at trends and patterns over time. Several interviewees also talked about the added benefit for strategic decision making of having SHREWD data
combined with data from the future 111 service:
“One of the things that will do for us is highlight where there are gaps in the system.
…The combination of data might highlight how we might want to redesign pathways” (strategic level interviewee).
Reflecting on the way system pressures are handled There were mixed views about whether SHREWD is yet enabling people to reflect on the way system pressures are handled. However, there was a greater consensus about its potential to do so. It was felt that having everything recorded in one place is a benefit, since it can provide an ‘audit trail’, rather than forcing a reliance on people’s memories. It was also felt that this will become more obvious as they get more data into the system, and when they can look back over previous years to examine what happened during periods of escalating pressure to see if any lessons can be learnt, or to use it to predict pressures they faced before. Observations of the SHREWD database also demonstrate that users in Medway have started to use a ‘lessons learnt’ section, although it is too early to say whether these lessons are followed up, or lead to improved decision making in the future.
SHREWD’s impact on efficiency and productivity
Efficiency and productivity are important as they are goals of the QIPP (Quality, Innovation, Productivity and Prevention (QIPP) programme that aims to meet the efficiency challenge across the health system while maintaining or improving quality. SHREWD can help with the QIPP agenda as it is an innovative tool that has the potential to deliver cost savings by enabling more effective and efficient sharing of information about health services demand and capacity. One of the key objectives of SHREWD was to streamline the complex process of daily/regular multiagency conference calls and free up time to identify and respond to pressures. SHREWD expects to do this by enabling those on the conference call to access the key data in advance of and during the teleconference meeting, and by providing a tool that supports E conference calls and provides a system for recording minutes and actions. Other key objectives were to create greater system resilience at no additional cost and provide training without cost or impact on staff time.
Interviewees were therefore asked about the cost implications of developing and using SHREWD, which covered system design, on-going maintenance and licence fees. We also asked about the additional time staff had needed to spend to implement and use SHREWD and the resulting savings in time or gains in efficiency they had seen. Collecting and reviewing detailed cost data was not part of this evaluation, so the findings are based on people’s perceptions of costs and time saved.
Costs of developing SHREWD
Few interviewees were able to comment on the cost of developing and maintaining SHREWD.
Considerable time had been invested in Medway to develop SHREWD from the early paperbased version, although some of the time-consuming work carried out was felt to have been necessary anyway (such as identification of key indicators and trigger points). Up-front development costs were subsidised by Transforming Systems. Continuing costs were perceived to be small, although the implementation of significant changes to the system (such as the rollout of automated data input) was recognised to incur up-front costs, mostly in the form of staff time. What is not clear from this evaluation is the extent to which the early development costs (largely in the form of time) can be reduced as the system is rolled out to new areas. The experience of rolling out SHREWD in Kent shows that there is a balance to be made between replicating the whole process undertaken in Medway (which would take considerable time), and adopting the Medway system, making adaptations where necessary (which made developing ‘ownership’ of the new system difficult, and led to some feelings of resentment that a system developed by someone else was being ‘dropped’ on them).
Staff time required
Before SHREWD could be used additional staff time had to be spent uploading data, getting relevant staff trained and in agreeing local indicators and triggers. Several people said this was a valuable exercise that had to be done anyway for their winter plans so did not see it in a negative light. Uploading data was not seen as an onerous or significant additional task, and most felt the training was very quick, simple and possibly not even needed. One person commented that spending time on development and implementation took time away from their main job.
Efficiencies and savings in staff time
Some operational level staff said they had had fewer chasing and checking phone-calls to field after they started putting their data on SHREWD. It was also said that in future this group would spend less time uploading data as these processes became automated, and that both these factors would be a benefit at times of extreme pressure. Using SHREWD was therefore expected to save time for the operational staff involved even though amounts were likely to be quite modest.
Tactical level staff said the main saving was in the length of teleconference calls which had roughly halved in Medway (where SHREWD was being used most effectively). Since calls occur twice a week during winter, and might increase to daily calls during times of escalated pressure, this identifies a significant time saving for a number of senior level staff. Having the data visible, and being able to add agreed actions and minutes also saved time for tactical level staff, who reiterated the view that having the data in one place saved them time in ringing around. They believed the conference calls had become more efficient as everyone started better informed;
they were able to understand the pressures other organisations were under and therefore were able to make better decisions. One or two people did not find SHREWD was any better than using paper and pencil or offered any advantage over other IT systems they had in place.
However these views were expressed by people who had made limited use or felt that SHREWD had been forced on them, so may have been based on perceptions rather than experience.
Staff at the strategic level felt strongly that when SHREWD was used properly the process of planning and managing winter pressures would be quicker and require less of senior managers’ time. However there was the feeling that the extent to which staff time was saved by SHREWD still needed to be demonstrated. One commented that trusts stood to make easy gains by moving patients to less costly community beds. An alternative side of this was mentioned by another interviewee, however, who commented that at a time when block tariffs are being replaced by payment by results tariffs, SHREWD may help non-hospital providers to demonstrate their worth to commissioners.
Lessons learned from implementation and use It is possible to draw out factors that have emerged that relate to the successful implementation
of SHREWD. Sites making most use and getting most benefit had the following characteristics:
- Introduced at a time when it was clearly needed (flu pandemic)
- Getting executive-level buy-in and funding
- Making a detailed assessment of need
- Modelling SHREWD on a system that already is effective
- Having responsive IT support
- Taking up to two years to agree indicators and trigger levels
- Having strong leadership
- Having training for all users that includes the overall purpose of SHREWD as well as how to use it
The following factors may be barriers to successful implementation:
- Competing or overlapping IT and information systems
- A reluctance to change existing ways of managing (pen and paper)