«Medium-Range Weather Prediction Austin Woods Medium-Range Weather Prediction The European Approach The story of the European Centre for Medium-Range ...»
Whether over-optimistic, or perhaps suffering from an attack of hubris, coming maybe from relief that progress to date had been so good, WiinNielsen reported to Council in May 1979 that: “reliable forecasts can be provided to the Member States up to about one week. The forecasts are, from time to time, remarkably good up to 10 days, but this is not the general result.” The Centre’s first real-time medium-range forecast was made in time for the official opening of the building at Shinfield Park on 15 June 1979. The staff then took a well-earned breather. Looking back to May 1976, WiinNielsen gave the Council a detailed plan for the Centre’s programme of activities, beginning with a “request for proposals for computer system”, through “completion of HQ building” and “acceptance of computer”, and with the date of 1 August 1979 as the date on which operational forecasting would begin, with “forecasts prepared 2–5 days per week”.
Operational forecasting did in fact begin on 1 August, with forecasts to ten days ahead five days per week. The first day of August 1979 was tense.
The day started smoothly with delivery of the data tapes on time. Decoding
84 Chapter 7
It was clear from the successful implementation of the operational system that the Centre had talented and motivated staff, and not only in research. The computers were at the leading edge of the technology, and as Burridge later remembered were “not the easiest to get working, or to keep working reliably”.
Forecasts were made seven days per week from 1 August 1980. Initially dissemination of the forecasts to Member States was restricted to the first seven days, in view of the uncertainty of the quality of the forecasts after day seven. However it transpired that some Member States were able to enter the Centre’s system via their telecommunications links — which they were fully entitled to do — and were downloading the forecasts for days eight to ten. This was clearly unfair to the others. The forecasts to day seven were then termed “operational”, the later forecasts “experimental”, and all products were made available to all Member States.
The Analysis System — from OI to 4D-Var
Previous Chapters have outlined the origins, establishment and beginning years of ECMWF. We are now starting to consider the development of the Centre’s activities in discrete areas. The first paragraphs of this Chapter are general; they apply to all of the Centre’s activities.
Many hundreds of man-years of the work of advanced, capable and talented scientists, and many thousands of hours of the most powerful computing resources, have been devoted to development and regular operation of one of the world’s most sophisticated computer models of the dynamics, thermodynamics and composition of the fluid envelope of our planet.
At the time of writing about 70 experienced scientists work directly on the ECMWF forecasting systems. When Lennart Bengtsson became Director in January 1982, Dr Lingelbach of Germany, having noted that “it would mean bringing coals to Newcastle explaining your abilities to the
audience,” went on:
And you are not alone. You have the helping force here of men and women, I think it is no exaggeration to call it a potential unique in the world. And you also have 17 nations behind you. The Member States will ask you from time to time to be as economical as possible. However, you can be sure that all these European nations wish to see the best results possible from the institute they have founded, having in mind the tremendous economic value of medium-range weather forecasts. All the members know very well that this has its price.
We won’t try to be comprehensive. A good way to be boring is to be sure to leave nothing out. Detailed documentation on the analysis system and model is available elsewhere: on the web, and in ECMWF publications and the open literature. We’ll try to give the reader an impression of the nature and extent of the research activity over the years. You will note the extent of collaboration with the scientists at Météo France. This exemplifies the 85
86 Chapter 8benefits of the close co-operation with scientists throughout Europe. Many scientists from the ECMWF Member States and other States, including researchers from the USA, Australia and China, shared in the work. They brought their expertise to the Centre, and went home with the benefit of their experiences — and enough personal contacts to last a lifetime!
Right at the start, Bengtsson made an important decision: the Centre’s research analysis and forecast models would be developed from the current version of the operational models. Each time the operational model was changed, this new model became the basis of the research model. Bits of it, for example surface effects, clouds and heating, would be examined intensively, off-line as it were, by a group of scientists. Successful research would lead to a change in the research model. Running this in parallel with the operational forecast for days or weeks tested research as it was coming to fruition. On an agreed date, the research model became the operational model, and the “old” operational model was switched off. This wasn’t only practical — it was a smart move. It concentrated the minds of the researchers, as their work had a clear objective and would be considered fruitful if there was an immediate impact on ECMWF products.
Fundamental groundbreaking research was going to be carried out, but this wasn’t a place that would appeal to ivory tower researchers.
The observation network that evolved in the 1970s was very different to that of a decade or two earlier. With the major initiatives of the Global Atmospheric Research Program (GARP) and First GARP Global Experiment (FGGE), it was clear that the pace of change would accelerate.
Much more data would come from satellites. In addition data would be sent from buoys scattered over the oceans of the world, and commercial aircraft traversing the major air routes of the world would increasingly send wind and temperature data. More importantly, all these data would be very different to those collected at regular “synoptic” hours from thermometers and other instruments on the ground or carried aloft by balloons. In the future, data from various observing systems, with an irregular distribution in space and time, and with varying and incompletely known error properties, would need to be assimilated. In the words of Aaron Fleisher to the Sixth Weather
Radar Conference of 1957:
More data, more data, Right now and not later, Our storms are distressing, Our problems are pressing, We can brook no delay For theorists to play, The Analysis System — from OI to 4D-Var 87 Let us repair to the principle sublime, Measure everything, everywhere, all of the time.
Bengtsson was fully aware that the highest-quality depiction of today’s atmosphere, with regular distribution of “field variables” such as wind, temperature and humidity, and with good estimates of their errors, would have somehow to be produced to provide the starting point for the Centre’s medium-range forecasts. And the work would have to be completed by 1979; a reliable and fully functioning analysis system had to be in operation by then.
The analysis system was of highest priority. In 1975, Bengtsson went to Paris to attend a Study Conference on Four-Dimensional Data Assimilation.
There he met Andrew Lorenc, who was working on an analysis system at the UK Met Office. Outside the conference one evening, Bengtsson had a beer with Lorenc, and after a chat invited him to apply for a post at the Centre. Lorenc started at the Centre, at that time still located in Bracknell, in April 1976.
When Lorenc joined the staff, Gorm Larsen from Denmark, already at the Centre, had written a two-dimensional “Optimum Interpolation” or OI analysis scheme. A six-hour forecast “background” carried information forward from the observations received earlier. New information was contained in the many thousands of observations arriving through high-speed telecommunications lines in the last six hours. The OI system was designed to combine these; the error characteristics of both sources of information were taken into account. The analysis system would also provide the basis for the Centre’s work with FGGE data, discussed further in Chapter 14. It was “multivariate”, coupling the height of the pressure surfaces with the wind. The initiative for using the OI system was Bengtsson’s. It turned out, in Tony Hollingsworth’s words, to be “a big gamble of Lennart’s that was hugely successful”, but a well-founded gamble coming from Bengtsson’s GARP experience. He was aware that other major analysis centres had achieved only limited success in analysing the data from tropical regions made available from the GARP Atlantic Tropical Experiment (GATE) in 1974.
We have noted that it is of the highest importance to reduce the errors in the initial analysis. Errors at the start — and no matter how hard we try, these can never be completely eliminated — will grow as the forecast proceeds. A small error in the analysis will give rise to a bigger error in the one-day forecast that, after a week, can have become large enough even to dominate the forecast.
Operational implementation of the OI approach required resolution of a number of practical issues. It was not easy to invert a matrix corresponding to a global data set. A series of local calculations requiring differing
88 Chapter 8compromises on data selection, continuity between adjacent analysis volumes, multivariate relationships, and so on had been required. Lorenc, whose previous work had involved “Observing System Simulation Experiments” or OSSEs for FGGE, took on the job of thinking how the Centre could build a three-dimensional OI system, incorporating the satellite measurements of thick slices of the atmosphere that were expected to be a key component of the future global observing system. Shortly afterwards, Ian Rutherford from Canada was recruited as visiting scientist and acting Head of Data Assimilation Section. Rutherford was influential in the overall approach and in the design of the system to be used at the Centre. He was perhaps the first to apply statistical interpolation in a data assimilation cycle with a forecast background — he had published a paper on this in 1972 in the Journal of the Atmospheric Sciences.
Data were analysed on pressure levels, 850 hPa, 500 hPa etc. However, the lowest model level followed the terrain, and the levels above were related to the lowest level; this greatly simplified the model equations. Thus, the model levels were so-called “sigma” levels — the pressure normalised by the surface value. Transformation from analysis levels to model levels was required before and after each analysis. Rutherford advocated an “incremental” approach; only the analysis increments would be interpolated to model levels, that is, the differences between the first guess and the analysis, and not the analysed fields. The boundary-layer structure provided by the first guess would be retained. A scientist from France Olivier Talagrand did the work leading to the implementation, and the change was made to the operational system in December 1980, some time after Rutherford’s departure. We will see later that Talagrand was influential in making a major improvement to the Centre’s data assimilation scheme.
Rutherford was also influential in the terminology used by the team at the Centre. He didn’t like the term “first guess” which was generally applied to the six-hour forecast that was used as the starting point for the analysis. In Rutherford’s opinion, that implied the analysis was a “second guess.” He didn’t like “Optimum Interpolation” either; he believed that in practice it was not optimum. Hence, he would have liked the team at the Centre to use the terms “background” for “first guess”, and “Statistical Interpolation” instead of “OI”, but in fact the term “OI” stuck! Lorenc remembered Rutherford as “a great mentor and friend”. In that early stage, his experience of operational schemes was invaluable to Lorenc, and to the work at the Centre.
Lorenc wrote most of the code of the three-dimensional OI system developed at the Centre. Rutherford and Larsen used early versions of the new OI The Analysis System — from OI to 4D-Var 89 system to make analysis error variance calculations for the FGGE observing systems. Results were fed into the design of the FGGE system. Tests of the OI system in 1977 were made using Data Systems Test (DST) data collected by the National Space Administration, USA, for two two-month periods, August–September 1975 and February–March 1976. The DST data were similar in quality and coverage to the data the Centre anticipated receiving in 1979–80, at the beginning of its operations. They included satellite temperature soundings of the atmosphere, winds estimated from satellite cloud observations, and aircraft weather reports. The early tests were already able to show the large impact of satellite data on Southern Hemisphere analyses, and some beneficial effect on analyses over datasparse oceanic areas of the Northern Hemisphere.
In the OI system, to analyse for example the wind at a single grid point, all observations containing relevant information — and this may be measurements of other “variables” such as pressure or temperature as well as wind — within a three-dimensional “radius of influence” were selected.