«Medium-Range Weather Prediction Austin Woods Medium-Range Weather Prediction The European Approach The story of the European Centre for Medium-Range ...»
At the end of 2001, IBM informed the Centre that a user group named SPXXL, made up of sites that had installed very large IBM systems, met twice a year. The next meeting would be in February 2002. IBM suggested that the Centre contact one of the sites and ask them to sponsor the Centre to join the user group. This would allow ECMWF representatives to attend, and thus have access to confidential information that was disclosed by IBM at The computer system: CDC, Cray, Fujitsu, IBM 209 that meeting. Further, the Centre could exchange information with other sites with similar systems. The National Energy Research Scientific Computing Center (NERSC) at the Berkeley Laboratory in California, kindly agreed to sponsor the Centre. The meeting gave IBM a platform to inform the sites about their future plans and products. Further, it collectively added weight to the views, opinions and arguments of these major customers to help shape the future direction and development of IBM’s high performance computing strategy.
In 2004 Phase 3 replaced the Phase 1 system. Phase 2 was skipped in favour of an increase over the committed performance of the Phase 3 system. Phase 3 consisted of two clusters each of 70 IBM pSeries p690++ servers, each with 32 CPUs with a clock cycle time of 1.9 GHz (7.6 Gigaflops peak) and 32 Gigabytes of memory. These nodes were connected by a “federation” switch (pSeries High Performance Switch), an IBM-proprietary interconnect.
We have seen the computing facilities at ECMWF evolving from a single-CPU CRAY-1 to a large dual cluster of shared-memory processor systems. The sustained performance has grown 70,000-fold and the complexity of the system has increased by at least an order of magnitude. There has been some simplification, for example in the adoption of Unix in one form or another as the operating system on the main computers, rather than the mixture of different proprietary operating systems of the early years.
One of the analysts at ECMWF remembered reading an article in the late 1970s that predicted that one day wristwatches would have more processing power than the CRAY-1. Mobile phones and personal digital assistants of 20 years later were approaching that. In the coming decades, mainframe computers will continue to increase in power. If the Centre is to keep its position in the forefront of NWP, it is clear that continuing access to some of the most powerful systems available at any particular time will be required.
Comparison of the 1977 CRAY-1A with the 2005 IBM supercomputer Chapter 17 Communicating the forecasts: mail and 50 baud to RMDCN In Chapter 3 we saw the far-sighted plans made in 1971.
High-speed data links (2400 bits/sec) between the Centre and associated national centres were indispensable for the dissemination of the computed medium-range predictions. Some of these data links were also necessary for rapid input of digital data in the form of grid-point values or pre-processed data originating from European and other centres. The satisfactory incorporation of all these data requirements into existing and projected WMO telecommunication channels appeared unlikely; a separate data net for the envisaged computing centre was needed. All these high-speed data links should be capable of operation in full or half duplex mode and hence would provide an ideal basis for teleprocessing of data.
In June 1975 the “First Meeting on Data and Telecommunications Needs”, convened by the Centre, was held in London with the approval of the Interim Council following a recommendation of the Scientific SubCommittee. Eleven of the Signatory States were represented, with an
observer from WMO. Jean Labrousse was elected Chairman. In his welcoming address, Director Wiin-Nielsen noted that:
The Centre is not located in the geographical centre of the participating countries, but rather in the corner to the north and west. I am also told, although these decisions were made before I had even heard of the Centre, that the site evaluation teams were told that they should not give preference to a central location because the communication network and its cost would be part of the budget of the Centre and thus shared on a proportional basis between the Member States. It goes without saying that if such a policy is not used in the future, but instead that each country has to pay for its lines to the Centre, it will be very inexpensive for 211
212 Chapter 17the host country and very expensive for the far-away countries, say Finland and Greece, and Turkey when she joins our family. The difficulty is of course that the statements made above are not discussed in the convention or protocol, but are only part of a general understanding, or you might say a gentlemen’s agreement. However it goes without saying that we are all gentlemen, and it will be expected that we live up to such general understandings.
The meeting was an informal planning meeting primarily concerned with technical considerations. It was considered important that, to minimise technical problems, the Centre would be fully responsible for implementation, operation and maintenance of the network. Requirements for the operational products of the Centre were discussed; around 40 million characters per day were foreseen as the total amount of information to be disseminated. The WMO representative stressed that no spare capacity would be available on the Global Telecommunications System (GTS), the network used for global exchange of weather data between all countries of the world, in particular during the FGGE period 1978–79.
Lennart Bengtsson considered the envisaged structure of the operational forecast routine, and presented preliminary views on the volume of analyses and forecasts to be produced and on principles of dissemination. Since it could be expected that the internal model parameters would be frequently modified, as a first principle it would be assumed that the form of the disseminated products would be independent of the internally-used parameters of the model.
Resulting interpolation errors would be small. Only basic quantities would be disseminated; it was expected that the Member States would compute derivatives (e.g. vorticity and divergence, mean and extreme values, and quantities such as thickness, dew-point depression and potential temperature). It was proposed that the Centre would restrict sending of products that had not been properly tested and verified.
The meeting agreed that as a minimum, 10 million characters would be disseminated each day. Procedures and protocols to be used on the lines were considered; it was clear that WMO GTS procedures would not be suitable in view of the special requirements of the Centre such as access to databases and remote use of the system. Thus sharing GTS networks would not be feasible; a dedicated ECMWF network would be required. [We will see that improvements in technology and software did allow just such sharing of the lines 25 years later.] In July 1975 the report of the meeting was considered by the “Consultative Sub-Committee on Scientific and Technical Matters of the Interim Committee of the European Centre for Medium-Range Weather Communicating the forecasts: mail and 50 baud to RMDCN 213 Forecasts”. The Sub-Committee endorsed the intention to confine dissemination to the basic variables, “with the understanding that such a restriction should not be applied too strictly”. Furthermore the Centre “should not be expected to provide operationally a variety of services normally associated with short-range predictions”.
The telecommunications links to the Member States, envisaged in this early planning of the Centre, were “to be postponed until 1981 or later” according to the 1976 programme presented to Council in May 1976. Not surprisingly some Member States did not welcome this delay. Finland tabled a Note on the subject, inviting Council “to express its view on the necessity of a dedicated telecommunications system and to instruct the Centre to find the cheapest possible solution to the problem”. Council set up an “Advisory Committee on matters relating to communications between the Centre and the Member States”, with Daniel Söderman of Finland as Chairman. The terms of reference of the Committee included evaluating Member State requirements for forecast products of the Centre, the means of distribution, how the Member States could use the computer system, and the technical and financial aspects.
In November 1976 the Council decided that a medium-speed network should be used, though for technical reasons it would start with a mixed network including some low-speed lines. In the Annual Report for 1976, a system of 20 lines of 9,600 bps was specified as the least required. In November 1977 the Council adopted the recommendations of the Committee, after amending the text so that it would be clear that the cost would be shared according to
Wiin-Nielsen’s “gentlemen’s agreement”:
[Council] approved that the cost for the telecommunications network proposed by the Advisory Committee be incorporated in the Centre’s programme of activities.
The Committee’s work led to a speedier implementation of the telecommunications system with the issue of an Invitation to Tender in July 1977, specifying a turnkey system for hardware and software. Protocols were agreed and accepted by Council in December 1977 that would become standard for communications between the Centre and the Member States.
The Committee continued its work until November 1978 when the Council established the Technical Advisory Committee (TAC), which elected Jean Lepas of France as its first Chairman. The work of the “Advisory Committee on the Use of the Computer System” (ACUCS) was also transferred to the TAC. Daniel Söderman became Head of the Operations Department in 1980.
214 Chapter 17By end 1977, a decision had been reached; the contract was signed on 14 March 1978. The chosen contractor was Service in Informatics and Analysis (UK) Ltd (or SIA) who supplied the software, with A/S Regnecentralen of Denmark as subcontractor for the hardware, centred on an RC8000 computer. The software design, including development of the international protocols, was largely completed by end 1978, using hardware temporarily installed in SIA’s London office. The Centre acted as agent for Germany, Denmark and Sweden to develop a link package to be used with Regnecentralen equipment; this was operating in the three States by March
1980. The first Network Front End Processor was installed in June 1979.
The communications system passed final acceptance at the end August
1980. By December 1980, medium-speed (2400 or 4800 bits per second) lines had been established to Denmark, Germany, Sweden, and the UK;
lines to France and Ireland were in test. Most other States had 50 or 100 Baud lines operational.
The operational forecast suite was ready from the start to disseminate ECMWF forecast products through the network. Dissemination could be started at the end of each post-processing time-step as well as according to a pre-defined time schedule, collecting the products required by a Member State from a Dissemination Data Base.
The forecast model predicted changes in wind, temperature etc. on pressure surfaces normalised with respect to surface pressure (so-called sigma surfaces) and with a horizontal resolution of 1.875°. These model surfaces and resolution would change relatively frequently. Hence the fields coming from the CRAY were transformed to standard pressure levels and a resolution of 1.5°; these standard levels and resolution would be maintained, thus insulating the user from the model changes.
It was envisaged that several thousand fields would be disseminated daily, having been transformed and restructured from the files coming from the CRAY into a format usable by the forecast offices on the Member States — either an “ECBIT binary code” or an internationally-recognised WMO GRID code. Only the link to Greece was established in August 1979 when the first operational forecast was run; 27 products were disseminated in total from this forecast.
A facsimile transmission to the UK Met Office at Bracknell, with onward transmission to Offenbach and Paris, was arranged. Otherwise analysis and forecast charts for the Atlantic and European area were despatched by mail the following morning. The mail service continued for many months to some States. In contrast, by November 1982, the UK delegation to Council was expressing “surprise at the high number of products (8,000) distributed Communicating the forecasts: mail and 50 baud to RMDCN 215 daily by the Centre to the Member States”. It was explained that this included all times, levels and sub-areas despatched to all Member States.
Multi-streaming software was developed to allow simultaneous transmission of forecast products and output from remote batch jobs. A second RC8000 computer was purchased to provide backup. Backup for the acquisition of observational data was provided by a second link to the GTS via Offenbach, as a complement to that via the UK Met Office. At the end of 1981 only five Member States were depending on low-speed links: Spain, Italy, Greece, Yugoslavia and Turkey.
From 1 August 1981, the Centre disseminated a range of its most important products on the GTS, making them available free of charge to all countries of the world. This was in line with the admirable tradition of the world of meteorology, whereby data and products were exchanged freely. Fields of surface pressure and 500 hPa height for the Northern Hemisphere to five days, and for the Southern Hemisphere to four days (at the time, the accuracy of the Southern Hemisphere forecasts was lower), and analyses for the wind fields for the tropics at 850 and 200 hPa, were made available, and were quickly being used in forecast offices world-wide. Although the model resolution was 1.875°, GTS dissemination was at lower 5° resolution. By 1983, Australia, New Zealand and South Africa were using Southern Hemisphere forecasts, while China, Japan, USA, Hong Kong and India were using those for the Northern Hemisphere. Valuable reports on the quality of the forecasts and their usefulness in operational prediction were received regularly.
Dissemination on the GTS increased steadily in the following years, additional products and levels added, the resolution increased, and the forecast period extended, as the forecast quality improved.