«Medium-Range Weather Prediction Austin Woods Medium-Range Weather Prediction The European Approach The story of the European Centre for Medium-Range ...»
The ECMWF Data Services, set up to deal with requests for archive data from research scientists worldwide, extended its work to include supply of software developed at the Centre, including MAGICS, to meteorological institutes. Further it found itself becoming more involved in assisting the Member States in their provision of real-time data and forecasts to their clients, simplifying the ECMWF Catalogue of Real-time Products and setting up an on-line system for costing items from the Catalogue.
By the year 2000, the MARS archive held 185 TB of data. In answering requests to save and retrieve data, the system typically handled up to 18,000 operations each day with up to 200 GB of data being transferred. The hourly rate peaked at over 1,000 save/retrieve operations, transferring 20 GB of data. The ECFS archive held 50 TB of data in 4.5 million files, transferring 150 GB of data daily in about 10,000 files.
However, ADSM was not designed for use in the way that the Centre was using it. Developments planned by IBM that would have helped considerably were shelved. The system struggled whenever a file-system grew to more than a million files; the support staff spent too much time in problem-solving. The Centre issued an Invitation to Tender for the “Acquisition of a Replacement Data Handling System” at the beginning of 2001. Before that, use of the existing DHS was painstakingly investigated. Logs were analysed and statistics were produced. From these, the likely trends out to 2007 were deduced.
In late 2002, following a competitive tender, a new IBM Data Handling System was installed, the cornerstone of which was the HPSS, the High Performance Storage System.
Once again all the archive data had to be transferred to the new system.
The back-archiving and migration from ADSM to HPSS was accomplished smoothly and transparently, a task that taxed the skill of the analysts of the Computer and Meteorological Divisions. The users of MARS and ECFS were entirely unaffected by the work. MARS data were the first to be migrated; this was accomplished in 2003. However, there was a delay in the Archives and Graphics: towards MARS, MAGICS and Metview 193 migration of ECFS data until a new version of HPSS was installed, better suited to how ECFS stored and accessed data. The Centre had been the “beta-test” site for this new version throughout most of 2003. Other HPSS sites were pleased with the Centre’s role in helping to ensure that the product finally produced was stable and secure.
The ECFS migration took about ten months to accomplish throughout
2004. Both MARS and ECFS were designed so that the underlying data structures could be re-arranged while allowing the end user to use the same data request and without preventing access to the data, even temporarily.
Because MARS and ECFS are so flexible in this respect, the same data could exist in both the old ADSM-based system and in the new HPSS-based system concurrently. Once the analysts were confident that the two copies were identical, the MARS or ECFS server could be instructed to start serving the data from the new HPSS system.
The Centre’s service to the research community was improved by developing a data server to supply immediate, free and direct access to data sets on-line.
At the time of writing, MARS holds observations from five decades. On 17 October 2004, MARS passed the symbolic milestone of 1 PB of primary data — not counting backups — where 1 PB is 1024 TB or 250 bytes.
MARS had at that time around 8.6 billion (8,600,000,000) fields of common weather variables — wind, temperature, rain — and others not so common — altimeter corrected wave height, depth of ocean salinity maximum, ozone mass mixing ratio etc. ECFS had about one quarter that amount of data, held in over 12 million files.
This mountain of valuable information can be mined for many kinds of research into our atmosphere and oceans. It is easily accessed through a standard web browser. A client can follow how his or her request is being processed by the MARS servers, and can reformulate later requests to get the most out of the system.
MARS has proved itself to be a flexible, reliable, user-friendly system. It has been able to accommodate many new kinds of data: observations from many satellite instruments, two-dimensional wave spectra, reanalysis data, ensemble forecasts, monthly and seasonal forecasts, output from special projects such as DEMETER, PROVOST, HIRETYCS (High Resolution Ten Year Climate Simulations), and much more. One of its great strengths is the backward compatible interface: a retrieval request that was submitted in 1985, if submitted in the same form today, would still work. MARS software has become an integral part of many Member States’ systems, and is also used in the Bureau of Meteorology in Australia. It stands the core of the Centre’s manipulation of its data; development will continue in the future.
The computer system: CDC, Cray, Fujitsu, IBM
There are three basic components of the Centre’s work:
• Observational data from the atmosphere, land and oceans.
• Advanced scientific software.
• Powerful computer system.
In this Chapter we review the development of the computer system.
As we have seen, the first version of the ECMWF model was developed in the period 1975 to 1978 on a Control Data Corporation 6600 computer, one of the most powerful systems available at that time. A Service Agreement with Control Data Limited, in force from August 1975, provided access to the machine. The Agreement, which initially allowed 40 hours use per week, increasing to 70 hours per week from August 1976, was changed to a Lease Agreement in December 1976; this gave the Centre unlimited access to the machine. With the early version of the forecast model, 12 days of elapsed time was required to produce a ten-day forecast!
In addition, the Centre negotiated limited time on the IBM 360/195 – 370/158 systems at the Met Office in Bracknell.
In May 1975 the Centre issued preliminary notification to manufacturers of its requirements for a computer system to be installed in 1978. Exploratory talks with interested manufacturers followed. Tor Bloch, of CERN, and David Burridge visited the United States in November 1975. There they surveyed the state of development of the most powerful computers. Burridge noted that “the software team at Cray Research are under considerable pressure and are in a state of high tension!” Their Report was followed by six months’ intensive effort by staff of the Operations and Research Departments, assisted by experts from the Member States: Dr D. Henze (Germany), Mr A. MonodBroca (France), Mr R. Longbottom (UK) and Mr N. Spoonley (UK), as well as Tor Bloch. Their work led to the issue in July 1976 of an Invitation to Tender for the computer system, which was sent to all Member States.
194 The computer system: CDC, Cray, Fujitsu, IBM 195
The minimum specifications for the main computer were:
• speed 50 MIPS (million instructions per second),
• central memory one million words,
• mass storage 200 million words,
• card reader capable of reading 1,000 cards per minute, and
• a line printer capable of printing 1,000 lines per minute.
A front-end computer system was required to control the work of the
main machine, with:
• speed 3 MIPS,
• central memory two million bytes,
• mass storage 3,000 million bytes,
• three card readers, each capable of reading 1,000 cards per minute,
• a card punch,
• nine 6,250 bpi (bite per inch) 9-track magnetic tape units,
• four line printers, each capable of printing 1,000 lines per minute, • 12 visual display units (VDUs), a microfilm recorder, four plotters, and
• a 20-line telecommunications system operating at 9,600 bits per second bps.
A nominal data transfer rate of 10 to 20 million bits per second (bps) between the main and front-end computers would be required.
There were three contenders for the main computer.
• CDC Star100C from Control Data Corporation
• CRAY-1 from Cray Research
• TI-ASC from Texas Instruments — the “Advanced Scientific Computer” No more than half a dozen CDC Star100 machines, designed by Jim Thornton, were sold. The Star100C later evolved into the CYBER 205 and eventually into the ill-fated ETA line of computers.
The CRAY-1 was the brainchild of Seymour Cray, the designer of the CDC6600 and CDC 7600 during his time at Control Data. He set up his own company (perhaps surprisingly, with a small amount of backing from Control Data - then a competitor) to build this revolutionary vector computer.
Just over half a dozen TI-ASC computers were built. Most were used for oil reservoir simulation and exploration data processing. None were sold after the arrival of the CRAY-1, and Texas Instruments then withdrew from the supercomputer market.
Staff of the Operations Department made a second visit to the USA in September 1976. As well as the main computer manufacturers, they visited Los Alamos Scientific Laboratory, NCAR, the Naval Research Laboratory and NASA Langley Research Center. On 11 October the tenders had been received, and a Tender Evaluation Board had prepared its recommendation on the choice of computer.
196 Chapter 16The signing of the contract for the CRAY-1 on 22 June 1977 by Prof Aksel Wiin-Nielsen and Mr Seymour Cray.
In November 1976 the Council created its first Advisory Committee, in addition to the two mentioned in the convention: the Scientific Advisory Committee and the Finance Committee. This new Committee was the “Advisory Committee to assist in assessing the financial aspects of the acquisition of the Centre’s computer system”. The Committee, in co-operation with the other two Committees, worked swiftly and efficiently under its chairman Mr M. Deloz from Belgium, and with Mr J. C. Hirel from France as chief technical advisor. By March 1977, the Council was able to authorize the Director to send a Letter of Intent to Cray Research Incorporated, informing the company of its decision in principle to acquire a CRAY-1 computer. The company was called “Cray”; the computers were called “CRAY”.
The choice of a front-end computer was not as clear-cut. The recommendation to Council was for a CDC CYBER 175. There was some debate in Council on the possibility of the Centre acquiring a European machine, in accordance with the hope expressed in the Convention relating to “the development of European industry in the field of data-processing”. In particular the UK delegation supported a computer manufactured by ICL. The Advisory Committee, with the exception of the UK representative, concluded that either a CYBER 175 or a CYBER 174 should be selected as the front-end system. The UK representative stated that “the ICL proposal could be regarded as fully acceptable”. Council agreed that further tests of the ICL 2976 and 2980 computers be made before reaching a decision. At its meeting in May 1977, after some debate on the outcome of the tests, and The computer system: CDC, Cray, Fujitsu, IBM 197 taking into account the extra work and an anticipated delay of more than a year in linking the ICL and CRAY-1 computers, Council voted in favour of the CDC CYBER 175. At this meeting, Council also approved the contract with Cray Research.
David Dent from the Centre worked at Chippewa Falls from April to October 1977, learning the CRAY software, and assisting his colleagues who were visiting Cray to carry out numerical experiments. The software was rather primitive at the time with new versions of the CFT compiler being installed almost daily. A tri-partite agreement was signed between ECMWF, Cray Research Inc. and Control Data to develop the “station software” that would enable the CDC computer to act as a front-end for the CRAY-1. From November 1977, the Centre’s scientists had access to Cray’s Serial Number 1, the first production model of the CRAY-1 series to leave the factory in Minnesota. It was installed in the Rutherford Laboratory. The CYBER 175 was installed there in January 1978. The CDC 6600 service then ceased. These machines were used to test the programs required to produce an operational forecast, allowing progress to be made in the work required for implementing the operational suite.
Wiin-Nielsen signed the contract with Control Data Limited on 28 November 1977. The following morning the Control Data account manager telephoned Wiin-Nielsen and asked if he could agree to a second signing of the contract. He sheepishly admitted that after celebrating the historic deed he had managed to mislay the original document somewhere on the London Underground system!
The staff, apart from the computer operators working at Rutherford, was still in the temporary offices at Bracknell. Remote job entry terminals and VDUs were connected to the Rutherford facilities. Data were interchanged via magnetic tape, with a courier service between Bracknell and Rutherford.
The prototype CRAY-1 gave an impressive average overall availability of over 95%. The availability of the CYBER — a tried and tested machine — was more than 99%.
Wiin-Nielsen, with the advice of Jean Labrousse and Lennart Bengtsson, decided at an early stage that portability was of paramount importance for the Centre’s software. For this, good documentation of the software would be vital. FORTRAN, a symbolic programming language, was chosen, against the advice of some colleagues in the Met Office that the Centre should use assembly language instead, in order to squeeze the last ounce of processing power from the machine.