Updates to Metlog

This page (begin 3/April/1997) is intended to let people know about:
  1. General matters of Antarctic Met. reporting.
  2. Downtime - times when our system failed
  3. Updates - enhancements to the system
  4. Bugs - errors we've found in past data.
  5. Oracle - problems etc.

General

It is worth noting that our data comes from the GTS which is primarily aimed at providing data for numerical weather prediction (NWP) centers. Some GTS nodes (eg Washington) therefore have a policy (to save bandwidth) of not forwarding "old" messages (those more than 6 hours out of date). Not all Antarctic stations have their own connections to the GTS; some rely on others to forward messages for them. This can mean that, even if data is put into the GTS, it may not reach us, due to delays. The solution, in this case, is probably to contact the national operator.

Downtime

Three sorts of downtime occur.

The most common is our pc-to-unix link falling apart, which regrettably happens once a month or so. However, this particular problem means that GTS data is still received on the PC and will eventually be entered in our archives.

Less commonly, the leased-line to the UKMO goes down, or the PC stops working, or something else terrible happens. In this case no GTS data is received. Non-BAS data is lost; data for our own stations will be entered in the station archives when it is returned from South at the year end.

Even less commonly, the oracle system goes down.

Log of Downtime

03/04/97    - pc-to-unix link is down.
26-27/04/97 - unix systems will be down
28/04/97    - upgrading our o/s to OSF4 seems to have broken the Fortran code,
              though Perl marches on regardless. This will not be fixed soon, and
              maybe never - we may just switch to the Perl-only decode.
04/06/98    - Oracle is down (disk crash). Sun have apparently been called.
30/05/99    - File not processed due to data not avialable due to problems with
              UKMO link at their end.
end06/00    - For about the last week in June 2000, there were problems while we
              reconfigured the web system. No data was lost but the web interface
              didn't work for a while.
13-14/02/2003-Oracle is down whilst being painted or somesuch

Updates

  • 2/4/97 - Added cl, cm, ch to gts_land_synop and gts_ship_synop tables.
  • 10/4/97 - Missing data values are now entered into gts_synop as nulls. Previously, they were 99.9 (reals) or 999 (integers). Use caution when averaging! Note: gts_land_synop and gts_ship_synop always have missing data entered as nulls.
  • 11/4/97 - Reorganise the decoded-in-real-time tables gts_land_synop and gts_ship_synop. New table gts_perl_synop was copied from gts_ship_synop. Both ships (BBXX) and land stations (AAXX) are now decoded into this table. Known land stations have their lat and lons inserted by look-up; unknown ones will have nulls.
  • 14/4/97 - Moving tables over together into gts_perl_synop has broken the where-id auto-feature. This is fixed internally and will be fixed for external users as soon as my patched script is copied into our public-access cgi-bin.
  • 25/4/97 - Add gts_perl_temp for the radiosonde reports.
  • 30/07/97 - Add gts_perl_zbuoy for the buoy reports (zbuoy so that it comes after synop alphabetically). Also add buoys-met for the most recent reports. Currently only the initial and first groups are decoded - I await feedback before deciding whether to add more.
  • 05/06/98 - GTS synop tables now for each year; gts_perl_synop now covers only from 1/1/98, '97 split off into z_gts_97.
  • [this is now hopelessly out of date...]
  • 06/2000 - Add gts_perl_zclimat table

    Bugs

  • 04/04/97 - Discover that the perl-scripts decoding has been, under some circumstances, including values which should have been null (coded with soliduses). So, Dew Temperatures which were coded as 2//// were inserted as 0 (actually, -0). Some missing MSLP values were inserted as 1000. This is fixed as of now. Further, the temperature group decoding would have suffered from the same problem (except, the temperature group is rarely null). Also, 1022/ (which can plausibly be decoded as 2.2 or 22) was decoded as 2.2; now it will decode "null".
  • 04/04/97 - Upgrade error checking to make sure that message consists of 5-figure groups. All groups past the ??XX ident up to 333/444/555/ICE are deleted if they are not 5-figure.
  • 30/04/97 - Discover that wind speeds were being inserted into gts_perl_synop in either knots or m/s according to how they were reported, which is undesirable since you can't tell the difference! Each station will be consistent, however. As of 30/04, all are inserted in knots.
  • 07/05/97 - Fix bug in the TEMP decoding that had prevented it working during may.
  • 29/07/97 - When coding up BUOY decoding, realise that the dew temperature group can be 2Sddd (S=0 or 1, the sign) or 29ddd (in this case, ddd is relative humidity). The second case appears rare; it will now be thrown away (since we dont't know what to do with an RH).
  • xx/06/98 - A problem in the TEMP decoding lead to incorrect values for the geopotential height. The problem was caused by a bug in the section that "guessed" the tens-of-thousands value. Any errors in the decode should be very obvious.
  • 07/08/2000 - While redo-ing old buoys a problem arose with inputting data that may have entered some old data with year,month 2000,08. To be sure, I deleted *all* 2000,07 and 2000,08 gts_perl_zbuoy data and redo-ed it.
  • 20/11/2000 - Was getting error messages from some dumb buoy that was encoding day and month the wrong way round. Add error check for 0<month<13.
  • 08/12/2000 - Realise that data was added to the CLIMAT table with "update on date and mslp", not the rather more sensible "update on date and station-id". Drop the table, and remake it. Sorry folks.

    Oracle Problems

  • [none known]