This Readme file contains the following five sections: README Data Set Description: README General Information: README Tape Information README Data README Changes ################################################################################ README Data Set Description: This file is largely excerpted from the TOGA COARE Integrated Sounding System Data Report - Volume I prepared by Erik Miller, National Center for Atmospheric Research, Boulder, CO 80307. That document contains a full description of the profiler system, data collection and data analysis procedures. Users should at least read Volume I before making use of the profiler data. Other volumes in that set, Volumes X through XVIII, give further details pertinent to individual sites and show daily plots of the available data. The material below gives a explanation of the post processed profiler data products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Profiler data products Profilers were operated for TOGA COARE at six ISS sites. Four on land at Kavieng, Kapingamarangi, Manus and Nauru, and two at sea on board the R/V Kexue #1 and Shiyan #3. In addition there was a profiler aboard the R/V Moana Wave. Profiler data products have been prepared for all these sites throughout the IOP. For Manus additional data has been prepared for the 4 months preceding and following the IOP. The same additional periods were prepared for Nauru but most of the usable data came in June 93. In addition data sets obtained with the National Oceanic and Atmospheric Administration's 50 MHz profilers operating in a high height coverage mode at Biak and Christmas Island have been processed in the same manner from July 92 through June 93. ASCII files are produced for each half hour of profiler data. The file naming convention was chosen to be compatable with DOS file name conventions for the convenience of PC users. Each file has a name of the form ssmyyddd.hhv where ss is the site abbreviation = bi for Biak 50 MHz profiler ch for Christmas Island 50 MHz profiler e3 for the R/V Shiyan #3 ISS (Experiment 3) kp for the Kapingamarangi ISS kv for the Kavieng ISS ma for the Manus ISS mw for the R/V Moana Wave 915MHz profiler na for the Nauru ISS s1 or (sc) for the R/V Kexue #1 (Science 1) m is the operating mode = l for low height mode h for high height mode r for RASS mode yy is the year = 92 or 93 for the IOP ddd is the day of the year (001 = Jan 1) hh gives the half hour of start of the record = 00 for 0000 U.T. = 47 for 2330 U.T. v gives the version number = a for TOGA COARE first release. Each site is a in a seperate tar file that contains directories with all files for a given julian week. The following directory format is as follows: /yyddx where the symbols are as above except that ddx just means that the included data goes from dd0 to dd9. Half Hour Wind Profiler Data Each file is in ASCII (UNIX) format. The first ten lines describe pertinent experimental details. Then follow two lines describing the columns in the data table to follow. Finally comes the data table, one line for each height sampled. The descriptive text starts off with the file name, site name and data mode. Then the starting time for data collection is given as calendar date followed by Universal time. The day of year count starts at 1 on Jan 1. The ending time for data used in the average is given. There is no indication of how uniformly the time interval is filled. The site latitude, longitude and altitude may not agree exactly with those given in the quick look data files. The ones used here are considered more accurate (even though the shipboard altitudes are just informed guesses). It should be noted here that the latitude and longitude given in the shipboard site quick look data was entirely fictitious. The quality number Q1 serves two different purposes. Normally, like all other quality flags, it is zero. However, if all the data in this record came from quick look data files, then it is incremented by 1. On shipboard systems it is incremented by 2 if the latitude and longitude were interpolated over an interval such that there might be an error in the least significant digit or it is incremented by 8 in circumstances where a larger error is possible. The quality number Q2 takes on non-zero values only for shipboard data. It has 1 added whenever either of the ship's velocity components changed by more than 1m/s, or the heading by more than 30 degrees, between two consecutive profiles, 2 added whenever the ship's velocity changed by more than 2 m/s during the data collection period and 4 added whenever the ship's direction of motion changed by more than 90 degrees during the data collection period. The number of profile sets (a triad of three measurments) is given along with the number of those characterised as being clear air sets, and the number characterised as having hydrometeors present in the measurements, which are designated as rain sets. When quick look data has been used exclusively for this record then there is no information available about the number of profile sets. The total number is then set, arbitrarily, to 1 and the number of clear air and rain sets are both put to 0. The signal to noise threshold numbers are provided to give a reference for use with the signal to noise numbers in the table. The threshold is determined by several instrumental settings and it is unlikely that any valid measuments could be made at s/n ratios below the threshold. The data table description lines indicate the column content and measurement units. Users should note that heights are above sea level, whereas the quick look reports had height above the radar. The horizontal velocities, u and v, have been corrected for vertical motions in the rain triads, but not in the clear air triads. For shipboard systems they are also corrected for ship heading and, when the velocity is above 1 m/s, for ship velocity. The columns labeled 'dev' contain the root mean square difference from the mean for the data sets used and hence are deviations of the data and not deviations of the mean. The columns labeled 'widn' contain the average of the width parameter for the that beam. When n is 1 it is for the oblique beam that points most nearly E-W and when 2 it is for the other oblique beam, which points most nearly N-S. For the vertical beam n is set to w. The signal to noise ratios give the arithmetic average of the signal to noise powers, expressed in decibels. The column labeled 'n' gives the total number of triads used at that height while nr shows how many of those were rain triads. The quality parameter Q ranges from 0 to 9, where 0 is most likely to be reliable and 9 is least likely. The odd quality numbers indicate data that came from the quick look processing, but are otherwise equivalent to the next lower even quality number. Users may wish to use this number as an aid in selecting data for use; for many purposes data with quality numbers of 3 or below will be suitable. Additionally they may consider using the number of triads used in the average as an additional quality control; an average of only one datum may often be suspect. Users should note that for the vertical data there is a separate height column. This is required because the data samples are taken uniformly in range. Hence the inclined oblique beams have a different range to height conversion than does the vertical beam. As only clear air (non-rain) data is used to compute vertical velocity there is only one count of sets used. The convention used for labelling heights is that the height given is the height of the midpoint of the transmitted pulse. In a homogeneous medium the reflected energy would fall off inversely with range squared and hence the weighted average response would correspond to a height lower than the labeled height. The effect would be greater at the shorter ranges. As the medium is unlikely to be homogeneous the situation is even more complicated than described above, and thus even more uncertain. Hence the convention has been not to apply any inverse square weighting. In the high height coverage modes the first sample is often taken before the pulse has completely left the antenna. In this case the height label will be an underestimate. The user should be aware that the profiler radars require a non-zero recovery time after the pulse is transmitted before reliable wind measurements can commence. In order to make measurements at the lowest possible heights the instrument settings are often such that the lowest height or two produce unreliable results. Special attention has been paid to the data in the lowest heights and the bulk of the obviously erroneous measurements have been removed. However it is certain that some erroneous results have been included in the data sets and the user should exercise greater care when analysing data at the lowest heights. Throughout the table 9999 is used to flag missing data. When the data comes from post processing, missing data will be associated with a count of zero items in the average. However, when quick look data are used, some columns can be missing data while others contain valid data on the same line. Half hour RASS data The organization of the RASS half hour average data files is essentially the same as for the wind. The column headings differ to reflect the fact that virtual temperature and vertical wind velocity are being measured. As both measurements are made with the same beam there is only one height column. The suffixes 1 and 2 on the width and s/n columns refer to the spectral regions looked at, 1 for the vertical wind echo and 2 for the sound echo. For temperature there are two data count columns, one labeled 'n' for the total number of measurements and a second 'nr' to indicate those effected by rain. Note that for temperature estimation the vertical velocity is subtracted from the sound velocity in clear air cases, but not in rain cases. The clear air vertical velocity estimate is made only from the clear air cases. Users should note that refinements to the constants used in calculating the virtual temperature result in numbers that are about 0.3 degrees lower than those calculated for the quick look data set using the same sound and air velocities! Daily wind vector plots For each day a summary wind vector plot was produced separately for the high height mode and the low height mode. The vectors show the speed and direction of the wind at each height and time. The quality number for each vector is indicated by a filled circle at the base of the vector, increasing in size corresponding to the quality numbers 2, 4, 6 and 8, or a filled square, increasing in size for the quality numbers 1, 3, 5, 7 and 9. In a panel below the vector display is a simple indicator of the clear air vertical wind. A set of horizontal bars are drawn, one for each height, with length proportional to the vertical velocity w (positive to the right). In order to avoid overlap all velocities above 0.5 m/s are represented as though they were 0.5 m/s. In a panel below the vertical velocity display is an indicator of the occurrence of rain (or snow) as determined by the profiler. The length of the vertical bar is proportional to the fraction of data sets that are thought to indicate rain compared to the total number of data sets in the half hour. When the data has been taken from the quick look data set there is no rain information available; this plot always shows a full length bar under those circumstances. Each file is a self contained Postscript text named ssmyyddd.00p with ss, m, yy and ddd as described above. They too were placed in a directory called postscript. Daily RASS temperature anomaly plots For the RASS data the main panel shows not the virtual temperature but the difference (anomaly) between the measurement and a straight line standard temperature. This straight line standard starts at 30 degrees and has a slope of -0.4 degrees for each range gate. Hence the anomaly is given by A = T - 30 - 0.4*i where A is the anomaly , T the virtual temperature and i the height number (i=1 being the lowest height). The standard is not meant to represent any particular model atmosphere but was chosen to keep the plots looking reasonable. The temperature anomaly is plotted as a horizontal line (positive to the right). A circle or square at the base of the line indicates the quality indicator for that measurement as described under the daily wind vector plot description. When measurements were obtained at adjacent heights then the tips of the horizontal lines were joined. The panels below the temperature anomaly display indicate clear air vertical velocity and rain as in the daily wind vector plots. Data availability Files have been generated which enable the user to determine quickly whether data in a particular mode is available at a particular site for any given time. These ASCII files have names of the form sssyyddd.dat where sss is a site designator and yyddd gives the first day of the period covered by the file. After the first two descriptive lines the files contain one line for each day of data. Each line commences with yyddd which gives the year and day being reported. Then follow 48 columns, one for each half hour of the day. Within each column is a number ranging from 0 (no data) to 7 (all modes). The number is composed by adding 1 if there is low height coverage wind data, 2 if there is high height coverage wind data and 4 if there is RASS data. Data quality count Files have been generated which enable the user to assess quickly the quality of data in a particular mode at a particular site. These ASCII files have names of the form ssmyyddd.tot where ss is a site designator, m indicates the mode and yyddd gives the first day of the period covered by the file. After the first three descriptive lines the files contain one line for each height in profiler record. Heights are indicated just by a number, 1 being the lowest. Then follow 20 columns, 10 for the horizontal wind or virtual temperature quality number and ten for the clear air vertical wind. Within each column is the count of number of occurrences of that quality number. Each file contains totals for the complete data set unless there are mutiple files, in which case the first file contains counts for data prior to the second file and so on. Ship position, speed and heading data For the three profilers on board the ships Kexue #1, Shiyan #3 and Moana Wave corrections had to be made to the data for the ship motion and heading. In order to illustrate the corrections made, files have been prepared for each ship giving half hour average values of the ship's position, velocity and heading. The time given in each row is the start time of the half hour average. As corrections were made on a profile by profile basis, and because the profiles were not always made uniformly throughout the half hour (RASS for instance usually occurred only within the first 6 minutes), these files only contain illustrative numbers, not the actual corrections applied to to individual records. The files are named sss.pos where sss gives the site. The files were prepared using the raw heading information, so the corrections subsequently applied to Kexue #1 and Shiyan #3 heading data have not been applied in these files! ################################################################################ README General Information: Cleaned profiler data, both winds and RASS, have been prepared for the ISS stations on Manus, Kavieng, Kapingamarangi, Nauru, Kexue 1 (exp) and Shiyan 3 (sci) for the IOP. Moana Wave profiler data has been prepared for the same period. In addition 50MHz profiler wind data has been prepared for the period July 92 through June 93. All of this data is available for general use through anonymous ftp from tcdm.coare.ucar.edu. Each directory contains data for one station and the directory name is an obvious three letter abreviation of the site name. ############################################################################### README Tape Information The wind profiler data set has been archived on a 8mm Eabyte tape. The tape contains 9 seperate tar files, one for each wind profiler site. All of the data has been archived in a uncompressed ASCII format. TAPE POSITION SITE ------------- ---- 0 Biak 50 MHz profiler 1 Christmas Island 50 MHz profiler 2 R/V Shiyan #3 ISS (Experiment 3) 3 Kapingamarangi ISS 4 Kavieng ISS 5 Manus ISS 6 R/V Moana Wave 915MHz profiler 7 Nauru ISS 8 R/V Kexue #1 (Science 1) The command tar -xf /dev/(exabyte tape) will produce the following type of files to be created: 9231x/ 9232x/ ...... postscript/ s1h92306.tot s1l92306.tot s1l92306.tot sci.pos sci92306.dat 9231x/ Each directory of this format contains all files for 10 julian days. This is the directory for 1992 period starting on julian day 310. The directory may have up to 48 files, one for each half hour, for each day (10*48 = 480 files). Each directory may also have up to 3 times this amount if the profiler was operating in all 3 modes. s1h*: h = High height mode. s1l*: l = Low height mode. s1r*: r = Rass mode postscript/ A directory containing quick look postscript files of each day for which data was available, and for each type of data (high height mode, low height mode and rass). s1h92306.tot A file which contains the quality of data in a particular mode for a particular site. s1l92306.tot s1r92036.tot sci.pos A file which contains ship position data. sci92306.dat A file which lists data availability. ################################################################################ README Data Potential users of this data should read the TOGA COARE Integrated Sounding System Data Report - Volume I prepared by Erik Miller, National Center for Atmospheric Research, Boulder, CO 80307. That document contains a full description of the profiler system, data collection and data analysis procedures. Another file in this directory, 'files.README' gives some useful excerpts from that volume. Other volumes in that set, Volumes X through XVIII, give further details pertinent to individual sites and show daily plots of the available data. The material below gives excerpts of the information in Volumes X through XVIII that are pertinent to the particular sites listed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Volume X, Biak This 50 MHz profiler operated throughout the period July 92 through June 93. There were almost complete data gaps in February, May and June 1993. It operated in only one mode with high height coverage of 48 heights separated by 495 m in range. The pulse length was 1004 m. The Biak site has severe radio interference problems. They overwhelmed the interference detectors in the processing, particularly from October through March, and make much of the data above about 13 km suspect during that period. At 50 MHz it takes very severe rain to overwhelm the clear air echo and none was detected during the processing. Volume XI, Christmas Island This 50 MHz profiler operated with very few data gaps throughout the period July 92 through June 93. It operated in only one mode with high height coverage of 48 heights separated by 495 m in range. The pulse length was 1004 m. At 50 MHz it takes very severe rain to overwhelm the clear air echo and none was detected during the processing. Independent measurements showed that considerable rain reached the ground during the second quarter of 93, but that there was very little during the rest of the period. Volume XII, Kavieng This data set is almost complete throughout the IOP with no data gaps with duration of more than half a day. However for much of the 1992 data the recorded data had severe parity error problems. Although much of the data was recovered and became available for post processing there still remained some moderately large gaps which were filled with quick look data. Volume XIII, Kapingamarangi This data set is almost complete throughout the IOP except for days 338 and 339. However, the RASS data prior to late on day 338 is entirely fictitious, as the sound system was not working. The plots for day 316 indicate that some extremely high velocity winds have a quality flag of 0. The quality numbers in the corresponding data files were changed to 8 after the plots were made. Volume XIV, R/V Kexue #1 Heading When the ship is underway the magnetometer can be compared with the direction of the velocity vector. As the magnetometer was never calibrated one would expect a difference in these two angles equal to the magnetic declination, about 8 degrees when on station. In actuality the difference, vector angle minus magnetometer angle, averaged 22 degrees, but with large fluctuations. Just after the initial cruise to site the difference was only 12 degrees, but by the next comparison on day 357 it was 23 degrees. Thereafter the daily averages of the difference ranged from 18 to 27 degrees. Most of the comparisons are clustered in small periods of time, followed by large periods with no data. Consequently it was decided to apply only the simplest possible correction. So a 22 degree correction was applied to all data, including the quick look data incorporated into the final data product. The sense of correction was to rotate all wind vectors 22 degrees clockwise. Volume XV, Manus During the IOP this site provided complete coverage in the low and high modes and had only a few short dropouts in the RASS mode. In the four month periods preceeding and following the IOP data coverage was marginally worse, with one dropout in each period of about 18 hours and several smaller ones. The plots for days 205 and 212 indicate that some extremely high velocity winds have a quality flag of 0. The quality numbers in the corresponding data files were changed to 8 after the plots were made. Volume XVI, R/V Moana Wave This profiler was operated differently from the profilers associated with ISS sites. Initially the low height coverage mode observed at 34 heights with a separation of 60 m and a pulse length of 60 m. For the second and third legs of the experiment, all data after day 345, the number of heights was increased to 40. In the high mode initially there were 25 heights separated by 210 m with a pulse length of 420 m. After day 345 all data in this mode has 31 heights, with the initial height in the oblique beams lowered from 175 m to 91 m. The RASS mode started out with only 20 heights for several days and then went to 25 for the remainder of the experiment. To simplify data processing the records for the early days had 5 heights added, filled with null data, so that they look exactly the same as the bulk of the RASS records. Ship heading, velocity and position There were several sources of information used here. Ship position and velocity were available from the profiler records throughout most of the second and third legs of the experiment (data after day 345). When available this data was used. Ship position, speed and heading data with a 5 minute cadence were available from a supplementary source. This data, appropriately interpolated, was used whenever the profiler data was unavailable. The final source was a data set at 1 minute cadence containing speed and heading information that was available for the first and third legs of the experiment (before day 345 and after day 20). Interpolated values from this data set were used when no other was available. Whenever duplicate values were available for for a particular parameter then they were checked against each other. The data header flag Q2 was set to 1 if the ship velocity did not agree within 0.5 m/s, the direction within 30 degrees, the latitude or longitude within 0.5 degrees, or the heading within 30 degrees These criteria were in addition to the normal criteria for setting Q2 to 1 for shipboard data. As the ship heading and the direction of the velocity vector when the ship was underway seemed to agree within measurement errors, no correction was applied to the heading. Despite the heavy investment of time in trying to get accurate values for the ship position, speed and heading parameters the resulting wind data sets appeared different in character from those of the other two ships. This probably indicates that the corrections to the winds applied to the profiler data were sometimes incorrect. However such differences could also be due to the beam pointing problem noted below. Beam pointing It is often very noticeable in the Moana Wave data that when the ship changed direction then the character of the vertical velocity data changed at the same time. This is probably an indication that the vertical beam is in fact pointing somewhat off vertical. If this is true then, at the very least, correction of the oblique velocity measurements during rain is going to be prejudiced. Incorrect pointing of the vertical beam could be due to antenna phasing errors. Experience with other antennas of this type suggests that significant phasing errors are unlikely. Alternatively it could be due to incorrect operation of the stabilized platform which was supposed to keep the antenna horizontal despite motions of the ship. In this case the two oblique beams are probably also pointing in other than their intended positions, which would make the conversions to horizontal components incorrect. This problem could account for, or help account for, the sometimes erratic wind measurements from the Moana Wave when compared to the measurements at other sites. Volume XVII, Nauru This ISS profiler had severe data recording problems. The only post processing that could be done during the IOP was from some spectral moments recovered from the hard disk at the site. These moments were fed directly into the second stage of the post processing. Even so a thirty day data gap covers most of January. The only other data came in June when proper data recording was established. The plots for days 350 and 2 indicate that some extremely high velocity winds have a quality flag of 0. The quality numbers in the corresponding data files were changed to 8 after the plots were made. Volume XVIII, R/V Shiyan #3 Receiver Faulty operation of a chip on the coherent integrator board caused an intermittent problem of false echos in the low height coverage mode from Nov 20 through Jan 21. The fault causes low velocity echos to appear repetitively in height throughout the profile. A clear example of the problem appears around 1000 U.T. on Dec 23. In addition to a low velocity the false echos also had a large width ( 3 to 4 m/s as opposed to the normal 1 to 2 m/s), a fact that may help users discriminate against these false measurements. Heading When the ship is underway the magnetometer can be compared with the direction of the velocity vector. As the magnetometer was never calibrated one would expect a difference in these two angles equal to the magnetic declination, about 8 degrees when on station. In actuality the difference, vector angle minus magnetometer angle, averaged 12 degrees, but with large fluctuations. During the initial cruise to site the difference was actually negative (-10 to -19 degrees), but on day 310 at 0225 it increased 20 degrees within 30 seconds. On day 314 it increased another 10 degrees. Most changes were smaller, and slower, but still daily averages of the difference ranged from 9 to 17 degrees. Most of these comparisons were clustered around the cruises to and from port and there is little information available while on station. Consequently it was decided to apply only the simplest possible correction. So a 12 degree correction was applied to all data, including the quick look data incorporated into the final data product. The sense of correction was to rotate all wind vectors 12 degrees clockwise. ################################################################################ README Changes 23 June 94 Some of the '*.tot' files generated originally do not have correct values in the column for quality number 8 of winds or temperature. The files will be replaced as they are corrected. So far (9:00am) corrected files have been generated for Kapingamarangi. Also (9:05am) Kavieng. Also (9:25am) Science 1. Also (11:17am) Experiment 3. Also (11:50am) Manus. Only the *.tot files for 92306 had the error above but mah93060.tot had errors in all columns. Also (1:02pm) Nauru (only *92306.tot files needed change) Christmas Island needed no changes. Biak needed no changes. Moana Wave needed no changes. 29 June 94 All Biak and Christmas Island files have been withdrawn. Data was actually in local time rather than the labelled UT. There were some other problems at Biak around days 189-190. 7 July 94 Biak and Christmas Island files have been replaced. #################### End of Readme file #########################