NESOB 1997 ARM-CART and National Weather Service Interpolated 5 hPa Vertical Resolution Sounding Composite Dataset 1.0 Dataset Overview This is one of the upper air sounding datasets developed for the Global Energy and Water Cycle Experiment (GEWEX) Continental-scale International Project (GCIP) 1997 Near Surface Observation (NESOB-97) Data Set conducted from 01 April 1997 to 31 March 1998. Included in this dataset are five Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) Clouds And Radiation Testbed (CART) sounding stations; Lamont (Central Facility), Morris (B5), Purcell (B6), and Vici (B4), OK and Hillsboro (B1), KS and 12 National Weather Service (NWS) rawinsonde stations in the NESOB 1997 domain. The NWS soundings were typically released at 00 and 12 UTC, however, several stations had additional releases on an as requested basis. At each of the ARM boundary facilities (B1, B4, B5, and B6) soundings were typically released once per weekday at 1800 UTC and at the central facility five times per weekday at 06, 12, 15, 18, and 21 UTC. However, during Intensive Observing Periods (IOPs) the sounding frequency increased to up to 8 per day (3-hourly; including weekends) at each of the sites. The soundings are terminated at the point nearest to, but not over, 3000 m. This dataset is a composite of data from all ARM-CART and National Weather Service sounding platforms interpolated to a constant vertical resolution of 5 hPa. 2.0 Detailed Dataset Description 2.1 5 hPa Interpolation Algorithm The 'native' resolution data for every sounding were interpolated to 5 hPa vertical resolution files. The surface data point was kept as the initial level in each sounding. The first interpolated data point was at the next lowest pressure evenly divisible by 5 and then every 5 hPa pressure level beyond that point to either 50 hPa or the lowest pressure level reached by the radiosonde, whichever came first. The first 15 lines of each file (the header information) were kept without change. For the interpolation, the software searched for two data points around the desired pressure level. The search was conducted by looking for two valid (i.e. non-missing) data points around the desired pressure level, while also paying attention to the time difference between the two data points as well as their quality control flags. There was a search for the two best possible data points to use in the interpolation. If the desired pressure level was within the original dataset, that data point was used without interpolation. There was first a search for values flagged as good within some time range (50 sec for temperature, humidity, and wind and 100 sec for pressure; hereafter termed the ARANGE) and the interpolated data point was flagged as good. Failing that, it searched for values flagged as estimated within the same time range and the interpolated data point was flagged as estimated. Then the search went for good values within a wider time range (100 sec for temperature, humidity, and wind and 200 sec for pressure; hereafter termed the BRANGE) the flag for the interpolated data point here was then degraded (even though two 'good' data points were used there was a significant time difference between them) to questionable. Then, in turn, estimated values within the BRANGE were used (flag set to questionable), questionable values within the BRANGE (flag set to bad), good values greater than the BRANGE apart (flag set to bad), estimated values greater than BRANGE apart (flag set to bad), questionable values greater than BRANGE apart (flag set to bad), finally any bad values (flag set to bad). This search was conducted separately for each interpolated variable (pressure, temperature, relative humidity, and the u and v wind components. Thus for each interpolated data point, the quality control flag was set to the worst case among the data points used in the interpolation, except, for each time range apart, the quality control flag was degraded one level (i.e. good to questionable, etc). The quality control flags should be carefully heeded in these files. While some of the data may look good, it may have been interpolated over large pressure intervals, and thus be suspect. For each interpolated data point the dew point was calculated from the temperature and relative humidity (Bolton 1980) and the total wind speed and direction were calculated from the interpolated u and v component values. Also, the altitude and time were interpolated using the same data points used for the pressure interpolation. The ascension rate was recalculated based on the time and altitude values from the two data points used to interpolate the 5 hPa data point. Thus the ascension rate values do not reflect the values based on the interpolated data. The latitude and longitude values were interpolated using the same data points used in the wind component interpolation. 2.2 High Resolution Data Description 2.2.1 National Weather Service High-Resolution Sounding Algorithms The detailed description of NWS sounding collection and instrumentation is located in NWS (1991). The native vertical resolution of the NWS soundings is six seconds. The NWS soundings during NESOB 1997 utilized either the VIZ B2. or the Vaisala RS-80 radiosonde. Vaisala radiosondes were utilzed at Amarillo (AMA), TX; Denver (DNR), CO; Santa Teresa (EPZ), NM; and Midland (MAF), TX. VIZ radiosondes were used by Dodge City (DDC), KS; Fort Worth, TX; Little Rock, AK; Norman, OK; Shreveport, LA; Springfield, MO; and Topeka, KS. The use of the raw 6-sec resolution elevation and azimuth angle data to derive the winds sometimes led to large oscillations in wind speed, due to the presence of oscillations in the elevation angle data, particularly at low elevation angles. The general approach to correct this problem was to remove the outlier radiosonde position data before computing the wind components (Williams et al. 1993). For both the azimuth and elevation angles from 360 sec to the end of the sounding, a ninth order polynomial was fit to the curve. The residuals were calculated and compared to the observed values. The outliers of the residuals were then removed. Then to help correct the more extensive problems at low elevation angles within 10 degrees of the limiting angles (LA) some additional smoothing was applied. If the elevation angle was between (LA + 7.5) and (LA + 10), the new elevation angle was computed with a 2 min linear fit. If the elevation angle was between (LA + 5) and (LA + 7.5), the new elevation angle was computed with a 3 min linear fit. If the elevation angle was less than (LA + 5), the new elevation angle was computed with a 4 min linear fit. If the number of observations with low elevation angles was greater than 20% of the total number of observations for the sounding no frequency smoothing occurred. Then, for the elevation angle only, a finite Fourier analysis was performed on the residuals. Periods from 90-190 sec were removed and those below 30 sec were flattened. Finally, a 2 min second order polynomial was then fit to the position to derive the u and v wind components, except for the beginning and end minute (or 1.5 minutes if over 50 mb) which used a 3 min fit. If there were less than 15% of the total number of points, not counting the beginning or end of the flight, on one side of the point for which the wind value was being computed, a linear fit was used. For further information on this methodology and its changes since Williams et al. (1993) please see Williams, et al. (1998). 2.2.2 ARM-CART High Resolution Sounding Algorithms All ARM-CART upper air soundings were converted from netcdf to University Corporation for Atmospheric Science/Joint Office for Science Support (UCAR/JOSS) Cross Chain LORAN Atmospheric Sounding System (CLASS) Format (JCF). JCF is a version of the National Center for Atmospheric Research (NCAR) CLASS format and is an ASCII format consisting of 15 header records for each sounding followed by the data records with associated Quality Control (QC) information. 2.2.3 UCAR/JOSS QC Procedures for High Resolution Soundings The following details QC procedures for the original high resolution soundings. However, quality control information derived by the following determines what points were used during the interpolation process (see Section 2.1). Some further information on the QC processing conducted by JOSS can be found in Loehrer et al. (1996) and Loehrer et al. (1998). ARM-CART soundings were Quality Controlled (Lesht 1995) and provided to UCAR/JOSS by the ARM-CART Project. This dataset underwent a JOSS QC process which consisted of automated internal consistency checks. This included gross limit checks on all parameters (see Section 2.2.3.1) and rate-of-change checks on temperature, pressure and ascension rate (see Section 2.2.3.2). JOSS did not perform visual quality control procedures on this data. NWS soundings underwent automatic QC process. The dataset underwent automated internal consistency checks. 2.2.3.1 Gross Limit Checks These checks were conducted on each sounding and data were automatically flagged as appropriate. Only the data point under examination was flagged. JOSS conducted the following gross limit checks on the 1997 sounding dataset. In the table P = pressure, T = temperature, RH = relative humidity, U = U wind component, V = V wind component, B = bad, and Q = questionable. __________________________________________________________________ Parameter(s) Flag Parameter Gross Limit Check Flagged Applied __________________________________________________________________ Pressure < 0 mb or > 1030 mb P B Altitude < 0 m or > 35000 m P, T, RH Q Temperature < -80C or > 45C T Q Dew Point < -99.9C or > 30C RH Q > Temperature T, RH Q Relative Humidity < 0% or > 100% RH B Wind Speed < 0 m/s or > 100 m/s U, V Q > 150 m/s U, V B U Wind Component < 0 m/s or > 100 m/s U Q > 150 m/s U B V Wind Component < 0 m/s or > 100 m/s V Q > 150 m/s V B Wind Direction < 0 deg or > 360 deg U, V B Ascent Rate < -10 m/s or > 10 m/s P, T, RH Q _________________________________________________________________ 2.2.3.2 Vertical Consistency Checks These checks were conducted on each sounding and data were automatically flagged as appropriate. These checks were started at the lowest level of the sounding and compared neighboring 6-sec data points for NWS soundings or 6-second averaged points for ARM-CART soundings (except for NWS soundings at pressures less than 100 mb and for where 30-sec average values were used). In the case of checks ensuring that the values increased/decreased as expected, only the data point under examination was flagged. However, for the other checks, all of the data points used in the examination were flagged. All items within the table are as previously defined. _____________________________________________________________________ Vertical Consistency Parameter(s) Flag Parameter Check Flagged Applied _____________________________________________________________________ Time decreasing/equal None None Altitude decreasing/equal P, T, RH Q Pressure increasing/equal P, T, RH Q > 1 mb/s or < -1 mb/s P, T, RH Q > 2 mb/s or < -2 mb/s P, T, RH B Temperature < -15 C/km P, T, RH Q < -30 C/km P, T, RH B > 5 C/km (not applied at p < 150mb) P, T, RH Q < 30 C/km (not applied at p < 150mb) P, T, RH B Ascent Rate change of > 3 m/s or < -3 m/s P Q change of > 5 m/s or < -5 m/s P B _____________________________________________________________________ 3.0 JOSS CLASS Format (ASCII text) Description 3.1 Header records The header records (15 total records) contain data type, project ID, site ID, site location, actual release time, nominal release time, and possibly other specialized information. The first five header lines contain information identifying the sounding, and have a rigidly defined form. The following 6 header lines are used for auxiliary information and comments about the sounding, and they vary significantly from data set to data set. The next line (line 12) contains the Nominal date and time of the release. The last 3 header records contain header information for the data columns. Line 13 holds the field names, line 14 the field units, and line 15 contains dashes ('-' characters) delineating the extent of the field. The six standard header lines are as follows: Line Label (fixed to 35 char in length) Contents 1 Data Type: Description of type and resolution of data. 2 Project ID: ID of weather project. 3 Release Site Type/Site ID: Description of release site. 4 Release Location (lon,lat,alt): Position of release site, in format described below. 5 UTC Release Time: Time of release, in format: yyyy, mm, dd, hh:mm:ss 12 UTC Nominal Release Time: Nominal release time. The release location is given as: lon (deg min), lat (deg min), lon (dec. deg), lat (dec. deg), alt (m) Longitude in deg min is in the format: ddd mm.mm'W where ddd is the number of degrees from True North (with leading zeros if necessary), mm.mm is the decimal number of minutes, and W represents W or E for west or east longitude, respectively. Latitude has the same format as longitude, except there are only two digits for degrees and N or S for north/south latitude. The decimal equivalent of longitude and latitude and station elevation follow. The seven non-standard header lines may contain any label and contents. The label is padded to 35 characters to match the standard header lines. 3.2 Data records The data records each contain time from release, pressure, temperature, dew point, relative humidity, U and V wind components, wind speed and direction, ascent rate, balloon position data, altitude, and quality control flags (see QC code description). Each data line contains 21 fields, separated by spaces, with a total width of 130 characters. The data are right-justified within the fields. All fields have one decimal place of precision, with the exception of latitude and longitude, which have three decimal places of precision. The contents and sizes of the 21 fields that appear in each data record are as follows: Field Format Missing No. Width Parameter Units Value ---------------------------------------------------------------------- 1 6 F6.1 Time Seconds 9999.0 2 6 F6.1 Pressure Millibars 9999.0 3 5 F5.1 Dry-bulb Temperature Degrees C 999.0 4 5 F5.1 Dew Point Temperature Degrees C 999.0 5 5 F5.1 Relative Humidity Percent 999.0 6 6 F6.1 U Wind Component Meters/Second 9999.0 7 6 F6.1 V Wind Component Meters/Second 9999.0 8 5 F5.1 Wind Speed Meters/Second 999.0 9 5 F5.1 Wind Direction Degrees 999.0 10 5 F5.1 Ascension Rate Meters/Second 999.0 11 8 F8.3 Longitude Degrees 9999.0 12 7 F7.3 Latitude Degrees 999.0 13 5 F5.1 Variable (see below) 999.0 14 5 F5.1 Variable (see below) 999.0 15 7 F7.1 Altitude Meters 99999.0 16 4 F4.1 QC flag for Pressure Code (see below) 99.0 17 4 F4.1 QC flag for Temperature Code (see below) 99.0 18 4 F4.1 QC flag for Humidity Code (see below) 99.0 19 4 F4.1 QC flag for U Component Code (see below) 99.0 20 4 F4.1 QC flag for V Component Code (see below) 99.0 21 4 F4.1 QC flag for Ascension Rate Code (see below) 99.0 ---------------------------------------------------------------------- Fields 13 and 14 are `variable' because depending on the sounding system the variables used in these positions can vary. For this data set, field 13 contains the Elevation Angle and field 14 contains the Azimuth Angle both in degrees. Fields 16 through 21 contain the Quality Control information (flags) generated locally at JOSS. These flags are based on the automated or visual checks made. See Section 2.2.3 for further information. The JOSS QC flags are as follows: Code Description ---------------------------------------------------------------------- 99.0 Unchecked (QC information is `missing.') (`UNCHECKED') 1.0 Checked, datum seems physically reasonable. (`GOOD') 2.0 Checked, datum seems questionable on physical basis. (`MAYBE') 3.0 Checked, datum seems to be in error. (`BAD') 4.0 Checked, datum is interpolated. (`ESTIMATED') 9.0 Checked, datum was missing in original file. (`MISSING') ---------------------------------------------------------------------- 3.3 Sample data The following is a sample portion of a JOSS CLASS format file including header records. The data portion is much longer than 80 characters and, therefore, wraps around to a second line. Data Type: Sounding Project ID: NESOB-97 5mb Interpolated Sounding Release Site Type/Site ID: B6 Purcell, OK Release Location (lon,lat,alt): 097 25.20'W, 34 58.20'N, -97.42, 34.97, 344.0 UTC Release Time (y,m,d,h,m,s): 1997, 10, 03, 11:30:00 Input File: sgpsondewrpnB6.a1.971003.113000.cdf / / / / / Nominal Release Time (y,m,d,h,m,s):1997, 10, 03, 12:00:00 Time Press Temp Dewpt RH Uwind Vwind Wspd Dir dZ Lon Lat Elev Azim Alt Qp Qt Qh Qu Qv Qdz sec mb C C % m/s m/s m/s deg m/s deg deg deg deg m code code code code code code ------ ------ ----- ----- ----- ------ ------ ----- ----- ----- -------- ------- ----- ----- ------- ---- ---- ---- ---- ---- ---- 0.0 969.9 17.5 16.5 94.0 -0.1 0.4 0.4 160.0 999.0 -97.420 34.970 999.0 999.0 344.0 3.0 3.0 3.0 99.0 99.0 9.0 7.9 965.0 19.8 17.2 85.0 9999.0 9999.0 999.0 999.0 5.6 9999.000 999.000 999.0 999.0 387.9 3.0 3.0 3.0 9.0 9.0 99.0 16.6 960.0 20.4 17.0 80.7 0.7 6.4 6.4 186.0 5.2 -97.420 34.971 999.0 999.0 433.0 3.0 3.0 3.0 99.0 99.0 99.0 4.0 Quality Control Procedures for 5 hPa Interpolated Dataset No additional quality control was applied to the interpolated dataset. The quality control procedures were conducted on the high resolution data sets and the results were applied during the interpolation (see Section 2.1.3). 5.0 Data Quality Issues 5.1 ARM-CART In terms of items checked by the automated quality control processing, the ARM CART soundings performed well, with about average numbers of data points flagged. However, since these data have not undergone visual quality control processing, there are many problems that have not been flagged. This is especially true for the wind and relative humidity values. So these values should be used with some caution. 5.2 National Weather Service 5.2.1 Near Surface Winds A common problem in near surface wind speed values calculated from the 6-second position data is that the first radiosonde wind speed is much higher than the independently measured surface value. The calculated radiosonde winds then decrease rapidly so that within about 60 s (20-30 mb) after release the wind speeds are more realistic. The cause of this appears to be the acceptance of radiosonde position data prior to a"good lock" being achieved on the radiosonde by the tracking system. Thus there appear to be rapid positional shifts of the radiosonde while the tracking system "searches" for the radiosonde. 5.2.2 Wind Oscillations Despite the extensive efforts to remove oscillations in wind speeds caused by oscillations in elevation angles (see Section 2.2.1) there are occasional cases with remaining oscillations. Most of the remaining oscillations have periods just slightly longer than the 190 s maximum point of our notch filter. 6.0 Composite Sounding Platforms 6.1 National Weather Service Upper Air Release Locations The following is a list of the NWS upper air sites contained in the NESOB 5 hPa interpolated sounding composite. The list contains each station's 3 letter identifier, station name, 2 letter state code, latitude and longitude in degrees, elevation in meters and sonde instrumentation used. All sites used radiotheodolite wind finding. ABQ Albuquerque NM 35.00 -106.60 1615 VIZ B2 AMA Amarillo TX 35.20 -101.70 1094 Vaisala RS 80 DNR Denver CO 39.80 -104.90 1611 Vaisala RS 80 DDC Dodge City KS 37.80 -100.00 790 VIZ B2 EPZ Santa Teresa NM 31.90 -106.70 1257 Vaisala RS 80 FWD FT WORTH TX 32.80 -97.30 198 VIZ B2 LIT North Little Rock AR 34.80 -92.30 172 VIZ B2 MAF Midland TX 32.00 -102.20 873 Vaisala RS 80 OUN Norman OK 35.20 -97.40 357 VIZ B2 SHV Shreveport LA 32.50 -93.80 83 VIZ B2 SGF Springfield MO 37.20 -93.40 390 VIZ B2 TOP Topeka KS 39.10 -95.60 270 VIZ B2 6.2 ARM-CART Upper Air Release Locations The following is a list of the ARM-CART upper air sites contained in the NESOB 5 hPa interpolated sounding composite. The list contains each station's 3 letter identifier, station name, 2 letter state code, latitude and longitude in degrees, elevation in meters and sonde instrumentation used. All sites used Loran-C wind finding. SB1 Hillsboro KS 38.30 -97.30 447 Vaisala RS 80-15LH SB4 Vici OK 36.10 -99.20 622 Vaisala RS 80-15LH SB5 Morris OK 35.70 -95.80 217 Vaisala RS 80-15LH SB6 Purcell OK 35.00 -97.40 344 Vaisala RS 80-15LH SC1 Lamont OK 36.60 -97.50 315 Vaisala RS 80-15LH 7.0 References Bolton, D., 1980: The Computation of Equivalent Potential Temperature. Mon. Wea. Rev., 108, 171-180. Lesht, B. M., 1995: An evaluation of ARM radiosonde operational performance. Preprints, Ninth Symposium on Meteorological Observations and Instrumentation, Charlotte, NC, Amer. Meteor. Soc., 6-10. Loehrer, S. M., T. A. Edmands, and J. A. Moore, 1996: TOGA COARE upper-air sounding data archive: development and quality control procedures. Bull. Amer. Meteor. Soc., 77, 2651-2671. Loehrer, S. M., S. F. Williams, and J. A. Moore, 1998: Results from UCAR/JOSS quality control of atmospheric soundings from field projects. Preprints, Tenth Symposium on Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 1-6. NWS, 1991: Micro-ART Observation and Rework Programs Technical Document, National Weather Service, National Oceanic and Atmospheric Administration, Washington, D.C., March 1991. Wade, C. G., 1995: Calibration and data reduction problems affecting National Weather Service radiosonde humidity measurements. Preprints, Ninth Symposium on Meteorological Observations and Instrumentation, Charlotte, NC, Amer. Meteor. Soc., 37-42. Williams, S. F., C. G. Wade, and C. Morel, 1993: A comparison of high resolution radiosonde winds: 6-second Micro-ART winds versus 10-second CLASS LORAN winds. Preprints, Eighth Symposium on Meteorological Observations and Instrumentation, Anaheim, California, Amer. Meteor. Soc., 60-65. Williams, S. F., S. M. Loehrer, and D. R. Gallant, 1998: Computation of high-resolution National Weather Service rawinsonde winds. Preprints, Tenth Symposium on Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 387-391.