NESOB 1996 ARM-CART and National Weather Service Interpolated 5 hPa Vertical Resolution Sounding Composite Dataset 1.0 Dataset Overview This is one of the upper air sounding datasets developed for the Global Energy and Water Cycle Experiment (GEWEX) Continental-scale International Project (GCIP) 1996 Near Surface Observation (NESOB-96) Data Set conducted from 01 April to 30 September 1996. Included in this dataset are five Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) Clouds And Radiation Testbed (CART) sounding stations; Lamont (Central Facility), Morris (B5), Purcell (B6), and Vici (B4), OK and Hillsboro (B1), KS and 12 National Weather Service (NWS) rawinsonde stations in the ESOP 1996 domain. The NWS soundings were typically released at 00 and 12 UTC, however, several stations had additional releases on an as requested basis. At each of the ARM boundary facilities (B1, B4, B5, and B6) soundings were typically released once per weekday at 1800 UTC and at the central facility five times per weekday at 06, 12, 15, 18, and 21 UTC. However, during Intensive Observing Periods (IOPs) the sounding frequency increased to up to 8 per day (3-hourly; including weekends) at each of the sites. The soundings are terminated at the point nearest to, but not over, 3000 m. This dataset is a composite of data from all ARM-CART and National Weather Service sounding platforms interpolated to a constant vertical resolution of 5 hPa. 2.0 Detailed Dataset Description 2.1 5 hPa Interpolation Algorithm The 'native' resolution data for every sounding were interpolated to 5 hPa vertical resolution files. The surface data point was kept as the initial level in each sounding. The first interpolated data point was at the next lowest pressure evenly divisible by 5 and then every 5 hPa pressure level beyond that point to either 50 hPa or the lowest pressure level reached by the radiosonde, whichever came first. The first 15 lines of each file (the header information) were kept without change. For the interpolation, the software searched for two data points around the desired pressure level. The search was conducted by looking for two valid (i.e. non-missing) data points around the desired pressure level, while also paying attention to the time difference between the two data points as well as their quality control flags. There was a search for the two best possible data points to use in the interpolation. If the desired pressure level was within the original dataset, that data point was used without interpolation. There was first a search for values flagged as good within some time range (50 sec for temperature, humidity, and wind and 100 sec for pressure; hereafter termed the ARANGE) and the interpolated data point was flagged as good. Failing that, it searched for values flagged as estimated within the same time range and the interpolated data point was flagged as estimated. Then the search went for good values within a wider time range (100 sec for temperature, humidity, and wind and 200 sec for pressure; hereafter termed the BRANGE) the flag for the interpolated data point here was then degraded (even though two 'good' data points were used there was a significant time difference between them) to questionable. Then, in turn, estimated values within the BRANGE were used (flag set to questionable), questionable values within the BRANGE (flag set to bad), good values greater than the BRANGE apart (flag set to bad), estimated values greater than BRANGE apart (flag set to bad), questionable values greater than BRANGE apart (flag set to bad), finally any bad values (flag set to bad). This search was conducted separately for each interpolated variable (pressure, temperature, relative humidity, and the u and v wind components. Thus for each interpolated data point, the quality control flag was set to the worst case among the data points used in the interpolation, except, for each time range apart, the quality control flag was degraded one level (i.e. good to questionable, etc). The quality control flags should be carefully heeded in these files. While some of the data may look good, it may have been interpolated over large pressure intervals, and thus be suspect. For each interpolated data point the dew point was calculated from the temperature and relative humidity (Bolton 1980) and the total wind speed and direction were calculated from the interpolated u and v component values. Also, the altitude and time were interpolated using the same data points used for the pressure interpolation. The ascension rate was recalculated based on the time and altitude values from the two data points used to interpolate the 5 hPa data point. Thus the ascension rate values do not reflect the values based on the interpolated data. The latitude and longitude values were interpolated using the same data points used in the wind component interpolation. 2.2 High Resolution Data Description 2.2.1 National Weather Service High-Resolution Sounding Algorithms The detailed description of NWS sounding collection and instrumentation is located in NWS (1991). The native vertical resolution of the NWS soundings is six seconds. The NWS soundings during NESOB 1996 utilized either the VIZ Inc. or the Vaisala RS-80 radiosonde. Vaisala radiosondes were utilzed at Amarillo (AMA), TX; Denver (DNR), CO; Santa Teresa (EPZ), NM; and Midland (MAF), TX. VIZ radiosondes were used by Dodge City (DDC), KS; Fort Worth, TX; Little Rock, AK; Norman, OK; Shreveport, LA; Springfield, MO; and Topeka, KS. JOSS applied a correction to the VIZ radiosonde relative humidity (RH) calculations. Using the observed temperature and RH, the resistance ratio was rederived by iteration using both the "1A" and "1B" coefficients. With the calculated resistance ratio, the observed temperature and using the "1A" coefficients only, the new RH was calculated. The use of the raw 6-sec resolution elevation and azimuth angle data to derive the winds sometimes led to large oscillations in wind speed, due to the presence of oscillations in the elevation angle data, particularly at low elevation angles. The general approach to correct this problem was to remove the outlier radiosonde position data before computing the wind components (Williams et al. 1993). For both the azimuth and elevation angles from 360 sec to the end of the sounding, a ninth order polynomial was fit to the curve. The residuals were calculated and compared to the observed values. The outliers of the residuals were then removed. Then to help correct the more extensive problems at low elevation angles within 10 degrees of the limiting angles (LA) some additional smoothing was applied. If the elevation angle was between (LA + 7.5) and (LA + 10), the new elevation angle was computed with a 2 min linear fit. If the elevation angle was between (LA + 5) and (LA + 7.5), the new elevation angle was computed with a 3 min linear fit. If the elevation angle was less than (LA + 5), the new elevation angle was computed with a 4 min linear fit. If the number of observations with low elevation angles was greater than 20% of the total number of observations for the sounding no frequency smoothing occurred. Then, for the elevation angle only, a finite Fourier analysis was performed on the residuals. Periods from 90-190 sec were removed and those below 30 sec were flattened. Finally, a 2 min second order polynomial was then fit to the position to derive the u and v wind components, except for the beginning and end minute (or 1.5 minutes if over 50 mb) which used a 3 min fit. If there were less than 15% of the total number of points, not counting the beginning or end of the flight, on one side of the point for which the wind value was being computed, a linear fit was used. For further information on this methodology and its changes since Williams et al. (1993) please see Williams, et al. (1998). 2.2.2 ARM-CART High Resolution Sounding Algorithms All ARM-CART upper air soundings were converted from netcdf to University Corporation for Atmospheric Science/Joint Office for Science Support (UCAR/JOSS) Cross Chain LORAN Atmospheric Sounding System (CLASS) Format (JCF). JCF is a version of the National Center for Atmospheric Research (NCAR) CLASS format and is an ASCII format consisting of 15 header records for each sounding followed by the data records with associated Quality Control (QC) information. 2.2.3 UCAR/JOSS QC Procedures for High Resolution Soundings The following details QC procedures for the original high resolution soundings. However, quality control information derived by the following determines what points were used during the interpolation process (see Section 2.1). Some further information on the QC processing conducted by JOSS can be found in Loehrer et al. (1996) and Loehrer et al. (1998). ARM-CART soundings were Quality Controlled (Lesht 1995) and provided to UCAR/JOSS by the ARM-CART Project. This dataset underwent a JOSS QC process which consisted of automated internal consistency checks. This included gross limit checks on all parameters (see Section 2.2.3.1) and rate-of-change checks on temperature, pressure and ascension rate (see Section 2.2.3.2). JOSS did not perform visual quality control procedures on this data. NWS soundings underwent a two-stage QC process. First, the dataset underwent automated internal consistency checks. Second, each sounding was visually examined (see Section 2.2.3.3) to verify those parameters that are too variable for automatic checks (wind speed, wind direction and moisture). This stage of the QC process also allows for a verification of the QC flags generated by the automatic checks. 2.2.3.1 Gross Limit Checks These checks were conducted on each sounding and data were automatically flagged as appropriate. Only the data point under examination was flagged. JOSS conducted the following gross limit checks on the 1996 sounding dataset. In the table P = pressure, T = temperature, RH = relative humidity, U = U wind component, V = V wind component, B = bad, and Q = questionable. __________________________________________________________________ Parameter(s) Flag Parameter Gross Limit Check Flagged Applied __________________________________________________________________ Pressure < 0 mb or > 1030 mb P B Altitude < 0 m or > 35000 m P, T, RH Q Temperature < -80C or > 45C T Q Dew Point < -99.9C or > 30C RH Q > Temperature T, RH Q Relative Humidity < 0% or > 100% RH B Wind Speed < 0 m/s or > 100 m/s U, V Q > 150 m/s U, V B U Wind Component < 0 m/s or > 100 m/s U Q > 150 m/s U B V Wind Component < 0 m/s or > 100 m/s V Q > 150 m/s V B Wind Direction < 0 deg or > 360 deg U, V B Ascent Rate < -10 m/s or > 10 m/s P, T, RH Q _________________________________________________________________ 2.2.3.2 Vertical Consistency Checks These checks were conducted on each sounding and data were automatically flagged as appropriate. These checks were started at the lowest level of the sounding and compared neighboring 6-sec data points for NWS soundings or 6-second averaged points for ARM-CART soundings (except for NWS soundings at pressures less than 100 mb and for where 30-sec average values were used). In the case of checks ensuring that the values increased/decreased as expected, only the data point under examination was flagged. However, for the other checks, all of the data points used in the examination were flagged. All items within the table are as previously defined. _____________________________________________________________________ Vertical Consistency Parameter(s) Flag Parameter Check Flagged Applied _____________________________________________________________________ Time decreasing/equal None None Altitude decreasing/equal P, T, RH Q Pressure increasing/equal P, T, RH Q > 1 mb/s or < -1 mb/s P, T, RH Q > 2 mb/s or < -2 mb/s P, T, RH B Temperature < -15 C/km P, T, RH Q < -30 C/km P, T, RH B > 5 C/km (not applied at p < 150mb) P, T, RH Q < 30 C/km (not applied at p < 150mb) P, T, RH B Ascent Rate change of > 3 m/s or < -3 m/s P Q change of > 5 m/s or < -5 m/s P B _____________________________________________________________________ 2.2.3.3 Visual Quality Control Procedures NWS soundings were visually examined for problems that are difficult to captured via the automated checks described in Sections 2.2.3.1 and 2.2.3.2. These problems typically included oddities in the dew point and wind profiles. These two parameters can be highly variable, and hence, the automated checking is less effective. The visual checking procedure has two main objectives: First, as a check on the results provided by the automatic checks, and second, as a more stringent check on the more variable parameters. 3.0 JOSS CLASS Format (ASCII text) Description 3.1 Header records The header records (15 total records) contain data type, project ID, site ID, site location, actual release time, nominal release time, and possibly other specialized information. The first five header lines contain information identifying the sounding, and have a rigidly defined form. The following 6 header lines are used for auxiliary information and comments about the sounding, and they vary significantly from data set to data set. The next line (line 12) contains the Nominal date and time of the release. The last 3 header records contain header information for the data columns. Line 13 holds the field names, line 14 the field units, and line 15 contains dashes ('-' characters) delineating the extent of the field. The six standard header lines are as follows: Line Label (fixed to 35 char in length) Contents 1 Data Type: Description of type and resolution of data. 2 Project ID: ID of weather project. 3 Release Site Type/Site ID: Description of release site. 4 Release Location (lon,lat,alt): Position of release site, in format described below. 5 UTC Release Time: Time of release, in format: yyyy, mm, dd, hh:mm:ss 12 UTC Nominal Release Time: Nominal release time. The release location is given as: lon (deg min), lat (deg min), lon (dec. deg), lat (dec. deg), alt (m) Longitude in deg min is in the format: ddd mm.mm'W where ddd is the number of degrees from True North (with leading zeros if necessary), mm.mm is the decimal number of minutes, and W represents W or E for west or east longitude, respectively. Latitude has the same format as longitude, except there are only two digits for degrees and N or S for north/south latitude. The decimal equivalent of longitude and latitude and station elevation follow. The seven non-standard header lines may contain any label and contents. The label is padded to 35 characters to match the standard header lines. 3.2 Data records The data records each contain time from release, pressure, temperature, dew point, relative humidity, U and V wind components, wind speed and direction, ascent rate, balloon position data, altitude, and quality control flags (see QC code description). Each data line contains 21 fields, separated by spaces, with a total width of 130 characters. The data are right-justified within the fields. All fields have one decimal place of precision, with the exception of latitude and longitude, which have three decimal places of precision. The contents and sizes of the 21 fields that appear in each data record are as follows: Field Format Missing No. Width Parameter Units Value ---------------------------------------------------------------------- 1 6 F6.1 Time Seconds 9999.0 2 6 F6.1 Pressure Millibars 9999.0 3 5 F5.1 Dry-bulb Temperature Degrees C 999.0 4 5 F5.1 Dew Point Temperature Degrees C 999.0 5 5 F5.1 Relative Humidity Percent 999.0 6 6 F6.1 U Wind Component Meters/Second 9999.0 7 6 F6.1 V Wind Component Meters/Second 9999.0 8 5 F5.1 Wind Speed Meters/Second 999.0 9 5 F5.1 Wind Direction Degrees 999.0 10 5 F5.1 Ascension Rate Meters/Second 999.0 11 8 F8.3 Longitude Degrees 9999.0 12 7 F7.3 Latitude Degrees 999.0 13 5 F5.1 Variable (see below) 999.0 14 5 F5.1 Variable (see below) 999.0 15 7 F7.1 Altitude Meters 99999.0 16 4 F4.1 QC flag for Pressure Code (see below) 99.0 17 4 F4.1 QC flag for Temperature Code (see below) 99.0 18 4 F4.1 QC flag for Humidity Code (see below) 99.0 19 4 F4.1 QC flag for U Component Code (see below) 99.0 20 4 F4.1 QC flag for V Component Code (see below) 99.0 21 4 F4.1 QC flag for Ascension Rate Code (see below) 99.0 ---------------------------------------------------------------------- Fields 13 and 14 are `variable' because depending on the sounding system the variables used in these positions can vary. For this data set, field 13 contains the Elevation Angle and field 14 contains the Azimuth Angle both in degrees. Fields 16 through 21 contain the Quality Control information (flags) generated locally at JOSS. These flags are based on the automated or visual checks made. See Section 2.2.3 for further information. The JOSS QC flags are as follows: Code Description ---------------------------------------------------------------------- 99.0 Unchecked (QC information is `missing.') (`UNCHECKED') 1.0 Checked, datum seems physically reasonable. (`GOOD') 2.0 Checked, datum seems questionable on physical basis. (`MAYBE') 3.0 Checked, datum seems to be in error. (`BAD') 4.0 Checked, datum is interpolated. (`ESTIMATED') 9.0 Checked, datum was missing in original file. (`MISSING') ---------------------------------------------------------------------- 3.3 Sample data The following is a sample portion of a JOSS CLASS format file including header records. The data portion is much longer than 80 characters and, therefore, wraps around to a second line. Data Type: Sounding Project ID: ARM/GCIP NESOB-96 Release Site Type/Site ID: B6 : Purcell Release Location (lon,lat,alt): 097 24.00'W, 35 0.00'N, -97.40, 35.00, 344.0 UTC Release Time (y,m,d,h,m,s): 1996, 05, 24, 17:29:00 / / / / / / Nominal Release Time (y,m,d,h,m,s):1996, 05, 24, 18:00:00 Time Press Temp Dewpt RH Uwind Vwind Wspd Dir dZ Lon Lat Elev Azim Alt Qp Qt Qh Qu Qv Qdz sec mb C C % m/s m/s m/s deg m/s deg deg deg deg m code code code code code code ------ ------ ----- ----- ----- ------ ------ ----- ----- ----- -------- ----- - ----- ----- ------- ---- ---- ---- ---- ---- ---- 0.0 968.5 30.7 18.5 48.0 -3.8 1.4 4.0 110.0 999.0 -97.420 34.9 999.0 999.0 344.0 3.0 3.0 3.0 99.0 99.0 9.0 6.1 965.0 29.1 17.5 49.6 9999.0 9999.0 999.0 999.0 5.4 9999.000 999.0 0 999.0 999.0 376.7 3.0 3.0 3.0 9.0 9.0 99.0 13.7 960.0 27.4 16.7 51.8 -2.4 6.0 6.5 158.0 6.0 -97.420 34.9 0 999.0 999.0 422.3 3.0 3.0 3.0 99.0 99.0 99.0 4.0 Quality Control Procedures for 5 hPa Interpolated Dataset No additional quality control was applied to the interpolated dataset. The quality control procedures were conducted on the high resolution data sets and the results were applied during the interpolation (see Section 2.1.3). 5.0 Data Quality Issues 5.1 ARM-CART In terms of items checked by the automated quality control processing, the ARM CART soundings performed well, with about average numbers of data points flagged. However, since these data have not undergone visual quality control processing, there are many problems that have not been flagged. This is especially true for the wind and relative humidity values. So these values should be used with some caution. As in past years, the B4 (Vici, OK) soundings had much higher occurrence rates of superadiabatic layers than the other sites. This year, however, the problem has been significantly reduced. The other sites had 0.5 - 0.7% of all data points registering as superadiabatic (< -15 C/km). These values were typical of soundings from other projects. B4, however, had 1.3% of all data points registering as superadiabatic. 5.2 National Weather Service 5.2.1 Near Surface Winds A common problem in near surface wind speed values calculated from the 6-second position data is that the first radiosonde wind speed is much higher than the independently measured surface value. The calculated radiosonde winds then decrease rapidly so that within about 60 s (20-30 mb) after release the wind speeds are more realistic. The cause of this appears to be the acceptance of radiosonde position data prior to a"good lock" being achieved on the radiosonde by the tracking system. Thus there appear to be rapid positional shifts of the radiosonde while the tracking system "searches" for the radiosonde. 5.2.2 Wind Oscillations Despite the extensive efforts to remove oscillations in wind speeds caused by oscillations in elevation angles (see Section 2.2.1) there are occasional cases with remaining oscillations. Most of the remaining oscillations have periods just slightly longer than the 190 s maximum point of our notch filter. 5.2.3 Low Level Humidity Problems Santa Teresa Low Level Humidity Problem A new variety of low level humidity problem has appeared at Santa Teresa, NM (formerly El Paso, TX) since its conversion over to the Vaisala radiosonde. This type of problem did not appear in previous years when the site used VIZ soundings. This problem appears as an anomalous moist layer near the surface. An example is shown below for the sounding taken at 00 UTC 22 June 1996 at EPZ. Time Press Temperature Dew Relative Point Humidity -------------------------------------------------------- 0.0 868.8 40.0 4.0 11.0 6.0 865.4 39.4 2.2 10.0 12.0 862.1 38.8 0.3 9.0 18.0 858.5 38.2 2.6 11.0 24.0 854.9 37.6 4.5 13.0 30.0 851.3 36.9 7.0 16.0 36.0 847.7 36.3 8.2 18.0 42.0 844.0 35.7 10.0 21.0 48.0 840.5 35.1 10.9 23.0 54.0 837.7 34.6 7.6 19.0 60.0 835.0 34.1 2.9 14.0 66.0 832.2 33.6 -2.2 10.0 72.0 828.8 33.3 -2.4 10.0 78.0 825.8 33.0 -2.6 10.0 84.0 823.7 32.7 -1.6 11.0 90.0 821.5 32.4 -1.8 11.0 In this example the RH starts at about 11% then increases to over 20% then returns back to about 11%. But the temperature lapse rate is nearly dry adiabatic and indicative of a well mixed atmosphere which should not show such dramatic humidity contrasts. This problem appeared almost exclusively at EPZ (it did rarely appear at other sites, and always under the same atmospheric conditions as when it occurred at EPZ) and it appeared almost exclusively at 00 UTC under dry adiabatic conditions. There are often many missing (and hence interpolated by the Vaisala system) data points surrounding these anomalous moist layers and such missing data points do not appear nearly as often in soundings without this moist layer. The exact cause is not yet known, but given the prevalent occurrence of missing data surrounding these anomalous moist layers, there are significant instrumentation problems occurring. It may be due to some atmospheric effects that occur in such hot and dry conditions, possibly dust is having some effect. But it is not yet clear. General Low Level Humidity Problem Another frequent occurrence in NWS soundings is what appears to be a very dry surface relative to the radiosonde values. An example is shown below from Albuquerque, NM at 00 UTC 25 April 1996. Time Press Temperature Dew Relative Point Humidity ------ ------ --------------- ----- ----------- 0.0 834.4 28.9 -14.5 5.0 6.0 830.9 28.1 -8.7 8.4 12.0 827.4 27.7 -9.7 7.9 18.0 823.9 27.3 -10.0 7.9 24.0 820.5 26.8 -10.4 7.9 The independently measured surface RH is about 4% less than the first radiosonde value. Then the remainder of the radiosonde values appear to be consistent with the first radiosonde value and not the surface value. It has been suggested (Wade 1995) that when this problem occurs, the entire sounding RH may be to moist. 6.0 Composite Sounding Platforms 6.1 National Weather Service Upper Air Release Locations The following is a list of the NWS upper air sites contained in the NESOB 5 hPa interpolated sounding composite. The list contains each station's 3 letter identifier, station name, 2 letter state code, latitude and longitude in degrees, elevation in meters and sonde instrumentation used. All sites used radiotheodolite wind finding. ABQ Albuquerque NM 35.00 -106.60 1615 VIZ AMA Amarillo TX 35.20 -101.70 1094 Vaisala RS 80 DNR Denver CO 39.80 -104.90 1611 Vaisala RS 80 DDC Dodge City KS 37.80 -100.00 790 VIZ EPZ Santa Teresa NM 31.90 -106.70 1257 Vaisala RS 80 FWD FT WORTH TX 32.80 -97.30 198 VIZ LIT North Little Rock AR 34.80 -92.30 172 VIZ MAF Midland TX 32.00 -102.20 873 Vaisala RS 80 OUN Norman OK 35.20 -97.40 357 VIZ SHV Shreveport LA 32.50 -93.80 83 VIZ SGF Springfield MO 37.20 -93.40 390 VIZ TOP Topeka KS 39.10 -95.60 270 VIZ 6.2 ARM-CART Upper Air Release Locations The following is a list of the ARM-CART upper air sites contained in the NESOB 5 hPa interpolated sounding composite. The list contains each station's 3 letter identifier, station name, 2 letter state code, latitude and longitude in degrees, elevation in meters and sonde instrumentation used. All sites used Loran-C wind finding. SB1 Hillsboro KS 38.30 -97.30 447 Vaisala RS 80-15LH SB4 Vici OK 36.10 -99.20 622 Vaisala RS 80-15LH SB5 Morris OK 35.70 -95.80 217 Vaisala RS 80-15LH SB6 Purcell OK 35.00 -97.40 344 Vaisala RS 80-15LH SC1 Lamont OK 36.60 -97.50 315 Vaisala RS 80-15LH 7.0 References Bolton, D., 1980: The Computation of Equivalent Potential Temperature. Mon. Wea. Rev., 108, 171-180. Lesht, B. M., 1995: An evaluation of ARM radiosonde operational performance. Preprints, Ninth Symposium on Meteorological Observations and Instrumentation, Charlotte, NC, Amer. Meteor. Soc., 6-10. Loehrer, S. M., T. A. Edmands, and J. A. Moore, 1996: TOGA COARE upper-air sounding data archive: development and quality control procedures. Bull. Amer. Meteor. Soc., 77, 2651-2671. Loehrer, S. M., S. F. Williams, and J. A. Moore, 1998: Results from UCAR/JOSS quality control of atmospheric soundings from field projects. Preprints, Tenth Symposium on Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 1-6. NWS, 1991: Micro-ART Observation and Rework Programs Technical Document, National Weather Service, National Oceanic and Atmospheric Administration, Washington, D.C., March 1991. Wade, C. G., 1995: Calibration and data reduction problems affecting National Weather Service radiosonde humidity measurements. Preprints, Ninth Symposium on Meteorological Observations and Instrumentation, Charlotte, NC, Amer. Meteor. Soc., 37-42. Williams, S. F., C. G. Wade, and C. Morel, 1993: A comparison of high resolution radiosonde winds: 6-second Micro-ART winds versus 10-second CLASS LORAN winds. Preprints, Eighth Symposium on Meteorological Observations and Instrumentation, Anaheim, California, Amer. Meteor. Soc., 60-65. Williams, S. F., S. M. Loehrer, and D. R. Gallant, 1998: Computation of high-resolution National Weather Service rawinsonde winds. Preprints, Tenth Symposium on Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 387-391.