ARM/GCIP NESOB 1997 Hourly Surface Composite 1.0 General Description The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Global Energy and Water Cycle Experiment (GEWEX) Continental-Scale International Project (GCIP) Near-Surface Observation Data Set (NESOB) 1997 Hourly Surface Composite is composed of data from several sources (i.e., Automated Surface Observing System (ASOS), Department Of Energy (DOE) Atmospheric Radiation Measurement Surface (ARMSFC), National Oceanic and Atmospheric Administration (NOAA) Wind Profiler Network (NPN), High Plains Climate Network (HPCN), National Climatic Data Center (NCDC) DATSAV3, and the NOAA Atmospheric Turbulence and Diffusion Division (ATDD) Little Washita Meteorological site) for the ARM/GCIP NESOB 1997 domain. Data from these sources (approximately 100 stations) were merged and quality controlled to form this Surface Composite. This Surface Composite contains data for the ARM/GCIP NESOB 1997 time period (01 April 1997 through 31 March 1998) and for the ARM/GCIP NESOB 1997 domain only. The ARM/GCIP NESOB 1997 domain is approximately 34N to 39N latitude and 94.5W to 100.5W longitude. 2.0 Detailed Data Description The ARM/GCIP NESOB 1997 Hourly Surface Composite is composed of data from several different sources which report data at hourly frequencies. 2.0.1 Automated Surface Observing System Algorithms The following are descriptions of the algorithms used by the Automated Surface Observing System (ASOS) to produce hourly surface data. Complete details may be found in the ASOS User's Guide (1992). The ASOS hourly values were produced from ASOS 5-minute data by extracting all parameters (except precipitation) from the 55-minute observation of the previous hour and assigning those values to the current hourly observation. The precipitation is the sum of the precipitation values from the 05-minute of the previous hour through the 00-minute of the current hour. The following are descriptions of the algorithms used by ASOS to produce five minute surface data. Complete details may be found in the ASOS User's Guide (1992). Temperature/Dewpoint ASOS takes 30-sec measurements and computes a 1-min average. A 5-min running average of these 1-min averages is computed. A minimum of four 1-min averages are required to compute a valid 5-min average. 5-min averages are rounded to the nearest degree F. ASOS will report the latest valid 5-min average during the previous 15-min period. If one is not available, the data are reported as "missing". If the 5-min average dewpoint is 1 or 2 degrees higher than the 5-min average temperature, then the dew point is reported equal to temperature. If the 5-min average dewpoint exceeds the 5- min average temperature by more than 2 degrees, the dewpoint is reported as "missing". Station Pressure and Derived Pressure Elements ASOS takes 10-sec measurements from at least two independent pressure sensors and computes respective 1-min averages. A minimum of 5 measurements are required to compute a 1-min average. The 1-min averages from each sensor are compared to verify that differences do not exceed 0.04" Hg. If the sensors are in agreement, the lowest pressure reading from all sensors is reported. If the sensor differences exceed 0.04" Hg, the data are reported as "missing". The reported pressure is then used in the computation of derived parameters (e.g., altimeter setting, sea level pressure, and pressure remarks such as tendency). Wind ASOS takes 5-sec measurements of wind speed and direction and computes a 2-min running average. Wind direction is rounded to the nearest degree and wind speed is rounded to the nearest knot. If the 2-min running average is 2 knots or less, the wind is reported as calm. The gust is computed using the highest 5-sec average wind speed during the past 10-min period. A gust is computed only when the 2-min running average exceeds 9 knots and the highest 5-sec measurement exceeds the 2-min running average by 5 knots (during the past minute). Precipitation ASOS takes 1-min accumulated measurements and computes total precipitation over 5-min, 15-min, hourly, 3-hr, 6-hr, and daily increments. Monthly totals are summed from daily totals. Present Weather There are currently two automated ASOS present weather sensors. They are the Precipitation Identification (PI) sensor which discriminates between rain and snow and the Freezing Rain (ZR) sensor. Although there is no ASOS "Obstruction To Vision" (OTV) sensor, ASOS algorithms evaluate data from multiple sensors (i.e., visibility, temperature, dewpoint temperature, and PI) and infer the presence of obstructions to vision (fog or haze). Once each minute the PI sensor output is stored in memory (up to 12 hours). The latest 10 minutes of data are examined. If 3 or more samples are missing, ASOS reports "missing" for that minute. If 2 or more samples indicate precipitation, and at least 8 one minute samples are available, the algorithm determines the type and intensity to report. In general, to report anything other than light precipitation (P-), two of the samples are required to be the same type. If there is a tie between two types of precipitation, snow is reported. The highest intensity obtained from two or more samples determines the present weather type and intensity that is reported. Once each minute the ZR sensor output is stored in memory (up to 12 hours). Data from the latest 15 minutes are used to compute the current minute freezing rain report. If 3 or more sensor outputs in the past 15- minutes are missing, the report is set to "missing". If at least one positive freezing rain report occurs in the past 15-minutes, freezing rain is reported for the current minute. If freezing rain is reported, the PI sensor report is examined and a hierarchical scheme is used to compute the present weather report. This scheme follows the familiar reporting hierarchy of LIQUID- FREEZING-FROZEN in ascending order of priority. ASOS does not report mixed precipitation. The beginning and ending times of one minute freezing rain reports are used in the hourly SAO reports. Once freezing rain has been sensed and the ambient air temperature is 36 degrees F or below, it will be carried in subsequent SAO reports for 15-minutes after it is no longer sensed. The OTV algorithm continuously monitors the reported visibility once each minute. When visibility drops below 7 statute miles, the algorithm obtains the current Dew Point Depression (DD) to distinguish between fog and haze. If the DD is < or equal to 4 degrees F, then fog will be reported and appended to the present weather report. If DD is > 4 degrees F and no present weather is reported by the PI and ZR sensors, then haze is reported as present weather. When present weather is reported by the PI and ZR sensors, haze is not reported. In the event DD is missing, visibility is used to discriminate between haze and fog. If visibility is < 4 miles, fog will be reported. When present weather is also reported, fog will be appended to the report. If visibility is > or equal to 4 miles but < 7 miles and no present weather is reported, then haze is reported. 2.0.2 DOE ARM Surface Algorithms The raw DOE ARM Surface (ARMSFC) 5-minute data provided to UCAR/JOSS is taken from the Surface Meteorological Observation System (SMOS) instrumentation. UCAR/JOSS ARMSFC hourly values were produced from ARMSFC 5-minute data by extracting all parameters (except precipitation) from the 55-minute observation of the previous hour and assigning those values to the current hourly observation. The precipitation is the sum of the precipitation values from the 05-minute of the previous hour through the 00-minute of the current hour. The ARMSFC 5-minute values were derived from ARMSFC 1-minute data. The ARMSFC instrument readouts were every second for all variables except 1 minute for barometric pressure. The detailed descriptions of the algorithms used to produce ARMSFC one minute data are not currently available. A description of the uncertainties for each recorded parameter is given below. Temperature/Relative Humidity The ARMSFC 1-minute surface air temperatures were measured at two meters above the ground with a resolution of 0.01 degrees C. The measurement of the air temperature has an uncertainty of +/- 0.45 C for wind speeds >/= 6.00 m/s, +/- 0.89 C for wind speeds >/= 3.00 m/s, +/- 1.46 C for wind speeds >/= 2.00 m/s, and +/- 3.07 C for wind speeds >/= 1.00 m/s. Errors included in the uncertainty for temperature are radiation error, sensor interchangeability, bridge resistor precision, and polynomial curve fitting. Relative humidity (RH) was measured at two meters above the ground. The uncertainty in the recorded relative humidity values is +/- 2.06% RH for RH values from 0 to 90% and +/- 3.04% RH for RH values from 90 to 100%. Errors included in the uncertainty for relative humidity are calibration uncertainty, repeatability, temperature dependence, long term (1 yr) stability, and A/D conversion accuracy. Wind speed dependence and radiation dependence have not been considered. Dewpoint temperature was computed by the University Corporation for Atmospheric Research/ Joint Office for Science Support (UCAR/JOSS) using station pressure, temperature, and relative humidity (Bolton, 1980). Station Pressure and Derived Pressure Elements The barometric station pressures were measured at 1 meter above the ground. The uncertainty in the recorded station pressure values is +/- 0.035 kPa. Errors included in the uncertainty are linearity, hysteresis, repeatability, calibration uncertainty, temperature dependence, and long-term (1 year) stability. Wind speed dependence was not considered. Wind Speed and Direction The ARMSFC 1-minute wind observations were taken at 10 meters above the ground. The uncertainty in the recorded wind speed values is +/- 1% from 2.5 to 30 m/s, -0.12 to +0.02 m/s at 2.0 m/s, -0.22 to +0.00 m/s at 1.5 m/s, -0.31 to -0.20 m/s at 1.0 m/s, -0.51 to -0.49 m/s at 0.5 m/s. Errors included in the uncertainty are calibration accuracy, data logger timebase accuracy, and bias by underestimation due to threshold. The latter assumes a normal distribution of winds about the mean with standard deviations ranging between 0.25 and 1.00 m/s. The uncertainty in the recorded wind direction values is +/- 5.0 deg for wind speeds >1.0 m/s and +/- 180.0 deg for wind speeds /= 30 day) variations by using 30 day standard deviations computed for each parameter when determining the acceptable limits for "good", "questionable", or "unlikely" flags. "Expected values" were computed from inverse distance weighted station observations within a 300 km Radius Of Influence (ROI) centered about the station being quality controlled (the station being quality controlled was excluded); i.e.; theta_e = / Where theta_e is the "expected value" of the parameter at the site in question, theta(i) is the observed value of the parameter at site i, w(i) is the weighting factor for site i (here the inverse of the distance between site i and the station being quality controlled), and <...> is the sum over all stations "i" in the current ROI that have valid observations of the parameter at the time in question. Data were always compared at like solar times. To determine an observation's HQC flag setting, the difference between the actual observation and its "expected value" was compared to that parameter's normalized standard deviation. Normalizing factors (also called the sensitivity coefficients) were chosen to control the "good", "questionable", and "unlikely" flag limits for each parameter. See Table 3-1 for ARM/GCIP NESOB 1997 normalizing factors. Table 3-2 contains the HQC flag limit ranges derived from the normalizing factors given in Table 3-1 and estimated standard deviations for each parameter so that 95% of the QC limits applied to the ARM/GCIP NESOB 1997 data fell within these ranges. For example, 95% of the observed station pressure values that were flagged as "good" were within 1.8 mb of the expected value. The significant overlap of the ranges seen in Table 3-2 was partially due to seasonal and station differences in standard deviations. The actual HQC limits applied at any particular time depended upon the dynamic nature of the particular station's parameter values over time. Data were never changed, only flagged. HQC was only applied to station pressure, sea level pressure, calculated sea level pressure, temperature, dew point, wind speed and wind direction. If the calculated sea level pressure quality control information was available, its flag was applied to the station and sea level pressures. If the calculated sea level pressure could not be quality controlled, the sea level pressure quality control flag was applied to the station pressure. If the sea level pressure could not be quality controlled, the station pressure quality control flag was not overridden. Table 3-1 Normalizing factors used for ARM/GCIP NESOB 1997 Hourly Surface Composite Parameter Good Questionable Unlikely --------- ---- ------------ -------- Station Pressure 0.2 0.2 0.5 Sea Level Pressure (SLP) 0.2 0.2 0.5 Calculated SLP 0.4 0.4 1.0 Dry Bulb Temperature 0.5 0.5 1.0 Dew Point Temperature 0.5 0.5 1.0 Wind Speed 2.25 2.25 4.0 Wind Direction 1.22 1.22 2.2 Table 3-2 Ranges of HQC flag limit values for ARM/GCIP NESOB 1997 Hourly Surface Composite Parameter Good Questionable Unlikely --------- ---- ------------ -------- Station Pressure (mb) < 1.8 [0.4-4.4] > 1.0 Sea Level Pressure (mb) < 1.9 [0.5-4.9] > 1.1 Calculated SLP (mb) < 4.0 [0.9-10.1] > 2.2 Dry Bulb Temperature (deg.C) < 3.3 [1.1-6.6] > 2.2 Dew Point Temperature (deg.C) < 3.3 [0.9-6.6] > 1.7 Wind Speed (m/s) < 6.8 [2.3-12.2] > 4.0 Wind Direction(degrees) < 160.4 [61.0-180.0] >110.0 The squall/gust wind speed data were not quality controlled. General consistency checks were also applied to the dry bulb temperature, wind direction, and the relationship between precipitation and cloud amount/cloud cover. If the dew point temperature was greater than the dry bulb temperature both values were coded "questionable". Also, wind direction for observed "calm" winds was given the same QC code as the wind speed. If precipitation was reported, but the cloud amount was "none" or "clear", then both the cloud amount and precipitation values were coded "questionable". Several impossible values were also checked. Negative wind speeds were coded "unlikely". Negative squall/gust wind speeds were coded "unlikely". Wind directions of less than 0 degrees or greater than 360 degrees were coded "unlikely". If these consistency checks would have upgraded the quality control flags previously set by HQC or gross limit checks, then they were not applied. However, if these consistency checks would have degraded the previously set QC flags, they were applied. The JOSS HQC scheme relied on spatial and temporal continuity to flag the data. It has been shown that this method works very well for temperature, dew point, pressure, and wind speed, but is not a very good scheme for the wind direction. The flags appear to be overly lax and perhaps could be tightened. Gross limit checks were also used to determine the quality of the precipitation values. The gross limits are shown in Table 3-3. Certain "questionable" and "unlikely" data values were also manually inspected. After inspection, the quality control flag may have been manually modified to better reflect the physical reasonableness of the data. Data were never modified, only flagged. Negative precipitation was also coded "unlikely". Since the precipitation gages in the MOCAWS and Wisconsin AWON networks are not heated during the winter months, the quality control flags for all "unchecked", "good", and "trace" precipitation values in these networks have been reset to "questionable". The meanings of the possible quality control flags are listed in table 3-4. Table 3-3 - Precipitation Gross Limit Values Parameter Good Questionable Unlikely --------- ---- ------------ -------- Hourly Precipitation < 20.0 mm >= 20.0 mm >= 50.0 mm Table 3-4 - Quality Control Flags QC Code Description ------- ----------- U Unchecked G Good M Normally recorded but missing. D Questionable B Unlikely N Not available or Not observed X Glitch E Estimated C Reported value exceeds output format field size or was negative precipitation. T Trace precipitation amount recorded I Derived parameter can not be computed due to insufficient data. 4.0 References ASOS User's Guide, ASOS Project Office, NOAA, National Weather Service, Washington D.C., June 1992. Barnes, S. L., 1964: A technique for maximizing details in numerical weather map analysis. J. Appl. Meteor., 3, 396-409. Bolton, D., 1980: The computation of equivalent potential temperature., Mon. Wea. Rev., 108, pp 1046-1053. Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87, 367-374. National Climatic Data Center, 1999a: Data documentation for DATSAV3 surface, TD-9956, April 20, 1999, National Climatic Data Center, 151 Patton Ave, Asheville, NC 28801-5001 USA. National Climatic Data Center, 1999b: Surface Hourly Abbreviated Format, 09/02/99 National Climatic Data Center, 151 Patton Ave, Asheville, NC 28801-5001 USA. Smithsonian Meteorological Tables, Table No. 65, p.269. Smithsonian Institution Press, Washington, D.C., September, 1949. Wallace, J.M., P.V. Hobbs, 1977: Atmospheric Science, Academic Press, 467 pp. World Meteorological Organization (WMO), 1988: Manual on Codes Volume I, Part B - Binary Codes. WMO, Geneva, Switzerland. Wexler, A., and W. A. Wildhack, 1963: Humidity and Moisture. Vol. 3: Fundamentals and Standards. Reinhold Publ. Corp., 562 pp.