GIDS-1 Hourly Surface Composite 1.0 General Description The Hourly Surface Composite is one of several surface datasets provided as part of the GEWEX Continental-scale International Project (GCIP) Initial Dataset One (GIDS-1). The Hourly Surface Composite is composed of data from several sources (i.e., National Climatic Data Center Surface Aviation Observations (NCDC SAO), High Plains Climate Network, Illinois Climate Network, NOAA/Environmental Research Lab oratory (ERL) PROFS Mesonet, Wind Profiler Demonstration Network surface data, and GIDS-1 area extract from STORM-FEST Hourly Surface Composite). Data from these sources (over a thousand stations) were merged and quality controlled to form this surface composite. The Hourly Surface Composite contains data for the GIDS-1 time period (01 February 1992 through 30 April 1992) and for the GIDS-1 area of interest, only. The GIDS-1 area is approximately 85W to 106W longitude and 30N to 45N latitude. 2.0 Detailed Data Description The Hourly Surface Composite is composed of data from several different sources which report data at different frequencies, so it is important to note the conversion processes were applied to each these input datasets. Note that the AWOS, ASOS, and PAM data are only present in the STORM-FEST portion of this Hourly Surface Composite (i.e., 01 February 1992 through 15 March 1992). Handar AWOS data Handar AWOS data were based upon original 20-minute data. There are no present weather or sea level pressure reported for Handar data. Precipitation is the accumulated precipitation for the hour, but all other parameters are the values reported for the 20-minute observation that happens to fall on the hour. Qualimetrics AWOS data Qualimetrics AWOS data are based upon original 20-minute data. Precipitation is the accumulated precipitation for the hour, but all other parameters are the values reported for the 20-minute observation that happens to fall on the hour. Station pressure is calculated from altimeter setting. Sea level pressure is not reported for the Qualimetrics AWOS data. Present weather is not reported for Qualimetrics AWOS data. AWOS 1-minute data Data from these stations were summarized into hourly amounts and included in this hourly composite. Station pressure is calculated from altimeter setting. Sea level pressure and present weather are not reported. ASOS data In this hourly composite the ASOS hourly data were used. The processing to create the ASOS hourly reports was done at the ASOS stations. Refer to the ASOS documentation to understand how the ASOS stations create the hourly reports from raw data. Hourly ASOS data was taken nominally on the hour, but actual times are usually 2-4 minutes before the hour. The initial ASOS hourly observations were in SAO format. The ASOS 5-minute data were processed to obtain hourly observations and were also included in this Hourly Surface Composite. The hourly observations were generated by selecting the data from the observation period ending at 55 minutes before the hour for all parameters except precipitation, which was totaled from minute 56 of the preceding hour to minute 55 of the current hour. NCDC SAO's The NCDC SAO data contains selected airways observations from the Surface Records Retention System (SRRS), which is a recording of surface data from the NWS AFOS network. All standard hourly observations inside a window of +/- 15 minutes of the nominal hour were selected, converted to standard format and units, and included in the hourly composite. Other observations, such as specials and "hourly" observations that fell outside the time window, were also quality- controlled, and are available as a separate dataset on CODIAC. Wind Profiler Demonstration Network Hourly Surface Observations These data are the hourly surface observations from those wind profilers equipped with surface instrumentation. The data were obtained, converted to standard format and units, and included in this hourly composite. High Plains Climate Network The High Plains Climate Network reports only the moisture measurement of relative humidity, and does not report any pressure parameter. This hourly composite includes dewpoint as the moisture parameter. To convert relative humidity to dewpoint, the station elevation and the standard atmosphere were used to generate an estimate of the station pressure, which was then used in the relative humidity to dewpoint conversion. NOAA/Environmental Research Laboratory (ERL) PROFS Mesonet The data from this network of mesonet stations were obtained from ERL Forecast Systems Lab (FSL). The 5-minute frequency data were processed to obtain hourly observations and included in the Hourly Composite. The hourly observations were generated by selecting the data from the observation period ending at 55 minutes before the hour for all parameters except precipitation, which was totaled from minute 56 of the preceding hour to minute 55 of the current hour. NCAR PAM Mesonet The data from this network of mesonet stations were obtained from NCAR/RDP. The 5-minute frequency data were processed to obtain hourly observations and were included in this Hourly Surface Composite. The hourly observations were generated by selecting the data from the observation period ending at 55 minutes before the hour for all parameters except precipitation, which was totaled from minute 56 of the preceding hour to minute 55 of the current hour. Illinois Climate Network The data from this network of mesonet stations were obtained from the Illinois State Water Survey. The 5-minute frequency data were processed to obtain hourly observations and were included in this Hourly Surface Composite. The hourly observations were generated by selecting the data from the observation period ending at 55 minutes before the hour for all parameters except precipitation, which was totaled from minute 56 of the preceding hour to minute 55 of the current hour. The Hourly Surface Composite format contains ten metadata parameters followed by 38 data parameters and flags. This is the Office of Field Project Support (OFPS) Quality Control Format (QCF). The metadata parameters describe the station location and time at which the data were collected. The time of observation is reported both in UTC Nominal and UTC actual time. Days begin at UTC hour 0000 and end at UTC hour 2300. The data parameters are valid for the reported times. All records are hourly and identical in length. The table below details the data parameters in each record. Several data parameters have an associated Quality Control (QC) Flag Code which is assigned during the OFPS quality control processing. For a list of possible QC Flag values see the Quality Control Section 3.0. Parameters Units Date of Observation UTC Nominal Time of Observation UTC Nominal Date of Observation UTC actual Time of Observation UTC actual Network Identifier Abbreviation of platform name Station Identifier Network Dependent Latitude Decimal degrees, South is negative Longitude Decimal degrees, West is negative Station Occurrence Unitless Station Elevation Meters Station Pressure, QC flag Hectopascals (mb) Reported Sea Level Pressure, QC flag Hectopascals (mb) Computed Sea Level Pressure, QC flag Hectopascals (mb) Dry Bulb Temperature, QC flag Celsius Dew Point, QC flag Celsius Wind Speed, QC flag m/s Wind Direction, QC flag Degrees Total Precipitation, QC flag mm Squall/Gust Indicator Code Value Squall/Gust Value, QC flag m/s Present Weather, QC flag Code Value Visibility, QC flag Meters Ceiling Height (first layer) Hundreds of feet Ceiling Flag (first layer), QC flag Code Value Cloud Amount (first layer), QC flag Code Value Ceiling Height (second layer) Hundreds of feet Ceiling Flag (second layer), QC flag Code Value Cloud Amount (second layer), QC flag Code Value Ceiling Height (third layer) Hundreds of feet Ceiling Flag (third layer), QC flag Code Value Cloud Amount (third layer), QC flag Code Value The list of code values for the Present Weather is too large to reproduce in this document. Refer to WMO, 1988 for a complete list of Present Weather codes. The code values for the Squall/Gust Indicator are: Code Definition S Squall G Gust The following codes define ceiling information that can not be indicated by the numeric value: Code Definition 0 None 1 Thin 2 Clear below 12,000 feet 3 Estimated 4 Measured 5 Indefinite 6 Balloon 7 Aircraft 8 Measured/Variable 9 Clear below 6,000 feet (AUTOB) 10 Estimated / Variable 11 Indefinite / Variable 12 12-14 reserved 15 Missing The code values for the Cloud Amount Indicator are: Code Definition 0 0 ( or clear) 1 1 okta or less, but not zero or 1/10 or less, but not zero 2 2 oktas or 2/10-3/10 3 3 oktas or 4/10 4 4 oktas or 5/10 5 5 oktas or 6/10 6 6 oktas or 7/10-8/10 7 7 oktas or more, but no 8 oktas or 9/10 or more, but not 10/10 8 8 oktas or 10/10 (or overcast) 9 Sky obscured by fog and/or other meteorological phenomena 10 Sky partially obscured by fog and/or other meteorological phenomena 11 Scattered 12 Broken 13 13-14 Reserved 15 Cloud cover is indiscernible for reasons other than fog or other meteorological phenomena, or observation is not made. 2.1 Data Remarks The Hourly Surface Composite contains only the hourly observations for the NCDC SAO data. The associated NCDC SAO special observations are located in a separate dataset in CODIAC. The following error has been found in the 1992 GCIP Initial Data Set (GIDS-1) Hourly Surface Composite. Only the Illinois Climate Network (ICN) data are affected. These data are available on CODIAC and the GIDS-1 CD-ROM 1. 2.1.1 ICN Calculated Sea Level Pressure Improperly Computed. In the 1992 GIDS-1 Hourly Surface Composite every Illinois Climate Network (ICN) Calculated Sea Level Pressure (CSLP) was improperly computed. The ICN CSLP was computed using GEMPAK algorithms that require Kelvin temperatures. An incorrect conversion factor of 243.5 (instead of 273.16) was used to convert Celsius temperatures to Kelvin. Correction: To correct the data, all ICN CSLP must be recalculated and the quality control flags reset to unchecked. The correct equation can be found in Wallace, 1977. If you require assistance or need more detail concerning this correction please contact jossgcip@joss.ucar.edu. 3.0 Quality Control Processing The Hourly Surface Composite was formed from several sub- datasets (i.e., NCDC SAO, High Plains Climate Network, Illinois Climate Network, NOAA/ERL PROFS Mesonet, Wind Profiler Demonstration Network surface data, and GIDS-1 area extract from STORM-FEST Hourly Surface Composite). Each dataset was quality controlled by comparing the station observations with the MAPS (Mesoscale Analysis and Predictions System) hourly gridded surface analyses. MAPS was developed and the analyses were produced by the Forecast Systems Laboratory of the NOAA Environmental Research Laboratories (Miller and Benjamin, 1992). The same method which was used to quality control the STORM- FEST data (Scully and McGuirk, 1993) was used to quality control the GIDS-1 Hourly Surface Composite. Hourly MAPS gridded values were interpolated to a station's latitude, longitude, elevation and observation time and were then compared to the observed values. Observed values were then flagged "good", "questionable" or "unlikely" based upon the comparison with MAPS. Data were never changed, only flagged. Only the station pressure, sea level pressure, temperature, dew point, wind speed and wind direction were quality controlled this way. The following table shows the allowed variance from MAPS values for each parameter. Parameters Good Questionable Unlikely Station Pressure <3.00mb >=3.00mb >=10.00 mb Sea Level Pressure <3.00mb >=3.00 mb >=10.00 mb Dry Bulb Temp <5.00 C >=5.00 C >=8.00 C Dew Point Temp <5.00 C >=5.00 C >=8.00 C Wind Speed <5.00 m/s >=5.00 m/s >=20.00 m/s Wind Direction <=90.00 >90.00 deg ------------- Squall/gust speed <25.00 m/s >=25.00 m/s >=35.00m/s Precipitation <25.00 mm >=25.00 mm >=75.00 mm Gross limit checks were applied to the squall/gust wind speed data. The wind speed was compared with maximum allowable speeds shown in the table above. Gross limit checks were also used to determine the quality of the precipitation values. Several "questionable" and "unlikely" data values were also manually inspected. After inspection, the quality control flag may have been manually updated to better reflect the likelihood of the actual occurrence of the precipitation value. Data was never modified, only flagged. Negative precipitation was also coded "unlikely". A table of the possible quality control flags and their meanings is listed below. QC Code Description U Unchecked G Good M Normally recorded but missing D Questionable B Unlikely N Not available or Not observed. X Glitch E Estimated C Reported precipitation value exceeds 9999.99 millimeters or was negative. T Trace precipitation amount recorded. General consistency checks were applied to the dry bulb temperature, wind direction, and the relationship between precipitation and cloud amount/cloud cover. If the dew point temperature was greater than the dry bulb temperature both values were coded "questionable". Also, wind direction for observed "calm" winds was given the same QC code as the wind speed. If precipitation was reported, but the cloud amount was "none" or "clear", then both the cloud amount and precipitation values were coded "questionable". Several impossible values were also checked. Negative wind speeds were coded "unlikely". Negative squall/gust wind speeds were coded "unlikely". Wind directions of less than zero or greater than 360 degrees were coded "unlikely". The quality control performed on the GIDS-1 area extract of STORM-FEST Hourly Composite was the same as described above and was retained. 4.0 References For additional station information, refer to the station list provided as a seperate dataset in CODIAC. Scully, K.W. and McGuirk, D.E., 1993: The Use of MAPS Analyses for Quality Control of Surface Observations from STORM-FEST. Preprints, Eight Symposium on Meteorological Observations and Instrumentation, Anaheim, California, 17-22. Miller, P.L. and Benjamin S.G., 1992: A System for the Hourly Assimilation of Surface Observations in Mountainous and Flat Terrain. Mon. Wea. Rev., 120, 2342-2359. Wallace, J.M., P.V. Hobbs, 1977: Atmospheric Science, Academic Press, 467 pp. World Meteorological Organization (WMO), 1988: Manual on Codes Volume I, Part B - Binary Codes. WMO, Geneva, Switzerland. 5.0 Addenda/Corrections Network NCDC, Station FTW The 10:00 hour record for this station on 1992/03/19 contains an invalid actual observation time of 09:90. This particular data record should not be used. Network NCDC, Station RBD The 14:00 hour record for this station on 1992/03/28 contains an invalid actual observation time of 13:70. This particular data record should not be used. Network NCDC, Station FTW The 17:00 hour record for this station on 1992/03/01 contains an invalid actual observation time of 16:99. This particular data record should not be used. Incorrect location for St. Joe; was 39.46/-94.55; should be 39.77/-94.92. Incorrect location for Wood River; was 40.59/-98.37; should be 40.85/-98.60.