This dataset contains "Special" observation surface data in University Corporation for Atmospheric Research/Joint Office for Science Support (UCAR/JOSS) Quality Control (QC) format for the following networks:
Data for the IHOP 2002 domain (32N to 42N latitude and 90W to 105W longitude) and time period (13 May 2002 through 25 June 2002) are contained within this dataset. This IHOP 2002 Surface Miscellaneous Composite dataset contains data from 456 stations and is in the same format as the data in the IHOP 2002 Mesonet: Hourly Surface Meteorological Composite. No additional QC was performed on this miscellaneous surface composite by the University Corporation for Atmospheric Research/Joint Office for Science Support (UCAR/JOSS).
Section 2.0 contains a detailed description of the
instrumentation, siting, and algorithms used by the source network to collect
the data. Section 2.1 contains a detailed
description of the format of the composite dataset. See
Section 2.2 for information on data processing, and
Section 3.0 below for the quality control processing performed by UCAR/JOSS
on this dataset. Section 4.0 contains references.
All data contained in this composite dataset were collected on stations
manufactured by Artais of Columbus, OH, a division of Vaisala. Each station
records data at 20-minute frequencies. All data are included in this
miscellaneous composite. Precipitation is the accumulated precipitation for the
20 minute period. Hourly AWOS data, including accumulated precipitation for the
hour, can be found in the IHOP 2002 Mesonet: Hourly Surface Meteorological
Composite. Present weather and Sea level pressure are not reported for the
Artais AWOS data.
The following are descriptions of the AWOS station data.
Further details can be found in the AWOS Operations Manual
(USDOT, 1988).
Temperature/Dewpoint
AWOS takes at least 1-min measurements and computes a 5-min
running average. A minimum of four 1-min averages are required to
compute a valid 5-min average. 5-min averages are rounded to the nearest
degree F. AWOS will report the latest valid 5-min average during the
previous 15-min period. If one is not available, the data are reported as
"missing". If the 5-min average dew point is 1 or 2 degrees higher than the
5-min average temperature, then the dew point is reported equal to
temperature. If the 5-min average dew point exceeds the 5-min average
temperature by more than 2 degrees, the dew point is reported as "missing".
Station Pressure and Derived Pressure Elements
AWOS takes 10-sec measurements from at least two independent
pressure sensors and computes respective 1-min averages. A minimum of
5 measurements are required to compute a 1-min average. The 1-min
averages from each sensor are compared to verify that differences do not
exceed 0.04" Hg. If the sensors are in agreement, the lowest pressure
reading from all sensors is reported. If the sensor differences exceed
0.04" Hg, the data are reported as "missing". The reported pressure is
then used by AWOS in the computation of derived parameters (e.g.,
altimeter reading). UCAR/JOSS takes the altimeter reading and converts it back
to station pressure using the algorithms found in the
Smithsonian Meteorological Tables. Sea Level Pressure is then calculated
from station pressure using standard
GEMPAK algorithms
(Unidata, 2003).
Wind
AWOS takes 1-sec measurements of wind speed and direction and
computes a 2-min running average every 5-sec. Wind direction is rounded
to the nearest 10 degrees (magnetic north) and wind speed is rounded to
the nearest knot. Note that AWOS makes no correction to true north in
the archived data. If the 2-min running average is 2 knots or less, the
wind is reported as calm. The gust is computed using the highest 5-sec
average wind speed during the past 10-min period. A gust is computed
only when the 2-min running average exceeds 9 knots and the highest 5-
sec measurement exceeds the 2-min running average by 5 knots (during
the past minute). For Artais AWOS only, the raw data are reported with
respect to magnetic north. No corrections were applied to these Artais
AWOS winds.
Precipitation
AWOS takes 1-min accumulated measurements and computes total
precipitation over the period specified in the AWOS selected archival
interval (usually 5-min or 20-min). The total accumulation counter is
automatically reset each hour.
Present Weather
Present weather is not reported in the AWOS data.
For more information see the
Federal Aviation Administration (FAA) AWOS website (
FAA, 2003)
This dataset contains ASOS, AWOS, and MANUAL stations. Station ID's
are 3 characters long. Some networks use the 4-character ID to refer to
these stations. To obtain the 4-character id, prepend a "K". For example,
station ABR could also be referred to as station KABR.
Visibility values greater than 99999.99 m (~60SM) do not fit in the data format
so they were reset to -999.99 C.
On 10 April 2003, UCAR/JOSS was notified by Daryl Herzmann
(akrherz@iastate.edu), program assistant at the Iowa Environmental Mesonet
(http://mesonet.agron.iastate.edu), of a problem with precipitation
observations from 1-minute Iowa AWOS stations. The 1-minute Iowa AWOS
data are available in their "as is" format from UCAR/JOSS for the
IHOP period at
IHOP_2002 Mesonet: Automated Weather Observing System (AWOS) Iowa 1-min Data
. Note that the following
20 1-minute Iowa AWOS stations had erroneous precipitation values
as described in the email extract below:
ADU, AIO, AWG, CBF, CNC, CSQ, DNS, EOK, FFL, FSW, HNR, ICL, IKV, MPZ, MUT, OXV,
PEA, RDK, SDA, TNU.
Anyway, most (not all) of these stations are outfitted with tipping
buckets. All of you have been receiving the AWOS precipitation amounts
from the raw METARs, but up until today, there has been a fundamental flaw
with the observations. The site resets the hourly precipitation counter
at :55 after the hour, but METARs were only relayed at :05, :25 and :45
after the hour. This would result in 10 minutes of missing
potential rainfall data every hour.
The Iowa Department of Transportation sends us their 1 minute interval
data archive from the network at the end of every month. While looking at
this dataset, I noticed this timing problem last summer. The Des Moines
WFO and the Iowa Mesonet have been working closely with the DOT to get
this problem resolved. And today it has been. The Iowa AWOS sites are
now reporting at :15, :35 and :55 after the hour, which eliminates the
timing issue.
This dataset is a collection of data from many networks. The data are fed to
JOSS over the LDM by the FSL Meteorological Assimilation Data Ingest System
(MADIS). Some of these networks are themselves collections of data from
unrelated sources. Each of these networks/subnetworks can contain different
frequency data and different parameters. This code has been written to process
all of this data at once and convert it to JOSS QCF format.
Conventions
This composite contains "special" and off-hour records.
The hourly parameter value, which can be found in the IHOP 2002 Mesonet:
Surface Meteorological Composite, is the value closest to the hour that falls
at or after :45 minutes and before (but not at) 15 minutes after the hour. All
other records are included in this composite.
MADIS QC flags are not carried forward to JOSS format. However,
the MADIS QC summary value X "Failed QC stage 1", which is a gross limit check,
has been used to mask out extremely bad data. If the data fails this check, it
is set to missing. Precipitation data has not been included in this composite.
The following networks are represented in this dataset.
Latitude/longitude accuracy included in the station list that accompanies
this dataset is a best guess from observation of the raw data:
For more information on these networks, see the
FSL/MADIS
Surface Network Information webpage
(FSL, 2003).
2.0 Detailed Data Description
2.0.1 Automated Weather Observing System (AWOS) Algorithms
The Automated Weather Observing System (AWOS) is a suite of sensors, which
measure, collect and disseminate weather data to help meteorologists, pilots
and flight dispatchers prepare and monitor weather forecasts, plan flight
routes, and provide necessary information for correct takeoffs and landings.
The sensors measure weather parameters such as wind speed and direction,
temperature and dew point, visibility, cloud heights and types, precipitation,
and barometric pressure. The AWOS stations are operated by the Federal Aviation
Administration (FAA) and/or state Department of Transportation offices.
2.0.2 Unidata Local Data Manager (LDM) World Meteorological Organization
(WMO) Surface METAR (LDMSFCMETR) Algorithms
The Unidata Local
Data Manager (LDM) (Unidata, 2002)
distributes World
Meteorological Organization (WMO) Surface data. These data are ingested
by UCAR/JOSS in ASCII WMO
meteorological message structure format
(NOAA/NWS, 2002).
The primary feedset name is "WMO" which includes Public
Product Service (PPS), Domestic Data Service (DDS), High resolution Data
Service (HDS), and International Data Service (IDS) feedtypes. Only
products that match the patterns ^S[AP].* .... ([0-3][0-9])([0-2][0-9])
and ^SX..81 .... ([0-3][0-9])([0-2][0-9]) are collected. In these patterns,
S stands for surface, A for Aviation Routine Reports (FM 15 - METAR), P for
Special aviation weather reports (FM 16 - SPECI), and X for miscellaneous
text records. For information on the METAR format see the
ASOS User's Guide (
NOAA, 2003). Only special data and METAR data
that do not fall on the hour are included in this dataset. Hourly METAR
data are available in the dataset 'IHOP 2002 Mesonet: Hourly Surface
Meteorological Composite'. For 20 minute METAR stations, the observation
that falls between 15 minutes before the hour and the hour, inclusive, is
included in the hourly dataset. If there is no observation in this time
period, then the observation closest to the hour and falling between 1
minute and 15 minutes after the hour is included in the hourly dataset.
All other observations are included in this miscellaneous dataset.
Message from Daryl Herzmann (akrherz@iastate.edu)
* Program Assistant -- Iowa Environmental Mesonet
* http://mesonet.agron.iastate.edu
Sent 10 April 2003:
2.0.3 National Oceanic and Atmospheric Administration (NOAA) Forecast
Systems Laboratory (FSL) Meteorological Assimilation Data Ingest
System (MADIS)
Network accuracy
========== ============
ms:APRSWXN 5
ms:AWX 2
ms:GLDNWS 2
ms:GPSMET 2
ms:IADOT 4
ms:KSDOT 4
ms:MesoWes 2
ms:RAWS 5
ms:UDFCD 3
2.1 Detailed Format Description
The IHOP 2002 Surface Miscellaneous observation data contains
ten metadata parameters and 38 data parameters and flags. The metadata
parameters describe the station location and time at which the data were
collected. The time of observation is reported both in Universal Time
Coordinated (UTC) Nominal and UTC actual time. For the IHOP 2002
Miscellaneous data, the UTC Nominal and UTC actual times are equal to the
time of observation and are always identical. Days begin at UTC 0100 and
end at UTC 0000 the following day. The table below details the data
parameters in each record. Several data parameters have an associated
Quality Control (QC) Flag Code which are assigned by the Joint Office for
Science Support (JOSS). For a list of possible QC Flag values see the
Quality Control section 3.0. No additional QC
was performed by UCAR/JOSS.
Parameters Units
---------- -----
Date of Observation UTC Nominal
Time of Observation UTC Nominal
Date of Observation UTC Actual
Time of Observation UTC Actual
Network Identifier Abbreviation of platform name
Station Identifier Network Dependent
Latitude Decimal degrees, South is negative
Longitude Decimal degrees, West is negative
Station Occurrence Unitless
Station Elevation Meters
Station Pressure, QC flag Hectopascals (mb)
Reported Sea Level Pressure, QC flag Hectopascals (mb)
Computed Sea Level Pressure, QC flag Hectopascals (mb)
Dry Bulb Temperature, QC flag Celsius
Dew Point, QC flag Celsius
Wind Speed, QC flag m/s
Wind Direction, QC flag Degrees
Total Precipitation, QC flag mm
Squall/Gust Indicator Code Value
Squall/Gust Value, QC flag m/s
Present Weather, QC flag Code Value
Visibility, QC flag Meters
Ceiling Height (first layer) Hundreds of feet
Ceiling Flag (first layer), QC flag Code Value
Cloud Amount (first layer), QC flag Code Value
Ceiling Height (second layer) Hundreds of feet
Ceiling Flag (second layer), QC flag Code Value
Cloud Amount (second layer), QC flag Code Value
Ceiling Height (third layer) Hundreds of feet
Ceiling Flag (third layer), QC flag Code Value
Cloud Amount (third layer), QC flag Code Value
The list of code values for the Present Weather is too large to reproduce
in this document. Refer to WMO, 1988 for a
complete list of Present Weather codes.
The code values for the Squall/Gust Indicator are:
Code Definition
---- ----------
blank No Squall or Gust
S Squall
G Gust
The code values for the ceiling flag Indicator are:
Code Definition
---- ----------
0 None
1 Thin
2 Clear below 12,000 feet
3 Estimated
4 Measured
5 Indefinite
6 Balloon
7 Aircraft
8 Measured/Variable
9 Clear below 6,000 feet (AUTOB)
10 Estimated / Variable
11 Indefinite / Variable
12 12-14 reserved
15 Missing
The code values for the Cloud Amount Indicator are:
Code Definition
---- ----------
0 0 ( or clear)
1 1 okta or less, but not zero or 1/10 or less, but not zero
2 2 oktas or 2/10-3/10
3 3 oktas or 4/10
4 4 oktas or 5/10
5 5 oktas or 6/10
6 6 oktas or 7/10-8/10
7 7 oktas or more, but no 8 oktas or 9/10 or more, but not 10/10
8 8 oktas or 10/10 (or overcast)
9 Sky obscured by fog and/or other meteorological phenomena
10 Sky partially obscured by fog and/or other meteorological
phenomena
11 Scattered
12 Broken
13 13-14 Reserved
15 Cloud cover is indiscernible for reasons other than fog or
other meteorological phenomena, or observation is not made.
Table 3.1 - Quality Control Flags
QC Code Description
------- -----------
U Unchecked
G Good
M Normally recorded but missing.
D Questionable
B Unlikely
N Not available or Not observed
X Glitch
E Estimated
C Reported value exceeds output format field size
or was negative precipitation.
T Trace precipitation amount recorded
I Derived parameter can not be computed due to
insufficient data.
FSL, cited 2003: Meteorological Assimilation Data Ingest System (MADIS) Surface Network Information [Available online from http://www-sdd.fsl.noaa.gov/MADIS/network_info.html]
NOAA, National Weather Service, Automated Surface Observing System (ASOS), cited 2003: ASOS User's Guide [Available online from http://www.nws.noaa.gov/asos]
NOAA/NWS, cited 2002: WMO Message structure 2000 Paraphrased Version [Available from http://www.nws.noaa.gov/tg/head.html]
Smithsonian Meteorological Tables, Table No. 65, p.269. Smithsonian Institution Press, Washington, D.C., September, 1949.
Unidata, Cited 2002: Unidata LDM [Available online from http://www.unidata.ucar.edu/packages/ldm/]
Unidata, Cited 2003: Unidata GEMPAK/N-AWIPS [Available online from http://www.unidata.ucar.edu/packages/gempak/].
United States Department of Transportation (USDOT), 1988. AWOS Operations Manual, Federal Aviation Administration.
World Meteorological Organization (WMO), 1988: Manual on Codes Volume I, Part B - Binary Codes. WMO, Geneva, Switzerland.