Section 2.0 contains a detailed description of the
instrumentation, siting, and algorithms used by the source network to collect
the data. Section 2.1 contains a detailed
description of the format of the composite dataset. Please review
Section 2.2 for information on data processing,
and for specific issues that affect the data.
Section 3.0 below for the quality control processing performed by the
University Corporation for Atmospheric Research/Joint Office for Science Support
(UCAR/JOSS) on this dataset. Section 4.0 contains
references.
2.0 Detailed Data Description
2.0.1 Minnesota State Climatology Office Precipitation Monitoring Program
Algorithms
The Minnesota State Climatology Office (
Minnesota Department of Natural
Resources) has administered a high spatial density, volunteer-based
precipitation monitoring program for approximately 30 years.
The program consists of a network of networks, involving a number of
water-oriented agencies (federal/state/local). The program involves over 1400
volunteers (Map). The
volunteers make daily precipitation measurements, record their measurements on
a monthly form, and send the monthly form to their sponsoring agency at month's
end. The cooperating agency transfers the data to the State Climatology Office
electronically or through the mail. Data falling within the BAMEX area and time
of interest are included in this composite. Complete data, and products
derived from these data, can be viewed at the Minnesota State Climatology Office web page.
Most precipitation observers utilize four-inch diameter plastic rain gauges with resolution to 0.01 inch. Observation cycles vary widely, but the majority of the volunteers are morning (7:00 or 8:00 AM) observers. As the time of observation is not reported, UCAR/JOSS has arbitrarily reported an observation time of 7AM local (12 UTC) in the records in this composite.
Both incremental and accumulative values are included in this BAMEX 2003 Daily Precipitation Composite. The incremental value reported for any daily observation represents data collected during the previous 24 hours. The accumulative value is a total of precipitation over one or more days when the observer made no measurements. The days on which no measurements were made in an accumulation period are represented with the missing value of -999.99, but flagged with a quality control flag "1" to differentiate them from non-accumulation period missing data. The day which has a data value of the accumulated total of precipitation during all of the accumulation days is flagged with a quality control flag "2". (For a description of the quality control flags, see section 3.0 below.)
Please make attribution to the Minnesota Department of Natural
Resources, the Minnesota Climatology Working Group, and the hundreds of
volunteers who make such data sets possible.
The NCEP/EMC daily precipitation dataset was formed by
extracting data with NCEP/EMC element codes of PC and PP that fall on
the hour. The NCEP/EMC observations of daily precipitation can include
multiple measurements for a given day. When this occurs, the precipitation
for that day is set to missing in this UCAR/JOSS BAMEX 2003 Daily
Precipitation Composite. The value reported for any daily observation
represents data collected during the previous 24 hours.
All raw precipitation data can be found on codiac in the
GCIP/EOP Surface: Precipitation NCEP/EMC Gage Only Daily Dataset.
NCEP/EMC excludes any precipitation value greater than 20 inches from the
NCEP/EMC daily precipitation dataset. The information above was
gathered from personal correspondence with NCEP/EMC during 1997 and
1998.
Data from cooperators networks such as NCEP/EMC has hundreds of
reporting locations which change from incremental to accumulator and back
to incremental. Others changed a couple of times and have now
settled on accumulators.
Anyone using this data must be aware of the dynamic changes in the
network of sites that are processed by HADS and be aware of the potential
lag time from when a change is implemted to when NCEP/EMC becomes aware of the
change. This is far, far from a stable environment.
To help users in identifying these changes, UCAR/JOSS checks the
NCEP/EMC daily precipitation data for possible accumulated values and
resets the UCAR/JOSS Quality Control flags for all "unchecked", "good",
and "trace" accumulated precipitation values in the NCEP/EMC network to
"questionable". For this purpose, UCAR/JOSS defines accumulated
precipitation to be six identical, sequential daily precipitation values
potentially followed by additional identical values or values that
increase over time.
The NWS Cooperative Observer Daily Precipitation dataset was formed by
extracting the daily incremental and accumulative precipitation values provided
in the National Climatic Data Center (NCDC) TD 3200 dataset. Both incremental
and accumulative values are included in this BAMEX 2003 Daily Precipitation
Composite. The incremental value reported for any daily observation represents
data collected during the previous 24 hours. The accumulative value is a
total of precipitation over one or more days when the observer made no
measurements. The days on which no measurements
were made in an accumulation period are represented with the missing value of
-999.99, but flagged with a quality control flag "1" to differentiate
them from non-accumulation period missing data. The day which has a data value
of the accumulated total of precipitation during all of the accumulation days
is flagged with a quality control flag "2". (For a description of
the quality control flags, see section 3.0 below.)
The TD3200 data contained many sequences of missing days with no indication
that they were part of an accumulation period, followed by a day with a value
marked as an end accumulation total. Since there is no way to know when the
accumulation period began in these instances, missing days in the TD3200 dataset
that are not marked as being part of an accumulation period and which are
followed by an end accumulation total, are recorded simply as missing days in
this composite dataset with a "7" quality control flag. For more
information, see NCDC, 1991.
Due to a coding error in the conversion of the LDMSFCMETR data to QCF
format, the record for the last hour of the last day a LDMSFCMETR station
reported during the BAMEX time of interest was inadvertently deleted.
As most LDMSFCMETR stations report for the entire BAMEX TOI, most of these
missing records occur on July 6, 2003. A total of 783 records were
deleted (one for each LDMSFCMER station). As the LDMSFCMETR daily
precipitation was extracted from the hourly surface data, the last daily
precipitation value for each LDMSFCMETR station is missing from this
BAMEX 2003 Daily Precipitation composite. This bug affects the
BAMEX 2003 Surface Meteorological (Hourly) Multi-Network Composite (but
does NOT affect the BAMEX 2003 Surface Meteorological (Miscellaneous)
Multi-Network Composite), and thus affects the BAMEX 2003
Precipitation (Hourly) Multi-Network Composite and the BAMEX 2003
Precipitation (Daily) Multi-Network Composite as well.
The NCDC Summary of Day Co-operative data have observation
hours that vary by station. Data values are valid for the 24 hours
preceding the observation hour given for the day.
The Minnesota State Climatology Office Precipitation Monitoring
program data have observation cycles that vary widely. The majority of the
volunteers are morning (7:00 or 8:00 AM) observers. As the time of
observation is not reported, UCAR/JOSS has arbitrarily reported an
observation time of 7AM local (12 UTC) in the records in this composite.
UCAR/JOSS also checks the NCEP/EMC daily precipitation data for
possible accumulated values and resets the UCAR/JOSS Quality Control
flags for all "unchecked", "good", and "trace" accumulated precipitation
values in the NCEP/EMC network to "questionable".
Due to a coding error in the conversion of the LDMSFCMETR data to QCF
format, the record for the last hour of the last day a LDMSFCMETR station
reported during the BAMEX time of interest was inadvertently deleted.
As most LDMSFCMETR stations report for the entire BAMEX TOI, most of these
missing records occur on July 6, 2003. A total of 783 records were
deleted (one for each LDMSFCMER station). As the LDMSFCMETR daily
precipitation was extracted from the hourly surface data, the last daily
precipitation value for each LDMSFCMETR station is missing from this
BAMEX 2003 Daily Precipitation composite. This bug affects the
BAMEX 2003 Surface Meteorological (Hourly) Multi-Network Composite (but
does NOT affect the BAMEX 2003 Surface Meteorological (Miscellaneous)
Multi-Network Composite), and thus affects the BAMEX 2003
Precipitation (Hourly) Multi-Network Composite and the BAMEX 2003
Precipitation (Daily) Multi-Network Composite as well.
Gross limit checks were used to determine the quality of the data in
this composite. Data values less than 101.6 millimeters (4 inches) were
flagged as good. Any data value greater than or equal to 101.6 millimeters
but less than 127 millimeters (5 inches) was flagged as questionable. Any
data value greater than or equal to 127 millimeters was flagged as
unlikely. Several questionable and unlikely data values were also manually
inspected. After inspection, the quality control code and flag may have
been manually updated to better reflect the likelihood of the actual
occurrence of the precipitation value based on an isohyet analysis.
UCAR/JOSS also checks the NCEP/EMC daily precipitation data for
possible accumulated values and resets the UCAR/JOSS Quality Control
flags for all "unchecked", "good", and "trace" accumulated precipitation
values in the NCEP/EMC network to "questionable". For this purpose,
UCAR/JOSS defines accumulated precipitation to be six identical,
sequential daily precipitation values potentially followed by additional
identical values or values that increase over time. Data were never
modified, unless the precipitation value was negative or too large to be
represented in the output format. In these cases the data value was flagged
with a QC code of 'C' and changed to a missing value (i.e., '-999.99').
Table 1 and 2 list the quality control flags and codes plus their meanings.
After the datasets were merged to form the BAMEX 2003 Daily
Precipitation Composite, a statistics program was executed to ensure that
the quality of the individual datasets had been retained.
2.0.2 NCEP/EMC Daily Precipitation Algorithms
The Environmental Modeling Center (EMC), previously the Climate Prediction
Center (CPC), is a component of the National Centers for Environmental
Prediction (NCEP).
2.0.3 NWS Cooperative Observer (COOP) Daily Precipitation Algorithms
2.0.4 Daily Precipitation Extracted from BAMEX 2003 Hourly
Precipitation Composite
Each daily precipitation incremental value in this dataset was
formed by summing the 24 hourly precipitation values extracted from each
BAMEX 2003 Hourly Precipitation Composite record. The hourly values are
summed from hour 0100 through hour 2400 (0000 UTC next day). The
value reported for any daily observation represents data collected during
the previous 24 hours.
2.1 Detailed Format Description
The Daily Precipitation Composite contains six metadata parameters
and four data parameters. The metadata parameters describe the station
location and time at which the data were collected. The four data
parameters repeat once for each day in the monthly record. Every record
has 31 days reported, regardless of the actual number of days in the
month. For months with less than 31 days, the extra days are reported as
missing (i.e., '-999.99 7 M'). Each 24 hour precipitation value has an
associated observation hour. The observation hour is the ending UTC hour
for the 24 hour period for which the precipitation value is valid. Data
values are valid for the 24 hours preceding the observation hour given for
the day. A Quality Flag and a Quality Control (QC) Code are assigned to
each data value during quality control processing. For a list of possible
Quality Flag and QC Code values see the Quality Control
section 3.0. The
table below details each parameter.
Parameters Units
---------------------- -------------------------------------
Date of Observation UTC
Network Identifier Abbreviation of platform name
Station Identifier Network Dependent
Latitude Decimal degrees, South is negative
Longitude Decimal degrees, West is negative
Station Occurrence Unitless
Daily Precip Value millimeters
Quality Flag See list of Definitions
QC Code See list of Definitions
Observation hour Beginning UTC hour for 24 hour period
for which value is valid.
2.2 Data Remarks
Negative precipitation and values too large to be represented in the
output format were changed to missing ('-999.99') and were assigned the
'C' QC code.
3.0 Quality Control Processing
This precipitation composite is composed of data from several sources
Each of these datasets was independently Quality Controlled.
Table 1
Quality Control Flag Description
--------------------------- ---------------------------------------------
0 No qualifier required
1 Accumulation period
2 End of Accumulation period
3 Deleted value. Original data was unreadable.
4 Trace of precipitation.
5 Probable amount as a result of melting frozen
precipitation.
6 Suspect amount relative to time or period of
occurrence.
7 Missing value
Table 2
QC Code Description
----------------- ------------------------------------------
U Unchecked
G Good
M Normally recorded but missing
D Questionable
B Unlikely
N Not available or Not observed.
X Glitch
E Estimated
C Reported value exceeds output format field
size or was negative precipitation.
T Trace
I Derived parameter can not be computed due
to insufficient data.
4.0 References
National Climatic Data Center, 1991: Surface Land Daily Cooperative;
Summary of the Day TD 3200. National Climatic Data Center,
Asheville, NC, 25 pp.