BAMEX 2003 Daily Precipitation Composite

1.0 General Description

The BAMEX 2003 Daily Precipitation Composite is one of several precipitation datasets provided for the Bow Echo and Mesoscale Convective Vortex Experiment (BAMEX) 2003. This precipitation composite is composed of data from several sources: Data from these sources were quality controlled and merged to form this precipitation composite. This composite contains data for the BAMEX 2003 domain and time period (20 May 2003 through 6 Jul 2003). The BAMEX 2003 domain is approximately 80W to 104W longitude and 30N to 48N latitude.

Section 2.0 contains a detailed description of the instrumentation, siting, and algorithms used by the source network to collect the data. Section 2.1 contains a detailed description of the format of the composite dataset. Please review Section 2.2 for information on data processing, and for specific issues that affect the data. Section 3.0 below for the quality control processing performed by the University Corporation for Atmospheric Research/Joint Office for Science Support (UCAR/JOSS) on this dataset. Section 4.0 contains references.

2.0 Detailed Data Description

2.0.1 Minnesota State Climatology Office Precipitation Monitoring Program Algorithms

The Minnesota State Climatology Office (
Minnesota Department of Natural Resources) has administered a high spatial density, volunteer-based precipitation monitoring program for approximately 30 years. The program consists of a network of networks, involving a number of water-oriented agencies (federal/state/local). The program involves over 1400 volunteers (Map). The volunteers make daily precipitation measurements, record their measurements on a monthly form, and send the monthly form to their sponsoring agency at month's end. The cooperating agency transfers the data to the State Climatology Office electronically or through the mail. Data falling within the BAMEX area and time of interest are included in this composite. Complete data, and products derived from these data, can be viewed at the Minnesota State Climatology Office web page.

Most precipitation observers utilize four-inch diameter plastic rain gauges with resolution to 0.01 inch. Observation cycles vary widely, but the majority of the volunteers are morning (7:00 or 8:00 AM) observers. As the time of observation is not reported, UCAR/JOSS has arbitrarily reported an observation time of 7AM local (12 UTC) in the records in this composite.

Both incremental and accumulative values are included in this BAMEX 2003 Daily Precipitation Composite. The incremental value reported for any daily observation represents data collected during the previous 24 hours. The accumulative value is a total of precipitation over one or more days when the observer made no measurements. The days on which no measurements were made in an accumulation period are represented with the missing value of -999.99, but flagged with a quality control flag "1" to differentiate them from non-accumulation period missing data. The day which has a data value of the accumulated total of precipitation during all of the accumulation days is flagged with a quality control flag "2". (For a description of the quality control flags, see section 3.0 below.)

Please make attribution to the Minnesota Department of Natural Resources, the Minnesota Climatology Working Group, and the hundreds of volunteers who make such data sets possible.

2.0.2 NCEP/EMC Daily Precipitation Algorithms

The Environmental Modeling Center (EMC), previously the Climate Prediction Center (CPC), is a component of the National Centers for Environmental Prediction (NCEP).

The NCEP/EMC daily precipitation dataset was formed by extracting data with NCEP/EMC element codes of PC and PP that fall on the hour. The NCEP/EMC observations of daily precipitation can include multiple measurements for a given day. When this occurs, the precipitation for that day is set to missing in this UCAR/JOSS BAMEX 2003 Daily Precipitation Composite. The value reported for any daily observation represents data collected during the previous 24 hours. All raw precipitation data can be found on codiac in the GCIP/EOP Surface: Precipitation NCEP/EMC Gage Only Daily Dataset. NCEP/EMC excludes any precipitation value greater than 20 inches from the NCEP/EMC daily precipitation dataset. The information above was gathered from personal correspondence with NCEP/EMC during 1997 and 1998.

Data from cooperators networks such as NCEP/EMC has hundreds of reporting locations which change from incremental to accumulator and back to incremental. Others changed a couple of times and have now settled on accumulators.

Anyone using this data must be aware of the dynamic changes in the network of sites that are processed by HADS and be aware of the potential lag time from when a change is implemted to when NCEP/EMC becomes aware of the change. This is far, far from a stable environment.

To help users in identifying these changes, UCAR/JOSS checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the UCAR/JOSS Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable". For this purpose, UCAR/JOSS defines accumulated precipitation to be six identical, sequential daily precipitation values potentially followed by additional identical values or values that increase over time.

2.0.3 NWS Cooperative Observer (COOP) Daily Precipitation Algorithms

The NWS Cooperative Observer Daily Precipitation dataset was formed by extracting the daily incremental and accumulative precipitation values provided in the National Climatic Data Center (NCDC) TD 3200 dataset. Both incremental and accumulative values are included in this BAMEX 2003 Daily Precipitation Composite. The incremental value reported for any daily observation represents data collected during the previous 24 hours. The accumulative value is a total of precipitation over one or more days when the observer made no measurements. The days on which no measurements were made in an accumulation period are represented with the missing value of -999.99, but flagged with a quality control flag "1" to differentiate them from non-accumulation period missing data. The day which has a data value of the accumulated total of precipitation during all of the accumulation days is flagged with a quality control flag "2". (For a description of the quality control flags, see section 3.0 below.) The TD3200 data contained many sequences of missing days with no indication that they were part of an accumulation period, followed by a day with a value marked as an end accumulation total. Since there is no way to know when the accumulation period began in these instances, missing days in the TD3200 dataset that are not marked as being part of an accumulation period and which are followed by an end accumulation total, are recorded simply as missing days in this composite dataset with a "7" quality control flag. For more information, see NCDC, 1991.

2.0.4 Daily Precipitation Extracted from BAMEX 2003 Hourly Precipitation Composite

Each daily precipitation incremental value in this dataset was formed by summing the 24 hourly precipitation values extracted from each BAMEX 2003 Hourly Precipitation Composite record. The hourly values are summed from hour 0100 through hour 2400 (0000 UTC next day). The value reported for any daily observation represents data collected during the previous 24 hours.

Due to a coding error in the conversion of the LDMSFCMETR data to QCF format, the record for the last hour of the last day a LDMSFCMETR station reported during the BAMEX time of interest was inadvertently deleted. As most LDMSFCMETR stations report for the entire BAMEX TOI, most of these missing records occur on July 6, 2003. A total of 783 records were deleted (one for each LDMSFCMER station). As the LDMSFCMETR daily precipitation was extracted from the hourly surface data, the last daily precipitation value for each LDMSFCMETR station is missing from this BAMEX 2003 Daily Precipitation composite. This bug affects the BAMEX 2003 Surface Meteorological (Hourly) Multi-Network Composite (but does NOT affect the BAMEX 2003 Surface Meteorological (Miscellaneous) Multi-Network Composite), and thus affects the BAMEX 2003 Precipitation (Hourly) Multi-Network Composite and the BAMEX 2003 Precipitation (Daily) Multi-Network Composite as well.

2.1 Detailed Format Description

The Daily Precipitation Composite contains six metadata parameters and four data parameters. The metadata parameters describe the station location and time at which the data were collected. The four data parameters repeat once for each day in the monthly record. Every record has 31 days reported, regardless of the actual number of days in the month. For months with less than 31 days, the extra days are reported as missing (i.e., '-999.99 7 M'). Each 24 hour precipitation value has an associated observation hour. The observation hour is the ending UTC hour for the 24 hour period for which the precipitation value is valid. Data values are valid for the 24 hours preceding the observation hour given for the day. A Quality Flag and a Quality Control (QC) Code are assigned to each data value during quality control processing. For a list of possible Quality Flag and QC Code values see the Quality Control section 3.0. The table below details each parameter.
     
     Parameters               Units
     ----------------------   -------------------------------------
     Date of Observation      UTC 
     Network Identifier       Abbreviation of platform name
     Station Identifier       Network Dependent
     Latitude                 Decimal degrees, South is negative
     Longitude                Decimal degrees, West is negative
     Station Occurrence       Unitless 
     Daily Precip Value       millimeters 
     Quality Flag             See list of Definitions 
     QC Code                  See list of Definitions 
     Observation hour         Beginning UTC hour for 24 hour period
                                   for which value is valid. 
                     

2.2 Data Remarks

Negative precipitation and values too large to be represented in the output format were changed to missing ('-999.99') and were assigned the 'C' QC code.

The NCDC Summary of Day Co-operative data have observation hours that vary by station. Data values are valid for the 24 hours preceding the observation hour given for the day.

The Minnesota State Climatology Office Precipitation Monitoring program data have observation cycles that vary widely. The majority of the volunteers are morning (7:00 or 8:00 AM) observers. As the time of observation is not reported, UCAR/JOSS has arbitrarily reported an observation time of 7AM local (12 UTC) in the records in this composite.

UCAR/JOSS also checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the UCAR/JOSS Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable".

Due to a coding error in the conversion of the LDMSFCMETR data to QCF format, the record for the last hour of the last day a LDMSFCMETR station reported during the BAMEX time of interest was inadvertently deleted. As most LDMSFCMETR stations report for the entire BAMEX TOI, most of these missing records occur on July 6, 2003. A total of 783 records were deleted (one for each LDMSFCMER station). As the LDMSFCMETR daily precipitation was extracted from the hourly surface data, the last daily precipitation value for each LDMSFCMETR station is missing from this BAMEX 2003 Daily Precipitation composite. This bug affects the BAMEX 2003 Surface Meteorological (Hourly) Multi-Network Composite (but does NOT affect the BAMEX 2003 Surface Meteorological (Miscellaneous) Multi-Network Composite), and thus affects the BAMEX 2003 Precipitation (Hourly) Multi-Network Composite and the BAMEX 2003 Precipitation (Daily) Multi-Network Composite as well.

3.0 Quality Control Processing

This precipitation composite is composed of data from several sources Each of these datasets was independently Quality Controlled.

Gross limit checks were used to determine the quality of the data in this composite. Data values less than 101.6 millimeters (4 inches) were flagged as good. Any data value greater than or equal to 101.6 millimeters but less than 127 millimeters (5 inches) was flagged as questionable. Any data value greater than or equal to 127 millimeters was flagged as unlikely. Several questionable and unlikely data values were also manually inspected. After inspection, the quality control code and flag may have been manually updated to better reflect the likelihood of the actual occurrence of the precipitation value based on an isohyet analysis. UCAR/JOSS also checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the UCAR/JOSS Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable". For this purpose, UCAR/JOSS defines accumulated precipitation to be six identical, sequential daily precipitation values potentially followed by additional identical values or values that increase over time. Data were never modified, unless the precipitation value was negative or too large to be represented in the output format. In these cases the data value was flagged with a QC code of 'C' and changed to a missing value (i.e., '-999.99'). Table 1 and 2 list the quality control flags and codes plus their meanings. After the datasets were merged to form the BAMEX 2003 Daily Precipitation Composite, a statistics program was executed to ensure that the quality of the individual datasets had been retained.

     
     Table 1
     
     Quality Control Flag          Description
     ---------------------------   ---------------------------------------------
     0                             No qualifier required 
     1                             Accumulation period
     2                             End of Accumulation period
     3                             Deleted value. Original data was unreadable.
     4                             Trace of precipitation.
     5                             Probable amount as a result of melting frozen
                                   precipitation.
     6                             Suspect amount relative to time or period of
                                   occurrence.
     7                             Missing value
     
     Table 2
     
     QC Code             Description
     -----------------   ------------------------------------------
     U                   Unchecked
     G                   Good
     M                   Normally recorded but missing
     D                   Questionable
     B                   Unlikely
     N                   Not available or Not observed. 
     X                   Glitch
     E                   Estimated
     C                   Reported value exceeds output format field
                         size or was negative precipitation.
     T                   Trace
     I                   Derived parameter can not be computed due
                         to insufficient data.

4.0 References

National Climatic Data Center, 1991: Surface Land Daily Cooperative; Summary of the Day TD 3200. National Climatic Data Center, Asheville, NC, 25 pp.