NAME 2004 Precipitation Daily Multi-Network Composite

1.0 General Description

This Daily Precipitation Composite is one of several precipitation datasets provided for the North American Monsoon Experiment (NAME) 2004. This precipitation composite was formed from several data sources: Data from these sources were merged and gross limit checked to form this precipitation composite. This composite contains data for over 30000 stations in the NAME 2004 Tier 3 domain (5N to 50N latitude and 125W to 75W longitude) and time period (01 June 2004 through 30 September 2004). Duplicate stations between or within networks were not removed, but may be identified via the occurrence value setting in the NAME 2004 Ancillary Station List which can be downloaded separately from the NAME Master Table of Datasets. Data from the Climate Computing Project (CLICOM), NCDC COOP NWS Cooperative Observer Daily Observations, Precipitation NCEP/EMC Gage Only Daily Dataset plus data extracted from the NAME Precipitation Hourly Multi-Network Composite (i.e., Mesonet LDM Surface METAR Precipitation Data, Mesonet FSL MADIS Precipitation Data [NCAR/EOL], NCDC RecRainga Hourly Precipitation, and NCEP/EMC U.S. Gage-only Hourly Precipitation Data [NCAR/EOL]) cover the Tier 3 domain.

Section 2.0 contains a detailed description of the instrumentation, siting, and algorithms used by the source network to collect the data. Section 2.1 contains a detailed description of the format of the composite dataset. See Section 2.2 for information on data processing, and Section 3.0 below for the quality control processing performed by the National Center for Atmospheric Research/Earth Observing Laboratory (NCAR/EOL) on this dataset. Section 4.0 contains references.

2.0 Detailed Data Description

2.0.1 Climate Computing Project (CLICOM) Algorithms

The CLICOM daily precipitation dataset was formed by extracting the precipitation data values (element code=5) from CLICOM surface daily data. These CLICOM data are from the Servicio Meteorologico Nacional de Mexico (SMN)--the Mexican meteorological service--and have not been modified. Observations at each of the stations were made at 08:00 LST. The value reported for the daily observation represents data collected during the previous 24 hours, ending at 08:00. The CLICOM data set is the final station report per paper copy reports (one month entered on one summary page). It is the final official data set.

The data, which are generally key entered from paper copy sheets at state offices of the Mexican Water Commission (Comission Nacional de Agua, CNA), contain observations from stations throughout Mexico. Observers in each state send the data to state offices, which then digitizes the data from the paper copies. The CLICOM surface data update for Mexico for 2004 was provided by Lic Alejandro Gonzales Serratos of the SMN, and Art Douglas of Creighton University. CLICOM is a WMO acronym for a climate data management software system developed by the United Nations for developing countries.

2.0.2 Mexican GASIR-CNA Rain Gage Network Data

The Mexican government maintains a wide network of climate stations across the country. A majority of climate stations only record daily 24 hour precipitation amounts. The nationwide climate network is primarily supported by the Mexican Meteorological Service (Servicio Meteorólogico Nacional,SMN), the Mexican Water Commission (Comisión Nacional de Agua, CNA) or local county and state government agencies (Delegaciones). Readings are standardized to be taken at 8:00 LST each morning and do not switch to and from Daylight Savings Time. All equipment is standard WMO equipment.

The NAME precipitation data set is called the GASIR data set and it is maintained by the CNA. Reporting of data is very poor on the weekends and data is often missing when observers are away from their post. The CNA has a relatively active ongoing effort to recover missing data after the cutoff time for the morning reports. The CNA officially closes the monthly rainfall data set 6 months after the end of the month.

In contrast the SMN daily operational rainfall product (which NCEP ingests) often shows data gaps due to problems with weekend reporting or due to problems with data not being received from particular regional CNA offices. The latter problem arises when a regional office fails to transmit its rainfall reports to the central office in Mexico City. As the SMN daily rainfall report is prepared each morning, late data is frequently not reentered into the data files on a consistent basis. Instead, the SMN chooses to replace its operational rainfall data set with the GASIR data set approximately one to two months after the close of the month.

A final cautionary note needs to be added about missing data that may be erroneously indicated with a "0" rather than missing. It has been found that the SMN operational data set shows a dry bias country wide because many of the regional collecting agencies will place a "0" into a station report that actually was missing. This inconsistency has been a real problem in past years, but the state agencies working with the SMN have tried to differentiate between missing rainfall reports versus a station actually reporting no precipitation. Again, the CNA data set appears not to suffer as much from this problem as does the SMN operational product. The use of the "0" for missing data apparently developed out of a need for a "place holder" in data sets that were used with older computer software programs at the SMN.

2.0.3 NAME Supplemental Raingauge Nework [CNA/IMTA]

The network consists of a low-tech raingauge network installed at 113 high elevation sites and 97 mean and low elevation sites in the state of Sonora. Daily rainfall values are reported for most of the sites. The network utilizes the existing infrastructure (radio and personnel) of the Civil Protection Agency for the state of Sonora as well as of their agencies (police, army, highway) and volunteers. Each station reports via radio or telephone to its municipality, the municipality reports rain from their stations. Under special cases, data is quality controlled and validated. At the headquarters of CP-SON there is one person that contacts all the sites every day. Data are captured on a PC and sent to IMTA, SMN, local government and states users, and used for operational activities at CP-SON. The data are collected at 8:00 LST and do not switch to and from Daylight Savings Time.

2.0.4 NCDC COOP NWS Cooperative Observer Daily Observations

The NWS Cooperative Observer Daily Precipitation dataset was formed by extracting the daily incremental and accumulative precipitation values provided in the National Climatic Data Center (NCDC) TD 3200 dataset (
NCDC, 1991). Both incremental and accumulative values are included in this NAME 2004 Daily Precipitation Composite. The incremental value reported for any daily observation represents data collected during the previous 24 hours. The accumulative value is a total of precipitation over one or more days when the observer made no measurements. The days on which no measurements were made in an accumulation period are represented with the missing value of -999.99, but flagged with a quality control flag "1" to differentiate them from non-accumulation period missing data. The day which has a data value of the accumulated total of precipitation during all of the accumulation days is flagged with a quality control flag "2". (For a description of the quality control flags, see section 3.0 below.)

The TD3200 data contained many sequences of missing days with no indication that they were part of an accumulation period, followed by a day with a value marked as an end accumulation total. Since there is no way to know when the accumulation period began in these instances, missing days in the TD3200 dataset that are not marked as being part of an accumulation period and which are followed by an end accumulation total, are recorded simply as missing days in this composite dataset with a "7" quality control flag. For more information, see NCDC, 1991. The data also contained sequences of non-missing accumulation values that did not terminate with an accumulation code. These values were recorded as questionable in this composite with a "4" quality control flag. There were also instances where there was a sequence of valid accumulation values that did not terminate with the accumulation code. The value following the sequence was marked as the accumulation end in this composite with a question able quality control flag of "4".

2.0.5 Precipitation NCEP/EMC Gage Only Daily Dataset

The Environmental Modeling Center (EMC), previously the Climate Prediction Center (CPC), is a component of the National Centers for Environmental Prediction (NCEP).

The NCEP/EMC daily precipitation dataset was formed by extracting data with NCEP/EMC element codes of PC and PP that fall on the hour. The NCEP/EMC observations of daily precipitation can include multiple measurements for a given day. When this occurs, the precipitation for that day is set to missing in this NCAR/EOL NAME 2004 Daily Precipitation Composite. The value reported for any daily observation represents data collected during the previous 24 hours. All raw precipitation data can be found on codiac in the GCIP/EOP Surface: Precipitation NCEP/EMC Gage Only Daily Dataset. NCEP/EMC excludes any precipitation value greater than 20 inches from the NCEP/EMC daily precipitation dataset. The information above was gathered from personal correspondence with NCEP/EMC during 1997 and 1998.

Data from cooperators networks such as NCEP/EMC has hundreds of reporting locations which change from incremental to accumulator and back to incremental. Others changed a couple of times and have now settled on accumulators.

Anyone using this data must be aware of the dynamic changes in the network of sites that are processed by HADS and be aware of the potential lag time from when a change is implemented to when NCEP/EMC becomes aware of the change. This is far, far from a stable environment.

To help users in identifying these changes, NCAR/EOL checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the NCAR/EOL Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable". For this purpose, NCAR/EOL defines accumulated precipitation to be six identical, sequential daily precipitation values potentially followed by additional identical values or values that increase over time.

The user should pay close attention to the observation hour included with each daily value. The observation hour indicates the ending hour on the day the data were collected. Beware that this composite includes both the daily precipitation extracted from NCEP/EMC hourly stations via the "Daily Extract from NAME Precipitation Hourly Multi-Network Composite" and daily precipitation from "Precipitation NCEP/EMC Gage Only Daily Dataset". The daily NCEP/EMC values from the extraction have a observation hour of 23 while the "Precipitation NCEP/EMC Gage Only Daily Dataset" daily values have a observation hour of 0.

2.0.6 Northwest Mexico NAME Event Raingage Network (NERN) Daily Data

"The NAME Event-Based raingage network (NERN) consists of 87 tipping bucket raingages that have been installed in primarily east-west transects across the Sierra Madre Occidental in Northwest Mexico. 50 gages were installed in 2002 and 31 were installed in 2003 and 6 were installed in 2004. The objectives of the network are to improve the sampling of precipitation at a variety of elevations across the complex terrain of the Sierra Madre as well as to provide event based data which permits the calculation of instantaneous rain rates. Nearly 25% of the new event gages are collocated with existing manually measured daily precipitation gages operated by the cooperative network of the Comisión Nacional del Agua (CNA) of Mexico. Collocation permits cross-validation, quality-control and helps to ensure the safety of the gages. Detailed logs of the instruments, installation and downloading procedures, and quality control information are distributed with each data archive submission and are available from the investigators."

"The tipping bucket raingages used in NERN are the Texas Electronics TR-525USW which are calibrated at 0.01 inches per tip. Each tip of the bucket or "event", triggers an electronic signal which is then stored on an Onset Computer Corporation HOBO Event datalogger. Rainfall events are stored on the datalogger until manually downloaded. As of this writing there are no remote communication devices attached to the raingages. The HOBO datalogger has an 8000 event storage capacity which results in the effective storage of 2032 mm of precipitation. This quantity is substantially higher than the mean annual precipitation in the NAME region. However, it is not guaranteed that this capacity will never be exceeded in a particular year. Thus, the entire network is scheduled to be downloaded at least twice a year to preserve as much data as possible. Technical specifications on both the raingage and the datalogger can be obtained from the vendor's websites at the following URL's:


www.texaselectronics.com
www.onsetcomp.com"

More information about the network can be found in a technical document provided by the authors:

Gochis, D.G., J.-C. Leal, W.J. Shuttleworth, C. Watts, 2005: NAME Surface Raingage Network Station Files. Technical document available from the authors.

DATA COLLECTION AND PROCESSING

"Rainfall data is collected at a minimum of twice per year. As each gage is visited the data is downloaded to a laptop computer and the gage is visually inspected for damage, obstruction or anything that may inhibit the proper function of the gage. A instrument log is kept which details the status and any observed problems with each raingage. "

"The raw rainfall data is stored as rainfall events which correspond to tips of the tipping bucket mechanism. This data is reprocessed into the following equal interval periods:


5 min
15 min
30 min
1 hr
3 hr
12 hr
24 hr"

"Quality control of the tipping bucket gage data is performed by visual screening of the raw data time series and by direct comparison with manually operated cooperative raingages which are collocated at approximately 25% of the gage sites."

"As of Feb. 2005, an automated gross error check has been implemented into the data processing algorithms. This routine flags 05min rainfall values which are in excess of 12mm (=144 mm/hr). These 'extreme' events are then manually examined in order to determine whether or not the event was in fact plausible. Additional details of the gross error check methodology and full details of the quality control procedures and the results of the quality control assessment can be found in a Quality Control Document prepared by the investigators:"

"Gochis, D.G., J.-C. Leal, A. Jimenez, 2005: NAME Tipping Bucket Raingage Network Quality Control Log. Technical Document available from the authors or from NCAR/EOL." (NERN, 2006)

Only daily precipitation values from 86 NERN sites are included in this NAME 2004 Daily Precipitation Composite.

2.0.7 Daily Extract from NAME 2004 Precipitation Hourly Multi-Network Composite

Each daily precipitation incremental value in this dataset was formed by summing the 24 hourly precipitation values extracted from each NAME 2004 Hourly Precipitation Composite record. The hourly values are summed from hour 0100 through hour 2400 (0000 UTC next day). The value reported for any daily observation represents data collected during the previous 24 hours. This dataset is formed by extracting precipitation data from the NAME 2004 Precipitation Hourly Multi-Network Composite .

2.1 Detailed Format Description

The NAME 2004 Precipitation Daily Multi-Network Composite contains six metadata parameters and four data parameters. The metadata parameters describe the station location and time at which the data were collected. The four data parameters repeat once for each day in the monthly record. Every record has 31 days reported, regardless of the actual number of days in the month. For months with less than 31 days, the extra days are reported as missing (i.e., '-999.99 7 M'). Each 24 hour precipitation value has an associated observation hour. The observation hour is the ending UTC hour for the 24 hour period for which the precipitation value is valid. Data values are valid for the 24 hours preceding the observation hour given for the day. A Quality Flag and a Quality Control (QC) Code are assigned to each data value during gross limit checking. For a list of possible Quality Flag and QC Code values see the Quality Control
section 3.0. The table below details each parameter.
     
     Parameters               Units
     ----------------------   ----------------------------------
     Date of Observation      UTC 
     Time of Observation      UTC 
     Network Identifier       Abbreviation of platform name 
     Station Identifier       Network Dependent 
     Latitude                 Decimal degrees, South is negative
     Longitude                Decimal degrees, West is negative 
     Station Occurrence       Unitless 
     Hourly Precip Value      Millimeters 
     Quality Flag             See list of Definitions 
     QC Code                  See list of Definitions 
     Observation hour         Ending UTC hour for 24 hour period
                                  for which value is valid.

2.2 Data Remarks

Negative precipitation and values too large to be represented in the output format were changed to missing ('-999.99') and were assigned the 'C' QC code.

The NCDC Summary of Day Co-operative data have observation hours that vary by station. Data values are valid for the 24 hours preceding the observation hour given for the day.

NCAR/EOL also checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the NCAR/EOL Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable".

The user should pay close attention to the observation hour included with each daily value. The observation hour indicates the ending hour on the day the data were collected. Beware that this composite includes both the daily precipitation extracted from NCEP/EMC hourly stations via the "Daily Extract from NAME Precipitation Hourly Multi-Network Composite" and daily precipitation from "Precipitation NCEP/EMC Gage Only Daily Dataset". The daily NCEP/EMC values from the extraction have a observation hour of 23 while the "Precipitation NCEP/EMC Gage Only Daily Dataset" daily values have a observation hour of 0.

Duplicate stations between or within networks were not removed, but may be identified via the occurrence value setting in the NAME 2004 Ancilliary Station List which can be downloaded separately from the NAME Master Table of Datasets.

3.0 Quality Control Processing

This precipitation composite was formed from several data sources. Some of these datasets may have been Quality Controlled independently by the data sources.
Section 2.0 Detailed Data Description above indicates which data sources quality controlled their data sets and whether or not the QC flags assigned by the data source were carried forward into the NCAR/EOL QC formatted data. If the QC flags assigned by the data source are provided in the final composite data set, they were not overwritten by any gross limit checking done at NCAR/EOL.

Gross limit checks were also applied by NCAR/EOL to determine the quality of the data in the Precipitation Daily Multi-Network Composite [NCAR/EOL]. Data values less than 180 millimeters were flagged as good. Any data value greater than or equal to 180 millimeters but less than 220 millimeters was flagged as questionable. Any data value greater than or equal to 220 millimeters was flagged as unlikely. Several questionable and unlikely data values were also manually inspected. After inspection, the quality control code and flag may have been manually updated to better reflect the likelihood of the actual occurrence of the precipitation value based on an isohyet analysis. NCAR/EOL checks the NCEP/EMC daily precipitation data for possible accumulated values and resets the NCAR/EOL Quality Control flags for all "unchecked", "good", and "trace" accumulated precipitation values in the NCEP/EMC network to "questionable". For this purpose, NCAR/EOL defines accumulated precipitation to be six identical, sequential hourly precipitation values potentially followed by additional identical values or values that increase over time. The data were never changed, only flagged. Tables 1 and 2 contain the quality control flags and codes, respectively. Table 3 lists the applies gross limit values. After the datasets were merged to form the NAME 2004 Precipitation Daily Multi-Network Composite [NCAR/EOL], a statistics program was executed to ensure that the quality of the individual datasets had been retained.

     
     Table 1
     
     Quality Control Flag          Description
     ---------------------------    --------------------------------------------
     0                             No qualifier required 
     1                             Accumulation period
     2                             End of Accumulation period
     3                             Deleted value. Original data was unreadable.
     4                             Trace of precipitation.
     5                             Probable amount as a result of melting frozen
                                   precipitation.
     6                             Suspect amount relative to time or period of
                                   occurrence.
     7                             Missing value
     
     Table 2
     
     QC Code             Description
     -----------------   ------------------------------------------
     U                   Unchecked
     G                   Good
     M                   Normally recorded but missing
     D                   Questionable
     B                   Unlikely
     N                   Not available or Not observed. 
     X                   Glitch
     E                   Estimated
     C                   Reported value exceeds output format field
                         size or was negative precipitation.
     T                   Trace precipitation amount recorded.
     I                   Derived parameter can not be computed due
                         to insufficient data.
     
 Table 3 - Precipitation Gross Limit Values

     Parameter              Good       Questionable     Unlikely
     ---------              ----       ------------     --------
     Daily Precipitation  < 180.0 mm   >= 180.0 mm      >= 220.0 mm

4.0 References

National Climatic Data Center, 1991: Surface Land Daily Cooperative; Summary of the Day TD 3200. National Climatic Data Center, Asheville, NC, 25 pp.

NERN, cited 2006: NAME Event Rain gauge Network [Available online from http://data.eol.ucar.edu/datafile/nph-get/82.141/NAME_sfc_raingages_20050215.txt