ARM/GCIP NESOB-97 Shortwave and Longwave Radiation Composite 1.0 General Description This 30 minute Shortwave and Longwave Radiation Composite is one of several surface-layer data sets provided in the Atmospheric Radiation Measurement(ARM)/Global Energy and Water Cycle Experiment (GEWEX) Continental-Scale International Project (GCIP) Near Surface Observation Data Set - 1997 (NESOB-97). This Radiation Composite was formed from five data sources: (1) the 25m ARM/Clouds and Radiation Testbed (CART) Multi-Filter Radiometer (MFR) 20 second data of Upwelling Longwave and Shortwave Irradiance; (2) the Solar and Infrared Radiation Observing System (SIROS) 20 second data of Upwelling and Downwelling Longwave and Shortwave Irradiance, which changed over to (3) the Solar Infrared Station (SIRS) 20 second data of Upwelling and Downwelling Longwave and Shortwave Irradiance; (4) the Broadband Solar Radiation Network (BSRN) 1 minute data of Downwelling Longwave and Shortwave Irradiance; and (5) the High Plains Climate Network (HPCN) hourly Solar Radiation data. This composite was developed by the merging of the 30-minute averaged values of Irradiance as derived by University Corporation for Atmospheric Research/Joint Office for Science Support (UCAR/JOSS) from the 20-second values provided by ARM for its 25m MFR station and its SIROS and SIRS stations; the 30-minute averaged values of Irradiance as derived by UCAR/JOSS from the 1-minute values provided by ARM for its BSRN station; and the hourly values provided by HPCN. UCAR/JOSS computed standard deviations for the averaged irradiance data when at least 15 observations were available within the 30-minute averaging interval. JOSS did not do any other quality control on the data set. This radiation composite contains data within the NESOB 1997 domain (100.5W to 94.5W longitude and 34N to 39N latitude) and time period (01 April 1997 through 31 March 1998). 2.0 Detailed Data Description The NESOB-97 ARM Shortwave and Longwave Radiation Composite is composed of data from four sources which report data at different frequencies. The ARM/CART SIROS provides measurements of upwelling and downwelling hemispherical solar irradiances, direct-beam solar irradiance, diffuse hemispherical solar irradiance from the sky and up- and downwelling hemispherical infrared irradiances via its collection of instruments including the Multifilter Rotating Shadowband Radiometer (MFRSR), pyranometer, pyrheliometer, and pyrgeometer at 19 of the ARM extended facilities (including the central facility). The SIROS stations changed over to SIRS stations during August 1997. The ARM/CART Solar Infrared Radiation Station (SIRS) provides continuous measurements of broadband shortwave (solar) and longwave (atmospheric or infrared) irradiances for downwelling and upwelling components. The ARM Broadband Solar Radiation Network (BSRN) at the central facility provides measurements of infrared (pyrgeometer) and solar (pyranometer) irradiances as well as direct solar irradiance (pyrheliometer) and direct, diffuse, and total solar irradiances at six selected wave bands and a wide band (MFRSR). The ARM 25 m MFR provides measurements of upwelling longwave hemispheric irradiance (pyrgeometer) and upwelling shortwave hemispheric irradiance (pyranometer) from its location 25 m up the 60 m tower at the ARM central facility. The High Plains Climate Network (HPCN) provides measurements of solar radiation in Langley units. 2.0.1 ARM Data Quality Reports A few problems occurred with instrumentation during NESOB-97 which affected the quality of the data. Summarized below are the time period within NESOB-97, as well as the cause and possible resolution. This information was taken from Data Quality Reports provided by ARM. (ARM, 1999) 2.0.1.1 SIROS & SIRS 2.0.1.1.1 Site E4 MFRSR data missing from 05/08/1997 to 06/12/1997 The MFRSR head was damaged at site E4. MFRSR data was not collected from 970508 to about 970612. The head was replaced on 970612 about 1700 GMT and data collection again started. During the time the data was NOT collected, zeros will appear for all MFRSR data fields. 2.0.1.1.2 Site E16 data missing due to hw/communications problems from 04/03/1997 to 04/09/1997 Between 4/3 and 4/9 only 1-5% expected data volume where collected from the SIROS at E16 (Vici). Site maintenance found the system inoperable 4/9 and power-cycled. calibration of pyranometers from 04/01/1997 to 08/21/1997 The pyranometers at this SIROS were installed before ARM/CART calibrations could be performed. Thus, calibration coefficients provided by vendor were used to process the data. Analyses by the National Renewable Energy Laboratory (NREL) and the National Oceanic and Atmospheric Administration, (NOAA) however, indicate that use of vendor calibration data tend to produce estimates of irradiances that are systematically 1-3% smaller, averaging about 1.5%, than if calibration data produced by NREL for ARM/CART were used. This discrepancy is currently being investigated. Hence, users of the pyranometer data from this site should consider increasing the measured irradiances by about 1.5%. This change is within the targeted +/-3% uncertainty estimated for pyranometer calibrations. 2.0.1.1.3 SIROS site E13 (and BSRN site C1) downwelling solar irradiance measurement adjustments from 04/01/1997 to 08/20/97 A comparison of BSRN and SIROS solar radiometers for measuring downwelling irradiances at the SGP central facility was made with field standards and two absolute cavity radiometers brought to the site or a two-week period in April 1997 by Mike Rubes (formerly of the National and Oceanic Atmospheric Administration, Air Resources Laboratory, Surface Radiation Research Branch in Boulder, CO). A description of this effort can currently be found on the World Wide Web at http://www.srrb.noaa.gov/apr97iop/hagsie.html. Analyses of the data from these comparisons have resulted in several observations on the quality of data collected at the BSRN and SIROS platforms since October 13, 1995, which are probably valid to the present time, until these sensors are replaced with more recently calibrated sensors. On Oct. 13, 1995, the two BSRN pyranometers (PSPs) were replaced, so the observations do not apply to the BSRN measurements of global and diffuse irradiation before that date. Another source of information is inspection of the SIROS and BSRN equipment by Joe Michalsky (Atmospheric Science Research Center, State University of New York at Albany) at various times. The results of the findings are summarized as recommendations in the following several paragraphs. Some explanation and further comments are provided in the parenthetical remarks. ANALYSIS WHEN THE DIRECT BEAM WAS NOT OBSCURED BY CLOUDS Downwelling total hemispherical solar (global) irradiance measured by the BSRN unshaded pyranometer is approximately 2% too small (which is within the expected level of uncertainty for unshaded pyranometer measurements) compared to the values computed from the measured direct-beam and diffuse components. Downwelling total hemispherical solar irradiances measured by the SIROS unshaded pyranometer systematically underestimates the global irradiances by excessive amounts, i.e., by greater than 3%. The analyses leading to these recommendations are described in an extended abstract (Michalsky, 1997), and further relevant analyses were conducted by Kato et al. (Kato, 1997). UNSHADED PYRANOMETER PERFORMANCE WHEN THE DIRECT BEAM WAS NOT OBSCURED The above recommendations are based mostly on analyses conducted for cloudless, midday conditions. Because the data reported from the unshaded pyranometer were not corrected for cosine response, slight overestimates of global irradiance from unshaded pyranometers tend to occur in cloudless conditions at solar zenith angles less than 45 deg and slight underestimates tend to occur for zenith angles greater than 55 deg. The maximum deviations occur at extreme solar zenith angles and are about 2%. TRACKER-SHADING PERFORMANCE The data user should note, as has been noted in data release statements, that analyses of the direct, diffuse, and/or direct beam irradiances should be preceded by a check of sensor performances by summing the direct and diffuse components and comparing the result to the directly measured global component. When this is done, problems with solar tracking are usually apparent. For the time period addressed here, the modern tracking- shading assembly used with the SIROS sensors appeared to work well. A modern tracker-shader was installed for the BSRN sensors in January 1997. The tracker was not aligned as well as it could be. Efforts are underway to improve tracker alignment checks and procedures at all SIROS sites and the BSRN site. PARTLY CLOUD CONDITIONS An analysis by Chuck Long (formerly at the Pennsylvania State University and now with the University of Colorado and the National Oceanic and Atmospheric Administration) indicated that data users who are investigating partly cloudy sky conditions will usually find that the BSRN outputs are more reliable for short periods of time, say less than 30 min, than are the SIROS outputs. This tends to occur because the SIROS data are recorded only every 20 s while the BSRN data represent one-minute averages computed on the basis of sampling once per second. Under partly cloudy conditions, sampling only once every 20 s tends to provide inadequate statistical representation of downwelling irradiances. ESTIMATES FOR CLOUDY CONDITIONS The component sum technique is not applicable for overcast conditions. An average of data from the SIROS shaded pyranometer, the shaded BSRN sensor, and the shaded BSRN sensor multiplied by 1.02 might be the best estimate of global irradiance for cloudy conditions. However, a rigorous analysis on the results of this procedure has not been carried out, so the data user should approach this technique with caution. SOME ADDITIONAL INFORMATION The excessively large deviations noted above for the pyranometers result in part from a mixture of different sources of calibration procedures. The following table lists the sources of calibration: Sensor Coefficient used Calibration Installation to process data date date BSRN PSP DD Eppley June 1995 Oct. 13, 1995 SIROS PSP DD Eppley June 1995 July 25, 1995 DD = downwelling diffuse PSP = precision spectral pyranometer Eppley = denotes calibrations in an integrating sphere by the manufacturer, Eppley Laboratory, Inc. A concern at this time is the insufficiently frequent recalibrations of sensors in operation at the SGP site. The pyranometers typically should be recalibrated at least once every 12 months. Change out with freshly calibrated pyranometers and pyrheliometers at the SGP site will begin in 1997, with the goal of routinely replacing every pyranometer and pyrheliometer with freshly calibrated sensors once every year. Data users can inspect metrics provided on the World Wide Web by the SGP site scientist team on data quality at the following address: http://www.res.sgp.arm.gov/sst/dq_monitor/DISPLAYS.html Other observations/measurements impacted by this problem: Any derived estimates of downwelling solar radiation components using data from central facility SIROS or BSRN sensors (for downwelling solar radiation) for the time period indicated. Suggested Corrections of the Problem: (e.g. change calibration factor and recompute, flag data with this comment, etc.) Use of these recommendations by data users. Ideally, the component sum technique would be applied in a value-added product (VAP) implemented at the Experiment Center, but this has not been done yet. In the meantime, users of recent data can inspect plots of component sum technique on the World Wide Web site noted above. 2.0.1.1.4 SIROS site E2 solar tracker inoperative 04/12/1997 to 04/17/1997 On about 97/04/12 the solar tracker at site E2 stopped working. The solar tracker exhibits its broken state in a particular striking fashion on 97/04/14 - an almost cloud-free day. On this day, the NIP reads zero and the diffuse and direct SW radiation are about the same. The fact that these two data steams are not exactly the same indicates a calibration problem - probably with the diffuse instrument. This poor calibration is not typical of other ARM sites! On about 97/04/17 (1400 GMT) the solar tracker was fixed. (Note: we cannot pinpoint the exact time or date when the solar tracker went out and the date quoted above {97/04/12} should be regarded as approximate) Reprocess: wrong calibration 11/06/1997 to 11/26/1997 Mike Splitt found the following in the Site Ops OMIS data base from E2 on 11/26/97 : Found that the wrong calibration sticker was placed on the nip and the wrong Cal factor had been entered in the program. Also while editing the program I found that I had the wrong Cal factor for the DS. I inserted the correct Cal factors for the DS and Nip, complied and down loaded the new program into SIRS_02 at 1720GMT on 11/26/97 This entry was made by Craig Webb. The problem was reported in P971121.1 2.0.1.1.5 SIROS sites E1, E4, E6, E9, E11 NIP affected by precipitation 06/23/1997 to 06/24/1997 Comparison of the dowelling hemispheric solar broadband flux estimated from the unshaded PSP (down_short_hemisp) and that derived from the combination of the NIP (short_direct_normal) and shaded PSP (down_short_diffuse_hemisp) showed significant differences (greater than 70 W/m^2) at times on June 23, 1997. These differences appear to be due to questionable data from the NIP (short_direct_normal) that may have been caused by rainfall on that day. The NIP values recorded values as low as -14 W/m^2 in proximity to when the SMOS recorded precipitation. After the first precipitation event, the NIP appeared to be recording lower values during sunnier periods that may be due to collected precipitation in the NIP affecting the amount of solar radiation reaching the sensor. 2.0.1.1.6 SIRS sites E8, E11, E13, E15 SIRS NIPs affected by condensation 09/26/1997 to 09/26/1997 Comparison of data from the unshaded PSP to that derived from the "direct + diffuse" (NIP + shaded PSP) indicated low values of the "direct+diffuse" combination starting at sunrise and lasting for about an hour at E8, E11, E13 and E15. Further examination revealed that the NIPs were reading low and were likely affected by condensation during this brief period. Weather conditions were foggy over the area early that morning, and fog quickly dissipated after sunrise. 2.0.1.1.7 SIRS sites E20 MFRSR/SIROS E20 - ingest problems 12/24/1997 to 01/23/1998 From about 971224 to about 980123 the MFRSR ingest and the SIROS ingest were both running at site E20. Needless to say, this caused some data collection mix ups. Sometimes only the MFRSR data were collected while at other times both the MFRSR and broadband data were collected. These data are probably OK, but one should beware of the potential for problems and gaps in the MFRSR and SIROS data streams. The start and end dates indicated above are approximate. 2.0.1.2 BSRN 2.0.1.2.1 BSRN site C1 (and SIROS site E13) downwelling solar irradiance measurement adjustments from 04/01/1997 to 09/30/97 See section 2.0.1.1.3 Reference Broadband Shortwave Data at SGP Central Cluster 09/17/1997 to 10/03/1997 This is a recommendation for the best available broadband shortwave data from the SGP Central Cluster (Lamont, OK) during the Combined Fall IOP, 15 Sept - 5 Oct 1997. Data available from the SGP Radiometer Calibration Facility (RCF) has lower measurement uncertainties than similar measurements from the C1, E-13, and BSRN/BRS platforms. The RCF data were collected using the same Broadband Outdoor Radiometer CALibration (BORCAL) system used for routine calibration of pyranometers and pyrheliometers at the RCF. The data are available for 30-second interval. Direct normal irradiance measurements from a windowed RCF Absolute Cavity Radiometer during the IOP is considered more accurate (+/- 0.5%) than the Normal Incidence Pyrheliometers (NIP) at the Central Cluster (+/- 2%). The Automated Hickey-Frieden cavity radiometer is electrically self-calibrating and provides reference standard data suitable for the calibration of the NIPs used at all the CART sites. Diffuse horizontal irradiance is available as the average of two Eppley Precision Spectral Pyranometers and is considered slightly more accurate than the downwelling diffuse (DD) data from the Central Cluster instruments. The RCF data will have periodic gaps during the electrical calibration intervals (about 6-10 minutes, 4 or 5 times per day). The reference global horizontal (or Downwelling Shortwave - DS) has been computed from the measured direct normal and diffuse components: DS = NIP x Cos(Z) + DD, where, Z = solar zenith angle. All RCF data collected during the IOP are presently on the R1-Research System at the SGP. Tom Stoffel is working with Robin Perez to ensure these data are available to the widest possible audience of users. Additional corrections to the diffuse data may be possible after researching PSP nighttime offsets. Data from C1, E-13, and BSRN/BRS platforms during the SW-IOP '97 are still being investigated. 2.0.1.3 25m MFR 2.0.1.3.1 Site E13 Poor data quality from 08/22/1997 to 09/30/1997 In the middle of August, the logger board that collects data from the 25m MFR was upgraded. This upgrade did not work out as planned, and the data quality from the MFR - as well as the downward looking broadband instruments - are not good. The poor data is easily identifiable - you'll know it when you see it!! Please note that the exact date when these problems started is not known with precision; it could be several days prior to or after 970822. 2.0.2 ARM Algorithms The algorithms used by ARM to produce the radiation data are not currently available. 2.0.3 HPCN Algorithms The algorithms used by HPCN to produce the radiation data are not currently available. The Langley values from the HPCN data were converted to W/m2 by multiplying by 10. 2.1 Detailed Format Description The ARM/GCIP NESOB-97 Shortwave and Longwave Radiation Composite Dataset contains eight metadata parameters and twelve data parameters and flags. The metadata parameters describe the date, time, network, station and location at which the data were collected. Data values are valid for the 30 minutes preceeding the time of observation. All times are UTC. Table 1 below details the data parameters. The data parameters have an associated QC flag but UCAR/JOSS does not Quality Control the data at the present time. The Quality Control flag is set to "U" for "Unchecked", unless the datum is missing, in which case the flag is set to "M". The table below details each parameter in the composite data set. Parameter Unit --------- ---- Date of Observation UTC Time of Observation UTC Network Identifier Abbreviation of platform name Station Identifier Network Dependent Latitude Decimal degrees, South is negative Longitude Decimal degrees, West is negative Station Occurrence Unitless Station Elevation Meters Upwelling Longwave Irradiance W/m2 QC flag U or M Standard Deviation W/m2 Downwelling Longwave Irradiance W/m2 QC flag U or M Standard Deviation W/m2 Upwelling Shortwave Irradiance W/m2 QC flag U or M Standard Deviation W/m2 Downwelling Shortwave Irradiance W/m2 QC flag U or M Standard Deviation W/m2 2.2 Data Remarks None. 3.0 Quality Control Processing No QC was performed on this dataset by the University Corporation for Atmospheric Research/Joint Office for Science Support (UCAR/JOSS). 4.0 References ARM, 1999: Data quality reports for SIROS, SIRS, BSRN, and 25m MFR. ARM, 1999: Multi-Filter Rotating Shadowband Radiometer (MFRSR), http://www.arm.gov/docs/instruments/static/mfrsr.html (data for filter characteristics reflect filters installed since NESOB-97). Kato, S., Ackerman, T. P., Clothiaux, E. E., Mather, J. H., Mace, G. G., Wesely, M. L., Murcray, F., and Michalsky, J., 1997: Uncertainties in modeled and measured clear-sky surface shortwave irradiances, J. Geophys. Res., Vol. 102, pp. 25881. Michalsky, J., Rubes, M., Stoffel, T., Wesely, M., Splitt, M., and DeLuisi, J., 1997: Optimal measurements of surface shortwave irradiance using current instrumentation--the ARM experience. Preprints, Ninth Conference on Atmospheric Radiation, Boston, MA, Amer. Meteor. Soc., J5-J9. Solar Infrared Radiation Station (SIRS), http://www.arm.gov/docs/instruments/static/sirs.html; February 16,2001