Table of Contents
1. Introduction
2. Atlas Data Management Policy
3. ATLAS/Itex Investigator Needs
4. ATLAS/ITEX Data Management Strategy/Description of Support
4.3 Data Documentation Guidelines
4.4 Procedures for Submission of Datasets to JOSS ATLAS Archive
4.5 Collection and Availability of Supplemental Operational Datasets to ATLAS
4.6 Special Processing of Data to Produce Integrated or Composite Datasets
5. Components of the Modeling Research Dataset
6. Training on JOSS Data Management System
7. Data Archival and Long-term Access
8. ATLAS/ITEX Data Questionnaire
Table 1. Data Needs: In General and from Other Projects
Table 2. Data Exchange Between the ATLAS Projects
Table 3. Operational and Research Supporting Data for ATLAS/ITEX
Table 4. Modeling Projects
Table 5. Data Needs of Models from Field and Remote Sensing Measurements
Table 6. Model Products
Table 7. Respondents to Data Questionnaire
Appendix A Acronym List
Appendix B Complete Data Questionnaire
JAM 05/09/00
ATLAS/ITEX Data Management Plan
UCAR Joint Office for Science
Support
1.
Introduction and
Background
The
Land-Atmosphere-Ice Interactions (LAII) Program is part of the Arctic System
Science (ARCSS) Program of the National Science Foundation. LAII has overall goals of understanding how
the feedback processes within Arctic system affects global climate change,
including changes in the hydrology, biochemical, plants animals, and
ecosystems; as well as how these changes affect and are affected by human
activities. The LAII Arctic Transitions in the Land-Atmosphere System (ATLAS)
program is designed as an interdisciplinary multi-year project with many
investigators and varied instrumentation to address the aforementioned
objectives in Alaska and eventually in other parts of the Arctic. Information on the ATLAS Project is located
at: the LAII web page: http://www.laii.uaf.edu/
The International
Tundra Experiment (ITEX) is an Arctic ecology field program as a part of the
Man-And-the-Biosphere, Northern Science Network (MAB-NSN) initiative. The purpose of ITEX is to monitor the
performance of plant species and communities on a circumpolar basis in
undisturbed habitats with and without environmental manipulations. ITEX field
stations make measurements following standardized procedures outlined in the
ITEX Manual available from the ITEX web page at: www.systbot.gu.se/research/ITEX/ITEX.html The complementary nature of the ATLAS and
ITEX measurements suggest cooperation in data management activities would be
beneficial to both programs.
An integrated data
management plan is important to assure that a complete database is provided for
easy access to all ATLAS and ITEX project investigators and the science
community in general. The University
Corporation for Atmospheric Research (UCAR) Joint Office for Science Support
(JOSS) has primary responsibility for the development and implementation of a
comprehensive data management strategy for these projects. General guidelines will be given to JOSS by
the LAII Science Steering Committee (SSC) and in cooperation with the ATLAS
Science Management Office (SMO), the investigators and other projects as
appropriate.
The ATLAS
Implementation Plan comments specifically on the need to integrate and
synthesize the ongoing research and results from this project with other LAII
activities (SIMS, RAISE, etc.), OAII research initiatives and international
permafrost monitoring activities (CALM, PACE,
GCOS, etc.). Coordination and
data exchange between ITEX and ATLAS will be further encouraged as a unified
data management plan is implemented.
The data management strategy discussed in this document will streamline
the data integration process with other international programs as it becomes
important.
JOSS will work with
ATLAS and ITEX participants to provide data management support discussed in the
subsequent sections of this document.
The details contained in this plan will serve as a guide throughout the
life of ATLAS and increase the ease of data exchange among the
participants. It is anticipated that
there will be modifications to the ATLAS and ITEX experiment design and that
refinements to the data management requirements will be required. The data management activities proposed
represent a cohesive plan given current information. Modifications to the requirements and implementation strategy
will be handled as they occur.
2.
ATLAS/ITEX Data
Management Policy
The following data
protocols have been proposed to help guide the data management strategies
discussed elsewhere in this document.
This detailed protocol is meant as a guide for all ATLAS and ITEX
investigators with regard to the processing, quality control, dissemination and
sharing of data with other ATLAS participants and LAII cooperating projects, as
well as with the ARCSS investigators and the general scientific community. The objective of the protocol is to
facilitate the timely submission and exchange of high quality data in an open
fashion to all interested investigators, and to assure the long-term archival
of the ATLAS and ITEX data sets.
         1. All ATLAS and ITEX investigators are encouraged to share
their data with other members of the project science teams without restriction.
         2. All ATLAS/ITEX Science Team (AIST) members
will submit their data within one year of the end of each field program phase
to JOSS or other appropriate archive center to facilitate longer term archival
and distribution.
         3. All ATLAS and ITEX datasets will be
considered accessible in the larger LAII/ARCSS domain one year following
conclusion of the field measurements, now scheduled for autum 2001.
         4. Datasets submitted to the archive centers
must follow documentation guidelines set forth both in the ATLAS/ITEX Data
Management Plan and by ARCSS in the ARCSS Data Protocol (http://arcss.colorado.edu/) so that other users understand the characteristics and
attributes of the data and the chosen format. This includes information
concerning the quality of the data and may require that suitable caveats
regarding the data be included in any publication using that data.
         5. AIST members publishing ATLAS results must
always provide appropriate acknowledgement and citation of those who collected
and provided the data, regardless of contribution to the publication.
         6. AIST members are strongly encouraged to
submit any datasets resulting from collaborative ATLAS investigations to the
ATLAS archive at JOSS. This includes collaborative work outside the AIST.
         7. ATLAS will use a distributed archive
strategy. Investigators may keep their own data collected as part of ATLAS so
long as they establish appropriate links to this data through the ATLAS archive
at JOSS. In the longer term, it is
probably best to forward the data to JOSS and/or NSIDC for archive. All data will eventually reside at
NSIDC. An appropriate link provides
access through a national data center, or other similar long-term archive
center.
3. ATLAS/ITEX
Investigator Needs
The ATLAS
investigators have provided important details about their research and
operational data needs in the ATLAS Implementation Plan. Table 1 from that document shows the
diversity of data that are needed to answer the critical project scientific
questions. The data management strategy
described here must help assure that the investigators can have access to and
share these data. It is also planned
that the exchange of data can be streamlined by consciencitous use of the
format and documentation guidelines outlined below.
1.
Sturm - Winter met. data (1-2hr windspeed, direction, temperature (WO),
vegetation maps (SW), aerial photos/imagery for spring snowmelt (WO), fire
history (Seward)
2.
Oechel - Soil moisture and/or water table, radiation, temperature,
imagery, vegetation type, soil organic matter content and composition, and
precipitation along transect and region for extrapolation fire history
(Seward), snow depth, cover, permeability (MS), biomass (SW)
3.
Walker - Temperature and precipitation, snow cover, soils, permafrost,
fire history (Seward), corrected AVHRR time series imagery (WO)
4. Schimel - Snow depth, cover, permeability (MS), temperature (WO), precipitation, flux
measurement (WO)
5.
Hinkel - Temperature, active layer depth from CALM sites
6.
Hobbie - Flux data (WO, TC), annual/summer (monthly) solar radiation,
temperature (WO, TC), soil organic matter fractions (labile, lignin, cellulose
(CP)), peak season leaf and soil N (TC,WO), biomass (SW), LAI (SW), winter flux
(JS, WO)
7.
Chapin/Lynch - Winter met. (WO, ARM), vegetation-functional
types-composition-biomass (SW), vegetation map (SW), snow depth over time (MS),
progression of melt (MS), thermal conductivity (MS), time course of snow depth
(MS), precipitation, treeline (AL), winter met. data and regional pattern of
soil moisture (LH)
8.
Romanovsky - Met. data (WO, ARM), vegetation
map (SW), snow depth and thermal conductivity (MS), soil types and properties
(CP), future climate and vegetation dynamics
9.
Lloyd
- Vegetation map (SW), soil moisture and met. data (LH)
10.
Hinzman
- Vegetation map (SW), met. data, soil temperature and moisture (TC)
Abbreviations for projects: WO=Walt Oechel; TC=Terry Chapin; SW=Skip Walker; JS=Josh Schimel; MS=Matthew Sturm; CP=Chien-Lu Ping; AL=Andrea Lloyd; LH=Larry Hinzman
Table 2 presents the
anticipated data exchange among the various projects of ATLAS. The descriptions of the implementation
strategies in section 4 of this document provide the framework for the
mechanism to make this exchange as efficient as possible. This table clearly shows that data exchange
among the participants is crucial to answering the major scientific
questions. The data management strategy
must foster and streamline this process.
Table 2: Data Exchange Between the ATLAS Projects
Provider Receiver |
Sturm |
Oechel |
Walker |
Welker
Schimel Ping |
Hinkel |
Hobbie |
Chapin
Lynch |
Romanovksy |
Hinzman
Kane |
Lloyd
Fastie |
Sturm |
|
Met. Air photo |
Veg
map, LAI |
|
|
Shrub
mass |
Model
met. |
Soil
T- |
Met. |
Tree-line
dynam-ics |
Oechel |
Snow
depth, cover, perm. |
|
Veg
map, LAI, BM |
Soil
comp. and flux |
Soil
H20 |
TEM
fluxes |
Model
met., soil H2O |
Soil
T- and H20, unfrozen water, active layer thickn. |
Soil
H20 |
|
Walker |
Snow
cover |
Met., NDVI, air photos, AVHRR images |
|
Soils |
Thaw |
Bio-mass |
Soil
T-, met., LAI, biomass |
Soil
T-, met. and soils for the Siberian transect |
Soil
H20 |
Tree-line
dynam-ics |
Schimel
Welker Ping |
Snow
depth, cover |
Flux |
Veg
map |
|
Soil
T- and H2O |
SOM,
veg. |
Summer
flux |
Soil
T- and H20, unfrozen water |
Met. |
Tree-line
dynam-ics |
Hinkel |
Snow
depth, cover |
|
Veg
map, thaw |
Soils |
|
|
Soil
T- and met. |
Soil
and permafrost T-, active layer |
Met.,
soil thaw |
|
Hobbie |
Time
of snow melt |
Flux,
met. |
Veg
map, LAI, BM |
Soils,
flux, soil organic matter |
Soil
T- and H2O |
|
Flux,
met., biomass, LAI, N |
Veg
and soil maps, soil T- and H20 |
Soil
H20 |
Tree-line
dynam-ics |
Chapin
and Lynch |
Snow
depth, snow cover |
Flux,
met. |
Veg
map, LAI, BM |
Soils,
flux |
|
TEM
fluxes |
|
Veg
and soil maps, soil T- and H20 |
Soil,
H20 |
Tree-line
dynam-ics |
Romanovsky |
|
|
|
|
|
TEM |
ARCSyM |
|
Met. |
|
Hinzman
and Kane |
Snow
map |
Met. |
Veg
map |
|
Soil
T- |
|
Met.,
soil T- and H20 |
Soil
T- |
|
Tree-line
dynam-ics |
Lloyd
and Fastie |
|
|
Veg |
|
|
|
Met. |
|
Soil
H20 & met. |
|
4.
ATLAS/ITEX Data
Management Strategy and Description of Support
Central to the JOSS
data management is an on-line, interactive catalog, archival and distribution
system (CODIAC). It offers scientists a
means to identify datasets of interest, the facilities to view selected data
and associated metadata, and the ability to obtain data automatically from
geographically dispersed data centers via Internet file transfer (FTP) or
separate media (tapes, CD-ROM, disks, etc.).
This system will be used during and following the ATLAS field campaigns
as the primary access point for preliminary and final research and operational
(if needed) data as they become available.
Links via the World Wide Web (WWW) will also be made to other data
centers that may hold any data or information of interest to the ATLAS
investigators. The JOSS data management
system may be accessed via the WWW at:
http://www.joss.ucar.edu/atlas/
4.1
Investigator
Requirements
The success of the
ATLAS/ITEX data management strategy is dependent on the participants'
willingness to work together to share data and documentation. JOSS will facilitate these activities using
our experience and data management software tools.
The first step in
organizing ATLAS/ITEX data management support is to understand what data are or
planned on being collected by the various components of the program. JOSS has worked with the SMO to develop and
disseminate a questionnaire to participants that solicits this
information. The results of the
questionnaire to date are shown in summary form at: http://www.joss.ucar.edu/atlas/.
The second step in
our data management strategy requires the participants to take primary
responsibility for acquisition and provision of field data. This includes in-field record keeping,
backing up field data, data documentation and initial processing and quality
control of raw data. The consideration
of the specific dataset or subset to be provided to JOSS and the archive is
discussed in section 4.1.1.
The third step
includes the submission of preliminary and final datasets to JOSS for inclusion
in the ATLAS/ITEX data set. Sections
4.2 and 4.3 provide details of format and documentation guidelines for all
archived data. The submission process
and related information are described in Section 4.4.
4.1.1 What data should be submitted to the archive
Because of the
diverse nature of the datasets being collected by the investigators, the
question arises: what data should be submitted to the archive? Affirmative answers to any of the following
questions should be interpreted as justification to provide the data to JOSS.
-
Will the data be
shared with other ATLAS/ITEX or LAII investigators?
-
Will the data have
broader interest in the community in the future?
-
Is the data required
to meet your NSF grant objectives?
In addition to these
questions it is also important that you decide if others could realistically
utilize the data in a particular form. For example, original field notes, logs
and data logger output, though important, will likely not be so useful to
others. If original or "raw"
data will require further processing to make it more useable by the PI as well as
others, then provision of the processed data set to JOSS makes more sense.
4.2
Data format
guidelines
An inherent
flexibility of the JOSS data management system permits data in all different
digital formats to be submitted, stored and retrieved from CODIAC. JOSS is prepared to work with the
participants to bring their data to the archive and make sure it is presented,
with proper documentation, for exchange with other project participants. In
anticipation of receiving many data sets from the field sites in ASCII format
we are providing guidelines below that will aid in the submission, integration
and retrieval of these data. JOSS will
work with any participants submitting other formats including NetCDF, AREA, HDF
and GRIB to assure access and retrieval capabilities within CODIAC.
The following ASCII
data format guidelines are intended to help standardize the information
provided with any data archived for the project. These guidelines are based on JOSS experience in handling
thousands of different data files of differing formats. Specific recommendations are provided for
naming a data file as well as information and layout of the header records and
data records contained in each file.
This information is important when data are shared with other project
participants to minimize confusion and aid in the analysis. An example of the layout of an ASCII file
using the guidelines is provided below.
Keep in mind that it is not mandatory that the data be received in this
format. However, if the project participants are willing to implement the data
format guidelines described below, there are some improved capabilities for
integration, extraction, compositing and display via CODIAC that are
available. These capabilities are
described in Section 4.6.
NAMING CONVENTION
All data files should be uniquely named. For example, it is very helpful if date can
be included in any image file name so that the file can be easily time
registered. If the image summarizes a
period of time, the start and end time is very desirable.Also include an
extension indicating the type of file:
i.e. .gif = GIF
image format file
.jpg
= jpg image format file
.txt
= Text or ASCII format file
.cdf
= NetCDF format file
.tar
= archival format
if compressed, the
file name should have an additional extension indicating the type of
compression (i.e. .gz, .z, etc.)
B) For Text (ASCII)
files, the records should consist of both header records and data records. The
header records at a minimum should consist of:
ASCII DATA FORMAT
HEADER RECORDS SPECIFICATIONS
Standard header
records should precede the data records within the file itself. The header
records should contain the following information:
PI/DATA CONTACT = Text [PI and data contact name(s) and
affiliation(s)]
DATA COVERAGE = Start/Stop
time of continuous data or sampling interval (Use data/time format described
below)
PLATFORM/SITE = Text
[e.g. C130, CITATION, IVOTUK, TOOLIK, etc.]
INSTRUMENT =
Text [instrument name]
COORDINATES = Fixed
site coordinates (decimal degrees) or "mobile" (Text)
DATA VERSION = Alphanumeric
[unique ID (i.e. revision date, PRELIMNARY or FINAL]
REMARKS = Text
[PI remarks that aid in understanding data file structure and contents. Items
such as file type, how missing and/or bad data are denoted or any other information
helpful to users of this data]
Missing Value
indicator - Text or integer
[value used for data for missing information] (e.g. ‑99 or 999.99, etc)
Below Measurement
Threshold - Text or Integer
[Value used to signify reading below instrument detection threshold] (e.g.
<0.00005)
Above Measurement
Threshold - Text or Integer
[Value used to signify reading at or above instrument saturation]
**NOTE** This type of
header information cannot be contained within GIF and Postscript files. They
will need to be submitted with attached files or separate documentation
containing this information.
DATA RECORDS
SPECIFICATIONS:
1)
First data record consists of parameters identifying each
column. Multiple parameter names should
be shown as one word (e.g. thaw_depth or Leaf_area_index)
2)
Second data record consists of respective parameter
units. Multiple unit names should be
shown as one word (e.g. crystals_per_liter)
3)
Third data record begins actual data and consists of a
date/time column followed by position coordinates (if mobile) and subsequent
observations at that time and position.
!
Date/time must be in UTC and recommended format is:
YYYYMMDDHHmmss.ss
where: YYYY= Year
MM
= Month (00-12)
DD
= Day (01-31)
HH
= Hour (00-23)
mm
= Minute (00-59)
ss
= Second (00-59)
.ss
= Decimal Second (unlimited resolution based on sampling frequency)
!
For every mobile platform data set, position coordinates
(i.e. latitude, longitude) should be expressed in decimal degrees for each data
point. Altitude or elevation is given
in appropriate metric units. This may be done by: (a) providing date/time of
collection with position coordinates in each data record; or (b) providing
date/time of collection for each data point in the submitted file, with an
associated file containing date/time and location either from the platform
navigation database or GPS file.
Latitude - Northern
hemisphere expressed as positive or "N" and
Southern
hemisphere expressed as negative or "S".
Longitude - 0-360o
moving east from Greenwich; west longitude goes from
180o
to 360o; or
Eastern
hemisphere expressed as positive or "E" and
Western
hemisphere expressed as negative or "W".
NOTE - Having a
common date/time stamp and common position coordinates in each data record will
permit the ability to extract data and integrate multiple data records from
different data sets. If two times are
provided (e.g. UTC and local), they should be put at the beginning of each
record.
!
Preferred format for ASCII data files is space, comma or tab
delimited columns, with a UTC date/time stamp at the beginning of each data
record. If the data in the file are comma delimited, decimal places must be
periods, not commas.
!
All data files must contain variable names and units of
measurements as column headings (if applicable).
!
If, for some reason, the PI cannot provide the date/time in
the format shown above, it is important that the time be given in UTC. If local
time is also supplied, a conversion to UTC must be provided. In addition to UTC
and/or local time, other date/time formats (e.g. decimal days) can be used but
must be fully documented.
!
The internal format structure of the file should remain
constant after the first data record to ensure continuity and permit plotting
and graphing.
!
Only COMPLETE replacement or updated data/metadata files can
be accepted.
SAMPLE DATA SET
(ASCII FORMAT):
The following is an
example of an ASCII format data set in which the header precedes the reported
data, and the data is organized in columns separated by spaces. Each column is
identified by parameter and each parameter's units of measure are listed in the
respective column. Also each row has a date/time of observation reported in
Universal Time Coordinated (UTC) along with position coordinates. This data set
organization is ideal for plotting and integration of various data sets. This
data set format should be used whenever possible and could be easily produced
automatically from a spread sheet computer program.
‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑
PI/DATA CONTACT= Doe,
John (U of Hawaii)/ Doe, Jane (NCAR)
DATA COVERAGE =
START: 19990821133500; STOP: 19990821135500 UTC
PLATFORM/SITE = C130
INSTRUMENT = C‑130
External Sampler Data
LOCATION = mobile
DATA VERSION = 1.0
(10 March 1999), PRELIMINARY
REMARKS = National
Center for Atmospheric Research, INDOEX
REMARKS = ppm values
are mole fraction
REMARKS= nM/m3 at 25c
and 101.3 kPa; DMS and NH4 in Parts per million (PPM)
REMARKS = Missing
data = 99.9; Bad data = 88.8
REMARKS = Data point
Date/Time provided in UTC
DATE/TIME LAT LONG SAMPLE NO2 CO DMS NH4
UTC Deg Deg NUMBER nM/m3 ppm ppm ppm
19990821133500.00 -43.087 263.116 E1.160.1 1000.65 200.67 345.98 2342.980
19990821133510.00 -43.090 263.120 E1.160.2 1003.45 200.60 349.76 2353.345
.
.
.
etc.
4.3
Data Documentation
Guidelines
The metadata documentation
(i.e. the "Readme" file) that accompanies each project data set is as
important as the data itself. This
information permits collaborators and other analysts to become aware of the
data and to understand any limitations or special characteristics of data that
may impact its use elsewhere. The data
set documentation should accompany all data set submissions and contain the
information listed in the outline below.
While it will not be appropriate for each and every dataset to have
information in each documentation category, the following outline (and content)
should be adhered to as closely as possible to make the documentation
consistent across all data sets. It is
also recommended that a documentation file submission accompany for each
preliminary and final data set.
---TITLE: This should
match the data set name
---AUTHOR(S):
-Name(s)
of PI and all co-PIs
-Complete
mailing address, telephone/facsimile Nos., web pages and E-mail address of PI
-Similar
contact information for data questions (if different than above)
---DATA SET OVERVIEW:
-Introduction
or abstract
-Time
period covered by the data
-Physical
location of the measurement or platform (latitude/longitude/elevation)
-Data
source, if applicable (e.g. for operational data include agency)
-Any
World Wide Web address references (i.e. additional documentation such as
Project WWW site)
---INSTRUMENT
DESCRIPTION:
-Brief
text (i.e. 1-2 paragraphs) describing the instrument with references
-Figures
(or links), if applicable
-Table
of specifications (i.e. accuracy, precision, frequency, etc.)
---DATA COLLECTION
and PROCESSING:
-Description
of data collection
-Description
of derived parameters and processing techniques used
-Description
of quality control procedures
-Data
intercomparisons, if applicable
---DATA FORMAT:
-Data
file structure, format and file naming conventions (e.g. column delimited
ASCII, NetCDF, GIF, JPEG, etc.)
-Data
format and layout (i.e. description of header/data records, sample records)
-List
of parameters with units, sampling intervals, frequency, range
-Description
of flags, codes used in the data, and definitions (i.e. good, questionable,
missing, estimated, etc.)
-Data
version number and date
---DATA REMARKS:
-PI's
assessment of the data (i.e. disclaimers, instrument problems, quality issues,
etc.) Missing data periods
-Software
compatibility (i.e. list of existing software to view/manipulate the data)
---REFERENCES:
-List
of documents cited in this data set description
4.4
Procedures for
Submission of datasets to JOSS ATLAS archive
JOSS now has on-line
a home page dedicated to ATLAS/ITEX data management activities. It is located at: http://www.joss.ucar.edu/atlas/
The page and links
contained therein can be easily modified and updated to meet changing project
needs. Our plan is to provide a single
point of contact to access the variety of project research data over the life
of the program, provide a single location for accessing supporting operational
data and to reach related WWW pages.
ATLAS and ITEX
datasets can be submitted to JOSS at any time.
The data should be provided following the format guidelines described in
Section 4.2 and be accompanied by documentation that information described in Section
4.3. Datasets can be identified as
"preliminary" or "final" by the investigator in the file
name or documentation that accompanies the data.
It will be possible
for each research data set to be password protected so that only the
contributing investigator can access the data.
The LAII Steering Committee has discouraged use of password protected
datasets to maximize the ease of data exchange among project participants. It should be noted that CODIAC does keep
track of users who download data. This
information can be given to the data provider as requested.
JOSS could implement
password protection on ATLAS and ITEX research data sets, if specifically
instructed by the provider. A user ID
and password could be assigned to individual or groups of datasets to meet this
requirement as instructed by the PI.
The duration of password protection should be specified by the
investigator submitting the data. If
he or she wishes someone else to access the data, it will be that
investigator's responsibility to provide the password for access. JOSS will not act as the "data
police" in this regard.
All operational data
sets (i.e. those data acquired from NWS, other public sources on the WWW or
elsewhere) will be open data sets and accessible by anyone accessing the
ATLAS/ITEX web pages at JOSS. JOSS will
acquire and archive any of these supporting data that are used by the
participants in their analysis.
Otherwise, it will be possible for the users to link to these data
sources from the JOSS home page.
4.5
Collection and availability
of supplemental operational datasets to ATLAS
The ATLAS
Implementation Plan lists data available from other sources that are
desired. JOSS has begun a preliminary
collection of some operational products, these can be found on the JOSS ATLAS
web page www.joss.ucar.edu/atlas These data were collected during some
period of the recently completed 1999 field campaign. Initial operational data
collected in 1999 included:
- NOAA 14 and 15 polar orbiter passes for VIS
and IR channels (2,3,4)
-
Eta model output for
surface, 850,700,500,300hPa and selected cross sections in the Alaska region
- Links to Climatological data from;
National
Climate Data Center (Alaska climatology
data by month)
Western
Regional Climate Center
Alaska
Climate Research Center
Alaska
State Climate Center
JOSS
has been exploring different ways to get access to the supporting data. Table 3 lists the supporting data for the
2000 field season we have found from a variety of sources and includes comments
about the source, availability in real time or delayed mode and the format of
the data, if known. We have implemented
access to these data and/or links to the originating web sites available
through the ATLAS home page at JOSS.
Table 3. Operational and Research
Supporting Data for ATLAS/ITEX |
||||
Product |
Availability |
Source |
Product/Measurement Type |
Format |
Satellite Data |
||||
AVHRR Imagery - |
~ 1 day delay |
ARM Archive: http://www.xdc.arm.gov/data/prod/nsa/avhrr |
Browse plots for channels 2 & 4 from NOAA POES Satellites |
jpeg |
AVHRR Data |
Unknown delay |
ARM Archive |
All channels from NOAA POES Satellites |
tdf or hdf |
AVHRR bi-weekly composites |
? |
Need Source |
? |
? |
Translated Data (NDVI?) |
? |
Need Source |
? |
? |
SAR Data |
See web pages |
Alaska SAR Facility http://www.asf.alaska.edu/daac_documents/avail_data.html#sar_standard_images |
Various |
See web page |
Surface Data |
||||
Northern and Central Alaska NWS Stations (1-20) |
Real-Time |
JOSS via Family of Services (FOS) feed |
Hourly METAR reports |
ASCII coded metar reports |
NOAA CMDL Barrow - Hourly Data |
Real-Time |
NOAA/CMDL: http://www.cmdl.noaa.gov/obop/met |
Hourly, Daily, Monthly raw and summary data and plots |
ASCII |
BLM/RAWS Network |
Real-Time |
NWS Boise: http://www.boi.noaa.gov/FWXEXP/RAWS/SUMMARY/raws_summaries.htm |
Hourly Met reports |
ASCII |
LTER Toolik Site |
See web site |
http://www.mbl.edu/html/ECOSYSTEMS/lterhtml/datbase1.html |
See web site |
ASCII |
LTER Bonanza Creek Site |
See web site |
http://www.lter.uaf.edu/ |
See web site |
ASCII |
Radiation Data |
? |
CMDL-Barrow?, LTER sites? |
? |
ASCII |
Upper Air Data |
||||
NWS Alaska Stations |
Real-Time |
JOSS via Family of Services (FOS) Feed. |
WMO Coded Mandatory and Significant Level Data |
ASCII |
NWS Alaska Stations |
Months delay |
NCDC |
10 second Data |
ASCII |
CMDL Barrow Station |
? |
NOAA/CMDL: http://www.cmdl.noaa.gov/obop/index.html |
? |
ASCII |
Atqasuk |
? |
Need Source |
? |
? |
Model Output Fields |
||||
ETA -Alaska 47.5 km resolution grid Images |
Real-Time |
JOSS via Family of Services (FOS) Feed/National Weather Service - Anchorage |
Sea-level Pressure, 850 mb Ht, 700 mb Ht, RH, 500 mb Ht, vorticity |
GIF |
ETA -Alaska 47.5 km resolution Grids |
Real-Time |
JOSS via Family of Services (FOS) Feed |
Eta 216 Grids, model run every 6 hours, Forecast hours 0-24+ |
GRIB |
NOGAPS Model - Alaska Grids |
Real-Time |
FNMOC |
NOGAPS Global grids, model run every 12 hours, Forecast hours 0-144 |
GRIB |
AVN/MRF Model - Alaska Grids |
Real-Time |
JOSS via Family of Services (FOS) Feed |
AVN 203 grid, 190 km resolution, run every 12 hours, Forecast hours 0-120 |
GRIB |
Climatology Data |
||||
Monthly Mean Values for Alaska Stations |
Months delay |
NCDC, WRCC |
Monthly Climatological Report for the state of Alaska |
PDF Reports, ASCII |
Monthly Total Values for Alaska Stations |
Months delay |
NCDC, WRCC |
Monthly Climatological Report for the state of Alaska |
PDF Reports, ASCII |
Daily Mean Values for Alaska Stations |
Months delay |
NCDC, WRCC |
Monthly Climatological Report for the state of Alaska |
PDF Reports, ASCII |
Monthly Min and Max Values for Alaska Stations |
Months delay |
NCDC, WRCC |
Monthly Climatological Report for the state of Alaska |
PDF Reports, ASCII |
Climo Data for CMDL Station Barrow |
Months delay |
NOAA/CMDL http://wwwsrv.cmdl.noaa.gov/obop/met/index.shtml |
Daily max,mins, daily/monthly means |
ASCII |
Climo Data for LTER Station Toolik |
See web site |
http://www.mbl.edu/html/ECOSYSTEMS/lterhtml/datbase1.html |
See web site |
ASCII |
Climo Data for LTER Bonanza Creek Site |
See web site |
http://www.lter.uaf.edu/ |
See web site |
ASCII |
Historical (prior 1949) Climatological Data for Northern and Central Alaska |
? |
Need Source |
? |
? |
Historical and Recent Climatological Data from East Siberia |
Delay |
Romanovsky, others?? |
Air and ground temperatures, snow depth |
ASCII |
4.6
Special processing of
data to produce integrated or composite datasets
One of the important
contributions to be made by JOSS will be to help the ATLAS investigators
retrieve datasets from collaborating investigators, operational sources and
other data centers as necessary to meet ATLAS analysis objectives. This process should be done as easily and
efficiently as possible, giving investigators flexibility to retrieve the data
they need. At the same time, it is
important that any process discussed here will honor dataset access
restrictions imposed by the investigators.
The questionnaire suggests that some investigators want to be asked
directly by those requesting their data while others are willing to make their
data public at the time of submission to the archive.
There are three
important modes of data retrieval to be implemented in support of ATLAS.
First, and perhaps
most obvious, it will be possible for any ATLAS/ITEX investigator to retrieve
any complete dataset that has been submitted, including the documentation. Any requests for original submitted
datasets, as opposed to special datasets prepared by JOSS (see next paragraph),
will be provided to the requestor in whatever format it was submitted to
JOSS. We will work with the
investigators to standardize dataset submission where possible to simplify use
by others and streamline the access procedures. This will include both research and operational datasets archived
at the JOSS site. The operational
datasets will be accessible by anyone while research datasets can have access
restrictions as specified by the data provider.
Second, as an
enhancement for use by the ATLAS community, JOSS proposes to develop dataset
parameter extraction software that will permit an individual to access a
research dataset that might contain many parameters and extract only those
parameters of interest in a time window specified by the requestor. In this way the requestor can build a new
dataset. This capability will only be
possible if data providers are willing to follow format guidelines described in
Section 4.2 so that the extraction software can operate efficiently on the
dataset. The resultant data subset file
will contain only the parameters requested for the time window of
interest. This process will be possible
using a web-based interface that will be part of the JOSS data management system. JOSS will work with the users on use of the
parameter extraction capability.
Third, JOSS will also
provide links to other data centers, institutions, or individual URLs as
recommended by the ATLAS scientists.
This will allow ATLAS/ITEX participants and other interested scientists
to come to a single web site at JOSS and access all ATLAS related data. We will work with the LAII SMO to avoid
duplication while retaining flexibility and ease for reaching project
information.
5.
Components of the
Modeling Research Dataset
JOSS is committed to
helping the ATLAS and ITEX investigators share their data as quickly and easily
as possible. Tables 4 shows the variety
of modeling projects that will be undertaken during the projects. Table 5 summarizes how the data collected
during the field phase will be shared among the modeling projects after it is
collected. JOSS intends to help this
exchange as much as possible using techniques and expertise described in previous sections.
Model |
Investigators |
Description |
RACIE Regional Approach for CO2 flux Integration and Extrapolation |
Vourlitis, Oechel |
Phenomenological model for spatial extrapolation of fluxes of energy, moisture and CO2 |
RATS Remote-sensing Analysis of Tundra Systems |
Epstein |
Phenomenological models relating spatial and temporal climate characteristics to NDVI traits |
SPA Soil-Plant-Atmosphere Model |
Rastetter, Williams |
A multi-layer canopy model to predict hourly carbon and water exchange at a single site |
ACM Aggregated Canopy Model |
Rastetter, Williams |
A one-layer canopy model to predict daily carbon and water exchange at a single site |
WSR Winter Soil Respiration Model |
Schimel |
A model to predict single site soil respiration at as-yet undetermined temporal scales |
GEM General Ecosystem Model |
Rastetter |
A model to predict annual estimates (up to centuries) of biomass, respiration, leaf area and carbon exchange at a single site |
TEM/DVM Terrestrial Ecosystem Model, Dynamic Vegetation Model |
McGuire |
The former predicts the spatial distribution of biogeochemical properties and the latter also predicts vegetation distribution, on as-yet undetermined temporal scales |
ARCSyM Arctic Regional Climate System Model |
Lynch |
An atmosphere-land surface-sea ice-ocean climate model that predicts all relevant parameters at spatial scales from 10 to 100km and temporal scales of months to a few years, and can also run at a single site |
LSM Land Surface Model |
Bonan, Lynch |
A plot scale soil-vegetation model which is also a component of ARCSyM, which predicts atmosphere-surface exchange of energy, moisture and carbon, soil moisture and temperature |
RAMS Regional Atmospheric Modeling System |
Olsson |
A high resolution atmospheric model that predicts all relevant parameters at spatial scales from 1-10km and temporal scales of hours to days |
Snow |
Liston |
A snow redistribution model |
Permafrost |
Romanovsky |
A detailed site specific numerical permafrost model and a spatial distributed analytical model, to be adapted for use in LSM and TVM |
ARHYTHM Arctic Hydrologic and Thermal Model |
Kane, Hinzman |
A spatially distributed model of hydrologic and thermal processes including soil moisture dynamics, stream runoff, evapo-transpiration, snowmelt, and active layer freezing and thawing." |
ALFRESCO Alaskan Frame-Based Ecosystem Code |
Lloyd, Fastie |
A spatially explicit model of treeline change |
Table 5: Data Needs of Models from Field and Remote Sensing Measurements
Model Suite Date Source |
RACIE |
RATS |
SPA,CSA ACM GEM TEM,WSR |
DVM ALFRESCO |
ARCSyMLSM |
RAMS Snow |
Permafrost ARHYTHM |
Chapin, Lynch |
Met., soil H20 |
Veg, LAI, met. |
Fluxes, met. |
Met., veg. |
Fluxes, met., veg. |
Met. |
Met. |
Hinkel |
Soil moisture |
Thaw, soil H20 |
Soil H20 |
Soil H20, thaw |
Soil H20, thaw |
Soil H20, thaw |
Soil H20, thaw |
Hobbiea |
|
|
|
|
Veg |
|
Snow |
Oechel |
|
Corrected AVHRR time series images |
Fluxes, met. |
Met. |
Fluxes, met. |
Met. |
Met. |
Romanovsky |
Soil H20 |
Thaw, soil H20 and T- |
Soil H20 |
Soil H20, thaw |
Soil H20, thaw |
Soil thaw |
Soil H20, thaw |
Sturm, Liston, Olsson |
Snow depth, cover |
Snow cover, snow depth, time of snow melt |
Time of snow melt |
Time of snow melt |
Snow depth, time of snow melt |
Snow depth |
Snow depth |
Walker |
Veg, LAI |
Veg, LAI |
Veg, LAI |
Veg, LAI |
Veg, LAI |
Veg, LAI |
Veg, LAI |
Schimel, Welker, Ping |
Soils, flux |
Soil N |
Soils, flux |
Soils, flux |
Soils |
|
Soils |
Hinzman, Kane |
Met., soil H20 |
Met., soil H20 |
Met., soil H20 |
Met., soil H20 |
Met., soil H20 |
Met., soil H20 |
Met., soil H20 |
Lloyd, Fastie |
Tree distribu-tion |
Tree distribu-tion |
Tree distribution |
Tree distribution |
Tree distribu-tion |
Tree distribu-tion |
Tree distribution |
a. This is a modeling project with no field component.
There will be a
number of model output products generated during ATLAS and ITEX. The list of known products is given in
Table 6. JOSS will make these output
datasets available through the CODIAC archive.
It may be necessary to store particularly large output files
off-line. We intend to work with the
participants to make all critical model product datasets easily available to
others. This may require the
investigator to subset their data set prior to submission to JOSS so that
critical parameters in time series, multi-dimensional grids or other layouts
can be available for on-line ordering while less frequently used output would
be stored off-line. This will be
handled on a case-by-case basis
Model |
Product |
RACIE |
Spatial extrapolation of fluxes of CO2 |
RATS |
Vegetation biomass and phenology from NDVI |
SPA/ACM |
Vegetation, fluxes of energy, moisture, CO2 |
GEM |
Biomass, C flux, N biogeochemistry |
WSR |
Winter CO2 flux |
TEM/DVM |
Vegetation distribution, C flux, N biogeochemistry |
ARCSyM/LSM |
Climate, soil moisture and temperature, energy and moisture fluxes at field sites (6 hourly) and on regional grids (monthly means), each year for the previous years field season. ARCSyM output is available for download from the web by going to http://cires.colorado.edu/arcsym/ and selecting ATLAS results. |
RAMS/Snow |
Snow, energy and moisture flux aggregation information |
Permafrost |
Soil temperature and unfrozen moisture, thaw |
ARHYTHM |
Soil moisture, soil temperature, ET |
ALFRESCO |
Ecosystem distribution |
6.
Training on JOSS data
management system
JOSS will provide
user training to all interested people on the use of our data management system
(CODIAC). This includes matters of data
formatting, data and documentation submission, data and documentation content,
password protection, data ordering and any related data management task. The intent is to make the system as easy to
use as possible. We want to make sure
that ATLAS Project participants are comfortable with the system and its
capabilities. Assistance will be
available at any time to assure users are taking advantage of CODIAC
capabilities. This training will be
offered throughout the support period and provided by JOSS support staff.
7.
Data Archival and
long-term access
JOSS will be the
interim archive center for ATLAS and ITEX during the field and analysis phases
from now through 2002 and up to two years after the conclusion of field
activities. We have had initial discussions with NSIDC on certain aspects of
the ATLAS data management strategy as it fits within the larger context of
ARCSS. The ARCSS data protocol is
available at the following URL: http://arcss.colorado.edu/Protocol/protocol.html JOSS intends to
be responsive to specific research and data access needs of the ATLAS community
that go beyond NSIDC responsibilities during and immediately following the
field phase. This includes the use of CODIAC for archival and exchange of
primary ATLAS data, access to certain types of special data (composite
datasets) or arrangements for data exchange with other data centers. As noted in Section 4.6, JOSS also plans to
provide specific value added processing for the projects to aid in synthesis
and other analysis activities. All
final ATLAS datasets will be transferred to NSIDC for long-term archival. JOSS will work with NSIDC to make sure that
the complete and final project dataset is accessible in the long term.
8.
ATLAS/ITEX Data
Questionnaire
Over the past several months JOSS has
prepared, distributed and received responses to a data management
questionnaire. The purpose of the
questionnaire was to determine the data
sources and needs of the investigators in order to accomplish critical science
objectives. The questions asked in this questionnaire complement information
contained in the ATLAS Implementation Plan.
Table 7 contains a list of respondents to the
questionnaire. The information from
these people was used extensively in the preparation of the ATLAS data
management strategy discussed in this document. Following Table 6 is the text
summary of questions to which response was requested. This data questionnaire may be found at; www.joss.ucar.edu/atlas/ Many of the project participants responded
to the questionnaire and all
information received from investigators can be viewed at: www.joss.ucar.edu/atlas/ . The complete text of the Data Questionnaire
can be found in Appendix B.
Table 7. Respondents to Data Questionnaire
Howard Epstein University of Virginia
Steven F. Oberbauer Florida International University
Ed Rastetter Marine
Biological Laboratory, Woods Hole Institute of Oceanography
Kenneth M. Hinkel University of Cincinnati
Glen E. Liston Colorado State
University
Jace Fahnestock University of Wyoming
A. David McGuire University of Alaska
Fairbanks
Chien-Lu Ping University of Alaska
Fairbanks
Bonnie Kwiatkowski Marine Biological
Laboratory/Woods Hole
Institute of
Oceanography
Gaius R. Shaver The
Ecosystems Center, Marine Biological Laboratory, Woods Hole Institute of
Oceanography
Larry Hinzman University
of Alaska Fairbanks
Patrick John Webber Michigan
State University
Wanli Wu CIRES/University
of Colorado
Amanda Lynch University of Colorado
Robert D Hollister Michigan State University
Andrea Lloyd Middlebury College
Vladimir Romanovsky Geophysical
Institute, Dept. Geology and Geophysics University of Alaska Fairbanks
Allen Hope San Diego State
University
Doug Stow San Diego State
University
Jason Beringer University of Alaska
Fairbanks
Appendix A Acronym list
Appendix B Complete Data Questionnaire
A
ACM Aggregated
Canopy Model
ARCSS Arctic
System Science program (NSF)
ARCSyM Arctic
Regional Climate System Model
ARM Atmospheric
Radiation Measurement program (DOE)
ATLAS Arctic
Transitions in the Land-Atmosphere System
AVHRR Advanced
Very High Resolution Radiometer
AIST ATLAS/ITES
Science Team
BLM US
Bureau of Land Management
CALM Circumpolar
Active Layer Monitoring
CMDL Climate
Monitoring and Diagnostic Laboratory
CODIAC JOSS Data Management System
FTP (Internet)
File Transfer Format
G
GCOS Global
Climate Observation System (W CRP)
GCTE Global
Change and Terrestrial Ecosystems
(IGBP)
GEM General
Ecosystem Model
ITEX International
Tundra Experiment
JOSS Joint
Office of Science Support
LAI Leaf
area index
LAII Land-Atmosphere-Ice
Interactions (ARCSS)
LSM Land
Surface Model
LTER Long
Term Ecological Research
NSF National
Science Foundation
NDVI Normalized
Difference Vegetation Index
NSIDC National
Snow and Ice Data Center
NWS National
Weather Service
NOAA National
Oceanic and Atmospheric Administration
OAII Ocean-Atmosphere-Ice
Interaction
PACE Permafrost
and Climate Change in Europe
POES Polar
Orbiting Environmental Satellite
R
RAICE Regional
Approach for CO2 flux Integration and Extrapolation
RAISE Russian-American
Int of Shelf Land Environments in the Arctic
RAMS
RATS Remote-sensing
Analysis of Tundra Systems
RAWS Remote
Automatic Weather Station
S
SIMS Synthesis,
Integration and Modeling Studies
(ARCSS)
SMO Science
Management Office
SPA Soil-Plant-Atmosphere
Model
TEM/DVM Terrestrial
Ecosystem Model/Dynamic Vegetation Model
UCAR University Corporation for
Atmospheric Research
UTC Coordinated
Universal Time
WSR Winter
Soil Respiration Model
WWW World
Wide Web
Complete text of the ATLAS/ITEX Data Questionnaire
----------------------------------------------------------
All questions below, unless otherwise stated, refer to data
needed from operational and research sources for post-analysis/research. Please
answer questions as completely as possible and/or all that apply at this time.
Please fill out the following personal information for
general ATLAS reference:
Name: |
|
Affiliation: |
|
Mailing address: |
|
|
|
|
|
E-mail: |
|
Affiliation phone/fax: |
|
2.
What Polar-Orbiter Satellite data do you need to conduct
your research?
Visible band only
Other bands (please specify): __________________________________________
Format: ___________________________________________________________
Frequency:
_________________________________________________________
Other satellite data (Please
specify): _____________________________________
3.
What Operational Model data do you need for your research?
ETA
AVN/MRF
NAVY NOGAPS
Other model needs (please
specify): _____________________________________
4.
Do you require National Weather Service (NWS) Upper Air Data
for your research?
Other needs (please specify): __________________________________________
5.
Do you require NWS Surface Data for your research?
If yes, please specify stations
and/or regions required: _______________________
Other needs (please specify): __________________________________________
6.
Do you require Surface Data from non-NWS stations for your
research?
If yes, please specify:
________________________________________________
7.
Do you require Climatological Data from NWS or non-NWS
stations for your research?
If yes, please specify stations
required:
__________________________________
If yes, please specify type of
climatological data required: ____________________
8.
What other operational datasets do you need to conduct your
research?
Please specify: _____________________________________________________
9.
What dataset(s) are you responsible for collecting?
Instrument/model:
___________________________________________________
Variables measured:
_________________________________________________
10.
How often will you be collecting data from this instrument?
______________________
11.
Please specify the format(s) of these data:
Field format?
_______________________________________________________
Archive format?
_____________________________________________________
12.
To whom do you wish to release the data?
ATLAS researchers only
LAII only
ARCSS only
Public domain
Ask you (the PI) for confirmation
of all requests
13.
Please specify other research datasets you require to do
your research:
______________________________________________________________________
14.
Specify the preferred media for accessing archived ATLAS
data:
______________________________________________________________________
15.
Do you have any other comments/requests regarding data
collection and access for the ATLAS project?
______________________________________________________________________