The content of the ARM website is available to any browser, but for the best experience we highly recommend you upgrade to a standards-compliant browser such as the latest version of Mozilla, Netscape, Internet Explorer, or Opera (all available for free download).

skip to main content

Data Archive Data Management Facility Data Quality Program External Data Center Value-Added Products

Manage ARM Account

• View Account Information

Order ARM Data

• Data Browser Interface (?)
• Catalog Interface (?)
• Thumbnail Browser Interface (?)
• IOP Data (?)
• PI Data Products (?)

Data Plots

• Thumbnail Browser Login Interface
• Browse Quick Look Directory
• NCVweb
• Plot User-Requested Data
• Plot Standing Order Data


• ARM Archive Documentation
• About the ARM Archive

Processing and Access Instructions for ARM Archive Standing Orders

(Karen Gibson, 8/06/2003)

Note: The Standing Order feature is a special service provided by the ARM Archive for ARM sponsored researchers (PI's, Co-PI's, collaborators, and their students and staff) and ARM infrastructure staff (primarily instrument mentors and data quality staff). Other Archive users are encouraged to request data as needed from the user interface.

• An overview of the standing order logic
• How to compose a standing order
• How to access ordered data
• file storage locations, retention times
• file transfer options

Overview of Standing Order Logic

This information is provided to give you a general understanding of our process. This will help you understand how it works, how you can use it most efficiently, and what types of failures may occur. ost of the steps below are automated.

Processing Steps

1. A data user specifies a standing order (more detail in the next section)
2. The Archive maintains a cumulative list of needed data
3. New data files arriving at the Archive for storage are reviewed for standing order applicability.
4. Applicable files are temporarily saved as an additional copy on the FTP server at the Archive.
5. Delivery directories and notices are generated for each delivery interval (daily, weekly, or monthly)
1. Note: The deliveries are generated on the "anniversary" of the creation of the standing order. Therefore, weekly delivery notices may be sent any day of the week, and monthly notices are generated every 28 days.
2. Note: You will receive a "null notice" if no files are available for a time interval.
6. A data user transfers data to their system (more detail in last section).
7. The additional copy of the data files is removed after 35 days. (Note: This is subject to further revision).

Some expected, but "surprising" results

• The content of data delivery depends on data received during the previous time period. For data streams that are regularly sent to the Archive, the variation between deliveries are likely to be minimal. For data streams that are intermittently sent to the Archive (e.g., TWP data streams), the contents will vary considerably.
• The time span of a delivery period ends at midnight of the previous day.
• It is possible for a user to have more than one standing order.
• Multiple users can share a standing order, but each user will have their own delivery directories.

How to compose a standing order 

Contact ARM Archive User Services

A Standing Order can be created for you by Archive User Services (865-241-4851, FAX 865-574-4665. armarchive@ornl.gov ). When you contact user services, we need to know the following items to specify a Standing Order:

• An email address for sending notifications
• The delivery / notification interval (daily, weekly or monthly)
• A title for the order (this optional, but helpful).
• The list of data streams included in the order (Definition: A data stream is the series of files from a single source (instrument or algorithm), a single place, and a single processing level. For example, sgp30smosE13.a1 is the 30 minute SMOS (surface meteorology) for Lamont, OK.) The data stream names are sometimes complex; we can help you determine the ones you need.

Data Accessibility

Storage location: delivery directories

Data files requested by Standing Orders begin accumulating in temporary storage very shortly (less than one day) after the order is placed. You will receive a notification (one e-mail message) at the end of each delivery period of each order. This notification will describe the delivery period, the number of files included in the delivery ("null" if no files are available), and a list of file names. 

The notification will describe the location of the data files on ftp.archive.arm.gov. (This system is an FTP server that can be accessed in anonymous mode with username armguest and your e-mail as the password.) The notification will describe the data files as stored under a path of:

• /armguest/"username"/"standing order number.delivery date"
• Where:
• "username" is your username for the Archive user interface
• "standing order number" is the number assigned when it was specified
• "delivery date" is the date of the delivery / notification in YYYYMMDD format.
• Note: All deliveries for a standing order will be a series of directories under the standing order directory (has a numerical name).

REVISION: The retention time for data files depends on the delivery interval. File retention for standing orders uses the following rules:

• Delivery interval :: Retention time after delivery date (days)
• daily :: 7 days
• weekly :: 21 days
• monthly :: 28 days

Storage location: holding directories (for the impatient)

The data files for the entire collection of standing orders are actually stored in central data structure on the FTP server. If you need access to data files that have not been delivered (e.g., part way through a week), you may access the temporary copy with the following steps:

1. FTP to ftp.archive.arm.gov as username: anonymous, password: your e-mail address
2. cd to the standing_orders directory
3. cd to the sub-directory by data stream name (e.g. sgp1smosE13.a1)
4. Transfer files (ftp get command) as needed.
5. NOTE: This area contains file names with version numbers and all versions of the data files. The delivery directories only contain the highest version of the data files and are named without version numbers.

File transfer options

Data files from Standing Orders can be transferred with the following options:

• Manual FTP - Access FTP server as described in the section above called "Storage Locations: Delivery Directories" 
• FTP with "mirror" - This is a special FTP script that can be used to replicate a file collection (and directory sturcture) from the Archive server to your local computer. Information on how to configure "mirror" can be found at here.
• "automated" FTP with "mirror" - It is possible to put the "mirror" process into a crontab in UNIX (or other scheduling software) on your system. Information on how to configure an automated setup is also provided in the URL above.
• Tape - The Archive has hardware to write data to tape in 4mm, 8mm, DLT, and IBM 3590 format. If you need data transferred on tape, please contact the Archive Manager (Raymond McCord, mccordra@ornl.gov, 865-574-7827). Note: This is the least preferable option and involves significantly greater cost (labor and material) than the FTP options. 

---

Atmospheric Radiation Measurement (ARM) Archive / email: info@arm.gov / Call 1-888-ARM-DATA
The ARM Archive is located at Oak Ridge National Laboratory (ORNL) / Warnings and Disclaimers

Send comments for: WWW Administrator

Last Modified: March 11, 2005

Privacy & Security Notice

All rights reserved.