Frequently Asked Questions | STORET/WQX | Water

STORET/ WQX Frequently Asked Questions

Features

What is New in STORET/WQX?

What is New with Warehouse Services?

STORET Helpdesk

USGS/EPA Data Exchange Agreement

WQX Web Release Notes

Newest entry added May 05, 2011.

General STORET/ WQX Questions

What is STORET and how does it relate to WQX?

STORET refers overall to "STORage and RETrieval", an electronic data system for water quality monitoring data developed by EPA. STORET has taken various forms since the 1960's (see next FAQ). Since about 2000, STORET has referred to a local data management system ("Modernized STORET") as well as data repository ("STORET Data Warehouse") developed for purposes of assisting data owners manage data locally and share data nationally. Up until September 2009, the distributed STORET database has been used to compile data at the national level in the STORET Data Warehouse. As of September 2009, the Water Quality Exchange, or WQX framework, provides the main mechanism for submitting data to the STORET Data Warehouse. top

Can you explain the terms "Legacy STORET", "Legacy Data Center", "Modernized STORET", "Distributed STORET Database", "Water Quality Exchange" and "STORET Data Warehouse"?

"Legacy STORET" is the term we use when we mean the original (mainframe) STORET Water Quality File. From 1965 until 1998, this system was known simply as "STORET", and was the only STORET database. Storing data in it, and retrieving data from it could only be accomplished by someone with a mainframe User ID, and with specialized training. Since the middle of 1999, it has not been possible to use this system for data entry, and in the year 2000 it became unusable for retrieval as well.

In order to preserve the data content of Legacy STORET, EPA created the "Legacy Data Center". It uses modern relational database technology, and is housed on a UNIX/Oracle database server. The general public may browse or download its data, using a standard web browser such as Microsoft IE. All the data content from the Legacy STORET database was copied to this new database server during 1999, and the mainframe Legacy STORET was shut down. Updates are not permitted to the Legacy Data Center.

"Modernized STORET" refers to the system created to replace Legacy STORET for the storage and retrieval of water quality and biological data. Unlike its predecessor, Modernized STORET is a distributed database system, installed on personal computers all over the country for use by data owners. Often times, we refer to Modernized STORET as the "distributed STORET database". Up until September 2009, local STORET owners could upload mirror images of their STORET data for inclusion in the STORET Data Warehouse. This upload of the distributed STORET databases to the STORET Data Warehouse is no longer supported by EPA, rather users must submit data via the Water Quality Exchange, or WQX.

The "Water Quality Exchange", or WQX, is the primary framework for submitting data to the STORET Data Warehouse. WQX uses the technology, standards, and protocols of the National Environmental Information Exchange Network, or Exchange Network, to provide a means for data partners to share water quality monitoring data to the STORET Data Warehouse. WQX is not a distributed database, but rather a standard set of data elements that all data partners map to in order to share data.

The "STORET Data Warehouse"continues to be EPA's main repository of water quality monitoring data, including data that has been submitted via Modernized STORET as well as data that is now submitted via WQX. The STORET Data Warehouse houses approximately 80 million water quality records. This data continues to be provided by other federal agencies, region, states and tribal partners as well as voluntary organizations. top

What is WQX? Is WQX a database?

The Water Quality Exchange (WQX) is a framework that allows states, tribes, and other data partners to submit and share water quality monitoring data via the web to the STORET Data Warehouse. WQX uses the technology, standards, and protocols of the National Environmental Information Exchange Network, or Exchange Network, to provide a means for data partners to share water quality monitoring data to the STORET Data Warehouse. WQX is not a distributed database, but rather a standard set of data elements that all data partners map to in order to share data. States, tribes and others who store water quality data in a personal database and have the ability to generate XML files, can now submit data directly to the publicly-accessible STORET Data Warehouse using the WQX framework. top

What is WQX Web and how is it different from WQX?

WQX Web is a web-based application designed to assist smaller organizations with submitting data via WQX. With WQX, users submit WQX XML that they have already created via Exchange Network nodes or node clients. With WQX Web, users log into a web site where they have access to WQX Web for data submission. WQX Web converts spreadsheets and text files (such as comma or tab-delimited files) into a valid WQX XML submission file (based on the WQX schema). While WQX defines the framework for which data is submitted to the STORET Data Warehouse, WQX Web is a web-based tool with a user interface that provides for data submission using the WQX schema. top

What is the Exchange Network and how does it relate to WQX and STORET?

The National Environmental Information Exchange Network, or Exchange Network, is an EPA and state partnership that provides a secure framework for sharing environmental information over the internet. Many different types of environmental data are shared over the Exchange Network, including air quality data, hazardous waste handling information, or even standardized chemical names used across EPA. WQX is the water quality monitoring data flow on the Exchange Network. top

What's the relationship between EPA monitoring data in STORET and USGS monitoring data in NWIS?

In the past, the USGS and STORET have agreed to utilize common definitions and formats, to promote a common view of data between their two systems. The full text of this agreement, made in 2000, is available online. NWIS and STORET have met this agreement by creating common web services. These web services provide data out in a WQX schema format. top

Is there a document that describes the parameter codes in Legacy STORET?

Legacy STORET parameter codes can be found by clicking on the following URL ftp://ftp.epa.gov/storet/ and selecting the zip file labeled ALL_STORET_parameters_codes.zip. The zip files contains two text files, one text file containing the list of parameter codes and the other text file containing associated information on the parameter codes, including the definitions of the data columns. top

Downloading STORET/ WQX Data

How can I obtain water quality data maintained in the STORET Data Warehouse or in the Legacy Data Center?

Anyone with a web browser and internet access can browse and download monitoring data from the STORET Data Warehouse or Legacy Data Center. On the left hand side bar of this webpage, simply go to the link called "Data download". This link takes you to where you can choose between the two systems for downloading data. top

How often is the STORET Warehouse refreshed?

The STORET Data Warehouse is refreshed with new data submitted via WQX weekly. The refresh process is routinely started every week on Wednesday evening, so that data is available the following Thursday. So, if you submit your data on a Monday, you won't see your data in the STORET Data Warehouse until the following Thursday. The Legacy Data Center is a static, archived database that is not updated. top

Can I use "wildcards" in searching for characteristics?

Yes. The percent sign "%" is a wildcard search prefix that searches for parts of a characteristic name if the full name is unknown (example: typing "%disso" or "%oxy" or "%DO" all work). top

Submitting data to WQX Web

Are there example data submission text files available?

Yes. Example WQX Web data text files and XML files are provided by clicking on the following link Example WQX Web Text Files. Additional example XML files are available at the following link Example WQX XML files . top

I made a mistake in submitting my files is there any way to correct the error?

Yes. Previously submitted WQX Web data can be changed by performing an update operation or a delete operation. WQX Web update operations will delete existing data associated with an Activity, Monitoring location, or a Project and replace it with the resubmitted data. Data will only be updated to the level specified, i.e. if a Monitoring Location name is updated only, associated activities and results data will not be changed.
Alternatively, performing a delete operation will remove all levels of data associated with a specific component, deleting a Monitoring Location will also delete all activities and results from the system that are associated with the Monitoring Location. top

My WQX Web submissions have had the status pending for a long time, should I be worried?

When submitting through WQX Web, if your submitted file is ending for over 30 minutes, cancel the submission, delete the dataset, then resubmit your file a few moments later. If you are not able to delete your submitted file, proceed to reimport the file, When prompted by WQX Web if you would like to delete the previous dataset, select OK.