STORET/WQX: System Change Control Log
STORET/WQX System Change Log
This change log details the status of bugs and issues in the STORET/WQX System. Matters of contention that are resolved are listed as completed,while ongoing issues are listed as open.
| Issue Number | Title/Description | Proposed Fix | STORET Helpdesk Tracking #(s) | Fix Date | Status | Received Date |
|---|---|---|---|---|---|---|
| artf317 | Upper/ Lower Confidence Level Columns Removed | Update Post ETL rule to generate both fields concatenated into CONFIDENCE LEVEL | SIR# 317 | Open | 5/16/2014 | |
| artf316 | Stations Aren't Being Retrieved from the Warehouse by PROJECT | Update Post ETL rule to generate Station & Project assignments for stations WITHOUT project station weights. Table: STATION_PROJECT | SIR# 316 | 6/4/2014 | Closed | 4/16/2014 |
| artf11528 | Analytical Proc ID does not support tilda as delimiter between ALL concatenated fields Analytical Proc ID does not support tilda / '~' as delimiter between ALL concatenated fields (proc source ~ proc id) | ALL ETL concatenated fields must have a tilda / '~' as delimiter separating fields before storing in database column. | Open | 4/1/2013 | ||
| artf11658 | ETL Change:Numeric results that are loaded with comma (",") thousands separators Numeric results that are loaded with comma (“,”) thousands separators included are apparently treated as Text Result Values | ETL change to test for numeric result value text and store comma numerics as result value numbers | Open | 4/12/2013 | ||
| artf11657 | WQP impacted ResultMeasureValue is blank and the actual results are reported in the field ResultDetectionConditionText. WQP Tool, If ResultMeasureValue is blank and the actual results are reported in the field ResultDetectionConditionText. | ETL change to test for numeric result value text and store comma numerics as result value numbers | Open | 4/12/2013 | ||
| artf11656 | Two High Priority Bugs found WQX Web and Warehouse Downloads WQX Web does not generate an error/warning when more than one Activity Type is assigned to activity id | Code Change full comparison on all Activity elements and if the IDs are the same, but other element values are not, then we log a warning (or possibly an error). | Open | 4/12/2013 | ||
| artf11655 | 2.0 Questions to User Community: WQX schema has no field for storing metadata - Data Logger time interval (Sampling Interval) | Warehouse & ETL change create new column Sampling Interval (text) varchar2 256. Add to download report columns and WQX schema | Open | 4/12/2013 | ||
| artf11533 | "Retired" Taxon Names established data management procedures Checking the WQX Taxon domain list ( http://www.epa.gov/storet/wqx/wqx_getdomainvalueswebservice.html), that name is on the list and it is not on the cross-walk spreadsheet from September. Checking ITIS, I see “Diatoma mesodon” is a not accepted synonym for “Diatoma hiemale”, so I assume that’s why it’s being rejected. | Send Monthly Domain Value Alerts to Listerver (User Call) | Open | 4/1/2013 | ||
| artf11530 | Failed/Imported Datasets WQX Web: FYI: WQX Web Online Video Tutorial Failed/Imported Datasets WQX Web: WQXWEBTRAINING accounts does not match Spreadsheet Tutorial Template | Add columns to Import configurations | Open | 4/1/2013 | ||
| artf11529 | Javascript request suppress non email characters: Report of pending data requests 2013/03/20 Warehouse application fails query processing when email address contains non standard characters ie commas instead of periods | suppress characters for email address field | Open | 4/1/2013 | ||
| artf11534 | New WQX schema change request (Add Date Stamp (Analysis Date or Sampling Date) for Activity Group Fact Table)) Add Date Stamp (Analysis Date or Sampling Date) for Activity Group) Mandatory Metadata reports for ACTIVITY GROUP are filtered by organization_id Impacts warehouse schema and reports: Warehouse download delayed by ACTIVITY_GROUPS. | 1. Query by Date Stamp (Analysis Date or Sampling Date) for Activity Group 2. Update warehouse application optional download for ACTIVITY_GROUPS | Completed | 4/1/2013 | ||
| artf11532 | Water Quality Portal Issue: Warehouse Station Record column STATION_GROUP_TYPE not updated in post ETL Water Quality Portal Parameter: Site Type is NULL Queries fail to filter by Site type. | Add Post ETL checks to cross map STORET Station Types to WQP Site Types weekly. | 5/3/2013 | Completed | 4/1/2013 | |
| artf11531 | Placement of New columns (Actual Activity Horizontal Accuracy) in warehouse download report - Actual Activity Lat/Long* - group currently has: o Actual Activity Latitude o Actual Activity Longitude o Actual Activity Horizontal Datum o Actual Activity Horizontal Accuracy | New warehouse column email STORET Community via Listserver Alert http://www.epa.gov/storet/download/WarehouseReport_withWQXcolumns.xls | 5/3/2013 | Completed | 4/1/2013 | |
| artf10740 | Warehouse Impacted by WQX ORG_IDS change to UPPER CASE Remove Case Sensitivity from WQX & Warehouse ORGANIZATION_ID. Difficult to capture a list of stations in the Water Quality Portal query parameters | Change all Warehouse ORGANIZATION_ID to UPPER CASE | 1/15/2013 | Completed | 1/14/2013 | |
| artf10376 | Delete Duplicated STORETW Results In some rare instances, Activity ID's are duplicated in the STORET Warehouse. | Delete all records that were not effectively removed before the delete strategy was implemented by the ETL. | 11/14/2012 | Completed | 11/12/2012 | |
| artf10128 | Cleanup script for duplicate characteristics ETL creates duplicate characteristics in Warehouse Domain Value List. | Suppress / Display distinct list of Characteristic Domain Values. Then re-assign duplicate characteristics to minimum identifier. | 12/5/2012 | Completed | 10/10/2012 | |
| artf10125 | Warehouse Download - Revert query to FK_GEN_DB_CAT Warehouse stations and results queries inconsistent in data counts. ETL fails to populate foreign key / query key for huc8 parameters. Warehouse application filters on generated columns. The Warehouse GENERATED Column Rule: if given a "user supplied value" exist then the GENERATED column = "user supplied value". | Fix post ETL script to populate generated foreign keys used by warehouse query parameters. | 1/23/2013 | Completed | 10/10/2012 | |
| artf9756 | ETL truncates RESULT_VALUE_STATUS to 1 character ETL truncates RESULT_VALUE_STATUS to 1 character | Correct ETL to populate the full RESULT_VALUE_STATUS characters (12) | 9/22/2012 | Completed | 9/5/2012 | |
| artf8475 | Warehouse Download Enhancements - Do not clean previous selections when selection changes Do not clear Warehouse interface query selections when a Warehouse query is modified. | 5/8/2012 | Completed | 5/7/2012 | ||
| artf8404 | Add generated state/county to the results table Add generated state and couny fields to the STORET results table. Lat/long is the only required geographic location data in WQX submission | Populate warehouse generated state and county for accurate and consistent query parameters. | Completed | 4/20/2012 | ||
| artf7800 | Data Elements Result Depth and Result Depth Unit are not available in the STORET Warehouse Result Depth and Result Depth Unit are data elements available in WQX, but not queriable through the STORET Warehouse. | 1640 | 7/10/2012 | Completed | 12/27/2011 | |
| artf7672 | Results associated with EPA Beaches data are duplicating EPA Beaches results data is duplicated when downloaded from the STORET Warehouse. | 1571 | Completed | 12/1/2011 | ||
| artf7448 | Characteristic names are duplicated when the activity is data logger (historical results) Data Logger results are duplicated for historical STORET data | 1292 | Completed | 10/27/2011 | ||
| artf7432 | Detection Limits/Upper and Lower Quantification Limit Non detect values for characteristics are not consistently displayed out of the Warehouse. | 791, 644, 1128, 1477 | 12/21/2010 | Completed | 10/26/2011 | |
| artf7431 | STORET Warehouse is not returning results with result comment fields that exceed 3000 characters Results with an extensive amount of characters in the result comment fields are not being returned from the STORET Warehouse. This issue only affects result comment fields exceeding 3900 characters. | 1477 | Completed | 10/26/2011 | ||
| artf7429 | Activity comment field truncation The Activity Comment field is being truncated at 256 characters for biological results only. It should be 4000 characters. | 1477 | 1/6/2012 | Completed | 10/26/2011 | |
| artf7309 | STORET Helpdesk Ticket # 1473 Input and Output fields STORET System a spreadsheet showing the data elements from the WQX 2.1 data dictionary and the output fields from the warehouse. | Update Warehouse, application tool and ETL for reporting WQX data elemets | Completed | 10/12/2011 | ||
| artf7158 | Projects and Monitoring Locations are not being deleted out of the Warehouse. There is currently not a way to delete projects or monitoring locations from the STORET Warehouse. | Develop scripts to mirror actions executed in WQX deleting Monitoring Locations and Projects to concurrently perform the same actions in the STORET Warehouse. | 1351 | Completed | 9/17/2011 | |
| artf7143 | Missing Activity Attached Documents From the Warehouse query application, Activity Document / Graphic URLs are blank when returned in the downloaded text file. | The maximum Activity Document / Graphic unique identifier from WQX must be placed in the correct table and column within the STORET Warehouse, and the refresh process will confirm this unique ID is getting populated and the URL is populating the "Activity/Result Document Graphic" data element in the STORET Warehouse. | 1292, 977 | 1/13/2012 | Completed | 9/14/2011 |
| artf6996 | Activities repeated multiple times Activity ID's are repeated multiple times for results queried from the WQX web services. | Completed | 7/22/2011 | |||
| artf6730 | Activity Media Name field is returning incorrect values for QC submissions WQX data submitted with Activity Types containing the phrase "Quality Control" are not being consistently stored as QC data in the STORET Warehouse. | Explore the universe of data that falls within a QC media within the STORET Warehouse. Explore alternative ways of serving out QC data in order to preserve the original activity media submitted by users. | 420, 1174 | 6/1/2012 | Completed | 5/23/2011 |
| artf6705 | WQX -- STORET Warehouse issue with repeating projects When the STORET GetResults web service is called (http://iaspub.epa.gov/webservices/StoretResultService/), the returned data file sometimes contains repeating Project IDs. | Evaluate the extent of the issue to determine if this is an issue with all GetResults service calls or only a few organizations. Make any needed revisions to the GetResults service code. | 1289 | Completed | 5/17/2011 | |
| artf6628 | Activity Groups are not available out of the STORET Warehouse Activity group is a data element available in WQX, however not out of the STORET Warehouse. | Spool a separate report for Activity Groups based on Organizations data queried. | 1473 | 11/21/2012 | Completed | 4/15/2011 |
| artf6580 | Problems with binary attachments in WQX project files Problems with binary attachments in WQX project files. WQX business rule many attachments can be assigned to one project. ETL to Warehouse rule of one to one project and attachment assignment. Warehouse business rule requirement one per project id. | The Metadata file Project Summary could be updated to include the URLS to download the 'PROJ Binary Object'. ETL only the last (most recent) attachment assigned to WQX project id | 7/28/2012 | Completed | 4/4/2011 | |
| artf6338 | Alternative Monitoring Locations to Warehouse Station Alias Table The STORET Warehouse is not serving out Alternative Monitoring Locations when queried. | 1473 | 1/23/2012 | Completed | 2/16/2011 | |
| artf6337 | Detection Limits Handling It appears that the Warehouse is currently not able to report all detection limits associated with a result in separate columns (WQX data). For example, when data has an Upper Quantification Limit, that value is being reported in both the “Detection Limit” field as well as the “Upper Quantification Limit” field when exported from the Warehouse. | 791, 644, 1128, 1477 | 12/21/2011 | Completed | 2/16/2011 | |
| artf6279 | STORET Warehouse counts and EPA web services counts for Results and Stations should match STORET Warehouse query counts and STORET web services counts for results and stations do not match. When results or stations are queried with EPA web services, counts do not match replicate queries in the STORET Warehouse query application. | Completed | 2/3/2011 | |||
| artf6272 | Map scales for some Monitoring Locations have extra decimal places when downloaded from the STORET Warehouse Some Monitoring Location map scales are being assigned extra decimal places when downloaded from the Warehouse. | This has been fixed. | 725 | 2/28/2011 | Completed | 2/2/2011 |
| artf6271 | Project Document/ Graphic and Project/ Document URL fields are not being returned from the Warehouse From the Warehouse query application, Project Document / Graphic URLs are blank when returned in the downloaded text file. | The maximum Project Document / Graphic unique identifier from WQX must be placed in the correct table and column within the STORET Warehouse, and the refresh process will confirm this unique ID is getting populated and the URL is populating the "Project Document/Graphic" data element in the STORET Warehouse. | 1292, 1473, 725 | 4/13/2011 | Completed | 2/2/2011 |
| artf6203 | WQX toxicity data is improperly mapped to STORET regular results Much of the toxicity data in WQX is getting improperly mapped to the STORET regular result fact table instead of the biological result fact table where it should be mapped to. This behavior is not consistent; the WQX data with "Count" as the characteristic is getting placed in the STORET biological result fact table, but none of the other records are. | WQX Data that flags the activity media field as biological, and/ or Biological Intent as not null, and /or Taxonomic field as not null. will now be mapped as biological results. | Completed | 1/19/2011 | ||
| artf6027 | Characteristic Name is not populated when Category-Rep Num is Routine Habitat From the Storet warehouse, the Characteristic Name field is not populated when Routine Habitat is indicated for the field Category-Rep Num. This issue only relates to distributed STORET data. | 2/28/2011 | Completed | 12/22/2010 |