This R package is designed to obtain USGS or EPA water quality sample data, streamflow data, and metadata directly from web services. See: http://usgs-r.github.io/dataRetrieval/
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/",
warning = FALSE,
message = FALSE,
fig.height = 7,
fig.width = 7
)
library(dataRetrieval)
The dataRetrieval
package was created to simplify the process of loading hydrologic data into the R environment. It is designed to retrieve the major data types of U.S. Geological Survey (USGS) hydrology data that are available on the Web, as well as data from the Water Quality Portal (WQP), which currently houses water quality data from the Environmental Protection Agency (EPA), U.S. Department of Agriculture (USDA), and USGS.
⚠️ NWIS web services (readNWIS functions) will be replaced over time by USGS Water Data APIs (read_waterdata_). Read more about it here:
https://doi-usgs.github.io/dataRetrieval/articles/read_waterdata_functions.html
⚠️ USGS discrete water-quality data availability and format are changing. To view the status of changes in data availability and code functionality, visit: https://doi-usgs.github.io/dataRetrieval/articles/Status.html
If you have additional questions about these changes, email [email protected].
Get instantaneous USGS data (for example, discharge sensor data). Start here: ?readNWISuv
Get daily USGS data (for example, mean daily discharge). Start here: ?read_waterdata_daily
Get USGS groundwater data. Start here: ?readNWISgwl
Get discrete water quality data from a cooperative service that integrates publicly available water-quality data from the USGS, EPA, and over 400 state, federal, tribal, and local agencies. Start here: ?readWQPdata
Get USGS discrete water quality data. Start here: ?read_waterdata_samples
Get metadata about USGS time series data, including instantaneous and daily data. Start here: ?read_waterdata_ts_meta
Find Hydro Network-Linked Data Index (NLDI) data. Start here: ?findNLDI
For additional tutorials, see:
To install the dataRetrieval
package, you must be using R 3.0 or greater and run the following command:
install.packages("dataRetrieval")
The USGS is planning to modernize all web services in the near future. For each of these updates, dataRetrieval
will create a new function to access the new services.
You can register an API key for use with USGS water data APIs. There are now limits on how many queries can be requested per IP address per hour. If you find yourself running into limits, you can request an API token here: https://api.waterdata.usgs.gov/signup/
Then save your token in your .Renviron file like this:
API_USGS_PAT = "my_super_secret_token"
You can use usethis::edit_r_environ()
to edit find and open your .Renviron file. You will need to restart R for that variable to be recognized.
New functions will use a “snake case”, such as “read_waterdata_samples”. Older functions use camel case, such as “readNWISdv”. The difference is the underscore between words. This should be a handy way to tell the difference between newer modern data access, and the older traditional functions.
Historically, we allowed users to customize their queries via the ...
argument structure. With ...
, users needed to know the exact names of query parameters before using the function. Now, the new functions will include ALL possible arguments that the web service APIs support. This will allow users to use tab-autocompletes (available in RStudio and other IDEs). Users will need to understand that it is not advisable to specify all of these parameters. The systems can get bogged down with redundant query parameters. We expect this will be easier for users, but it might take some time to smooth out the documentation and test usability. There may be additional consequences, such as users won’t be able to build up argument lists to pass into the function.
As we develop functions for the modern USGS Water Data APIs, we’ll continue to explore updating package dependencies.
CRAN-stable documentation will be available on the GitHub pages:
https://doi-usgs.github.io/dataRetrieval/
In-development documentation will be available on the USGS GitLab pages:
https://water.code-pages.usgs.gov/dataRetrieval
Development of dataRetrieval
will happen on a git branch called “develop”. The “develop” branch will only move to the “main” branch when we submit to CRAN, unless there are bug fixes that pertain to the CRAN release. The “develop” branch WILL change frequently, and there are no promises of future behavior. Users must accept that they are using those functions at their own risk. If you willing to accept this risk, the installation instructions are:
library(remotes)
install_github("DOI-USGS/dataRetrieval",
ref = "develop")
Please consider reporting bugs and asking questions on the Issues page:
https://github.com/DOI-USGS/dataRetrieval/issues
citation(package = "dataRetrieval")
U.S. Geological Survey, 2023, National Water Information System data available on the World Wide Web (USGS Water Data for the Nation), accessed [April 26, 2023], at http://waterdata.usgs.gov/nwis/. http://dx.doi.org/10.5066/F7P55KJN
This can be done using the create_NWIS_bib
function:
dv <- readNWISdv("09010500", "00060")
NWIScitation <- create_NWIS_bib(dv)
NWIScitation
print(NWIScitation, style = "Bibtex")
Citations for specific datasets should use this format:
National Water Quality Monitoring Council, YYYY, Water Quality Portal, accessed mm, dd, yyyy, hyperlink_for_query, https://doi.org/10.5066/P9QRKUVJ.
This can be done using the create_WQP_bib
function:
SC <- readWQPqw(siteNumbers = "USGS-05288705",
parameterCd = "00300")
WQPcitation <- create_WQP_bib(SC)
WQPcitation
print(WQPcitation, style = "Bibtex")
General Water Quality Portal citations should use the following:
Water Quality Portal. Washington (DC): National Water Quality Monitoring Council, United States Geological Survey (USGS), Environmental Protection Agency (EPA); 2021. https://doi.org/10.5066/P9QRKUVJ.
The Water Mission Area of the USGS supports the development and maintenance of dataRetrieval
, and most likely further into the future. Resources are available primarily for maintenance and responding to user questions. Priorities on the development of new features are determined by the dataRetrieval
development team.