FDSN Dataselect Web Service

    Home » NCEDC Web Services » FDSN Dataselect

Description

The fdsnws-dataselect service provides access to time series data for specified channels and time ranges.

Data are selected using SEED time series identifiers (network, station, location & channel) in addition to time ranges.
Data are returned in miniSEED format.

This service is an implementation of the FDSN web service specification version 1.1.

To retrieve raw waveform data in miniSEED format, submit a request by either of two methods:

	* via HTTP GET: Provide a series of parameter-value pairs in the URL that specify the start-time and end-time, along with the
	  desired network(s), station(s), location(s) and channel(s) . Wildcards are supported.
	* via HTTP POST: Submit a file to the service containing a list of the desired networks, stations, locations, channels,
	  start-times and end-times. This service now supports wildcards. The rest of this page describes the POSTing method in detail.

This service is designed to handle very large data requests and can easily be used with command line programs such as wget, curl
or similiar utilities.

The fdsnws-dataselect web service has all the capabilities and more than the NCEDC's legacy ws-dataselect and
ws-bulkdataselect services.


Data selection

A data selection is composed of a list of network, station, location, channel, start time and end time entries.

An example selection, submitted using HTTP POST, might look like:

BK	CMB	--	BHZ	2010-03-25T00:00:00	2010-04-01T00:00:00
BK	BKS	*	BH?	2010-03-25T00:00:00	2010-04-01T00:00:00
BP	VARB	10	HHZ	2010-03-25T00:00:00	2010-04-01T00:00:00
NC	KLDB	00	BHN	2010-03-25T00:00:00	2010-04-01T00:00:00

Wildcards are allowed in all fields except date fields.


wget example

Requests can be made with a selection file and the wget unix command line utility.

$ cat waveform.request
quality=B
BK CMB -- BHZ 2010-03-25T00:00:00 2010-04-01T00:00:00
BK CMB -- BHE 2010-03-25T00:00:00 2010-04-01T00:00:00

$ wget --post-file=waveform.request -O BK.miniseed https://service.ncedc.org/fdsnws/dataselect/1/query


This will send the request to the server and save the results in a file name BK.miniseed


curl example

Requests can also be made with a selection file and the curl unix command line utility.

$ cat waveform.request
quality=B
BK CMB -- BHZ 2010-03-25T00:00:00 2010-04-01T00:00:00
BK CMB -- BHE 2010-03-25T00:00:00 2010-04-01T00:00:00

$ curl --data-binary @waveform.request -o BK.miniseed https://service.ncedc.org/fdsnws/dataselect/1/query


This will send the request to the server and save the results in a file name BK.miniseed


Considerations

In general, it is preferable to not ask for too much data in a single request. Large requests take longer to complete. If a large request
fails due to any networking issue, it will have to be resubmitted to be completed. This will cause the entire request to be completely
reprocessed and retransmitted. By breaking large requests into smaller requests, only the smaller pieces will need to be resubmitted
and retransmitted if there is a networking problem. Web service network connections will break after 5 to 10 minutes if no data is
transmitted.  For large requests, the fdsnws-dataselect web service can take several minutes before it starts returning data. When
this happens, the web service may flush the http headers with an optimistic success (200) code to the client in order to keep the
network connection alive. This gives about 10 minutes to the underlying data retrieval mechanism to start pulling data out of the
NCEDC archive. Thus for larger requests, the http return code can be unreliable. As data is streamed back to the client, the
fdsnws-dataselect service partially buffers the returned data. During time periods when the underlying retrieval mechanism stalls,
the web service will dribble the partial buffer to the client in an effort to keep the network connection alive.

It is less efficient to ask for too little data in each request. Each time a request is made, a network connection must be established
and a request processing unit started. For performance reasons, it is better to group together selections from the same stations and
place them in the same request. This is especially true of selections that cover the same time periods.