Help: noise-pdf-browser v.1

Description

• Overview of Service
• Intervals
• /gallery method
• /breakout method
• /spectrogram method
  • Plot date range selection and display
  • Plot Dimensions
  • Plot Color
  • Plot Power Range
  • Differencing power levels to noise models and median values
  • Custom Noise models
• /summary method
• /availability method
  • Using /availability method to automate data download
• target selection
  • Target selection for /breakout method
  • Target selection for /gallery and /summary methods
  • Wild carding and CSV lists
  • Automatic wild carding
• Simultaneous Request Limitations, Lazy Image Loading and Printing)
• See Also

Overview of Service

The noise-pdf-browser web service provides a calendar centric view into the Probability Density Function (PDF) information generated by the mustang system. Along with allowing for the discovery of what PDF calculations have been made, the service makes it possible to quickly discover instrument noise characteristics problems and to pinpoint the dates at which these problems have happened.

Intervals

The /gallery and /breakout methods use date range intervals. Five types of intervals are supported:

• all
• year
• month
• week
• day

The all interval represent the entire time range of PDF measurements for any given target. The year interval represents regular calendar years. The month interval represents regular calendar months. The week interval represents weeks running Sunday through Saturday. The day interval represent regular days. All dates are GMT.

The interval parameter is used to select the desired calendar interval type; for example interval=month.

/gallery

The /gallery method displays a grid of PDF images. The horizontal rows are selected targets. The vertical columns are calendar intervals. For example, if interval=year is selected, the columns might be the years 2001, 2002, 2003 and 2004. If no interval parameter is specified, interval=year is used.

The starttime, endtime and maxintervals parameters select which years, months, weeks or days are displayed.

If interval=all is chosen, rows are grouped by the first three letters of the channel name.

If interval=all is chosen, the maxintervals parameter is ignored. The starttime and endtime parameters can be used to time-window the displayed data.

starttime not specified. endtime not specified:

The most recent intervals, for which there is data available for the selected targets, are displayed. 
If the maxinterval parameter is not specified, at most 4 intervals are displayed. 
Example:
   If the most recent PDF data available was in the year 2016 and interval=year&maxintervals=5 were selected, 
   the years 2012, 2013, 2014, 2015, 2016 would be shown.

starttime specified. endtime not specified:

The earliest intervals, going back in time to the given starttime, for which there is data available 
for the selected targets, are displayed. If the maxinterval parameter is not specified, at most 4 
intervals are displayed.
Example:
   If PDF data is available from 2000 to 2010 and interval=year&maxintervals=3&starttime=2001-01-01 
   were selected, the years 2001, 2002, 2003 would be shown.

starttime not specified. endtime specified:

The most recent intervals, going forward in time to the given endtime for which there is data available 
for the selected targets, are displayed. If the maxinterval parameter is not specified, at most 4 intervals are displayed.
Example:
   If PDF data is available from 2000 to 2010 and interval=year&maxintervals=3&endtime=2009-01-01 
   were selected, the years 2006, 2007, 2008 would be shown.

starttime specified. endtime specified:

All intervals overlapping (either partially or completely) by the specified time range are displayed. 
maxintervals cannot be specified.
Example:
   If PDF data is available from 2000 to 2010 and ...@interval=year&starttime=2001-01-01&endtime=2004-06-01@... 
   were selected, the years 2001, 2002, 2003, 2004 would be shown.

Important Note On Interval Selection
The starttime and endtime parameters select which calendar intervals are displayed. They do not clip those intervals. The intervals which are selected are those that overlap with the selected data range; starttime inclusive, endtime exclusive. However, this behavior is different for interval=all. For interval=all the starttime and endtime clip the display data.

Assuming data is available, the following table illustrates what years would be returned for the given parameters:

/breakout

The /breakout method displays one main interval and the sub-intervals which cover it.

The selected main calendar interval is determined by the time and interval parameters. The time parameter defaults to the current time.
The selected calendar interval will include the given time (start inclusive, end exclusive).

For example: both interval=year&time=2001-01-01 and interval=year&time=2001-11-11 will select the year 2001

/spectrogram

The /spectrogram method displays spectrogram images for the selected targets and time ranges.

Plot date range selection and display

For any given request, the generated spectrogram plots will all have the same date ranges. If start[time] and/or end[time] parameters are specified, the generated plots will reflect those date range limits. If start[time] and/or end[time] parameters are not specified, the time ranges of the generated plots will be determined by the earliest and/or latest dates of measurements available for the selected targets.

Example (start and end specified): …start=2010-01-01&end=2015-01-01…
Example: (start specified)…start=2010-01-01…
Example: (end specified)…end=2015-01-01…

Plot Dimensions

By default, displayed spectrograms will have dimensions of 1000 × 500 pixels. The imgwidth and imgheight parameters allow this to be customized. They are passed to the spectrogram service as its plot.width and plot.height parameters

Example: …imgwidth=700&imgheight=200…

Plot Color

By default, a (non-inverted) rainbow scheme is used to color the spectrograms and power scales. The color.palette and color.invert options allow these attributes to be customized, and are passed implicitely to the spectrogram service as its plot.color.palette and plot.color.invert parameters. Colorblind-friendly and grayscale-compatible palettes are also provided. See help for more information on the different options available.

Example: …color.palette=RdYlBu&color.invert=true…

Plot Power Range

By default, displayed spectrograms will have auto generated power ranges. Although this is useful for identifying visual patterns in the spectrograms, it can be useful to have all displayed spectrograms use the same power range. This is accomplished with the powerrange parameter. The powerrange parameter is passed to the spectrogram service as its plot.powerscale.range parameter. It takes two comma-separated integers. The order of the numbers does not matter.

Example: …powerrange=-180,-110…

It is also possible to adjust how power ranges are auto generated. This is accomplished with the autorange parameter. It takes a number greater than zero (0) and less than or equal to one (1). The number determines which percentile values of range will be used. The default value is 0.95. If a value of 1.0 is selected, the full power range of the spectrograms will be displayed. This generally will have a washed out appearance as most spectrograms contain a few extreme outliers.

Example: …autorange=1.0…
Example: …autorange=0.8…

Differencing power levels to noise models and median values

The output parameter, which is passed directly to the spectrogram service determines what the plots display.

The default output value is power. With this value powers are directly displayed.

Example: …output=power…

output=powerdlnm will difference the power values to the Peterson low noise model.

Example: …output=powerdlnm…

output=powerdhnm will difference the power values to the Peterson high noise model.

Example: …output=powerdhnm…

output=powerdmedian will difference the power values to the median power levels per frequency for the selected date range per target. This is especially useful for highlighting noise level changes.

Example: …output=powerdmedian…

Custom Noise models

The noisemodel.byfrequency and noisemodel.byperiod parameters make it possible to specify custom noise models for noise level comparison. The values associated with these parameters will be directly passed to the spectrogram service via identically named parameters. See Spectrogram Noise Model Comparison for more information.

Please note that the noisemodel.byfrequency and noisemodel.byperiod parameters will only effect displayed plots when output=powerdhnm, output=powerdlnm or output=powerdnm is specified.

Custom noise models can be generated using the noise-pdf webservice. See PDF Noise Profile Options

/summary

The /summary method displays a list of available targets and the date extents of available PDF measurements. It provides hyper-links to the /breakout view with interval=all selected.

/availability

The /availability method returns CSV (comma separated value) lists of targets and time ranges. It is similar to the /summary method, but allows for date selection and for calandar time interval output (DAY, WEEK, MONTH, YEAR). The /availability method is especially useful for automating PDF and PSD downloads.

With no time options specified, the service returns the extents of data availability.

Example:

../availability?target=IU.ANMO.00.*.M

IU.ANMO.00.BH1.M,1998-10-26,2019-05-12
IU.ANMO.00.BH2.M,1998-10-26,2019-05-12
IU.ANMO.00.BHZ.M,1998-10-26,2019-05-12
IU.ANMO.00.HH1.M,2018-07-09,2019-05-01
IU.ANMO.00.HH2.M,2018-07-09,2019-05-01
IU.ANMO.00.HHZ.M,2018-07-09,2019-05-01
IU.ANMO.00.LH1.M,1998-10-26,2019-05-12
IU.ANMO.00.LH2.M,1998-10-26,2019-05-12
IU.ANMO.00.LHZ.M,1998-10-26,2019-05-12

With the interval option the service breaks the availability down to the year, month, week or day

Example:

../availability?target=IU.ANMO.00.BHZ.M&interval=year

IU.ANMO.00.BHZ.M,1999-01-01,2000-01-01
IU.ANMO.00.BHZ.M,2000-01-01,2001-01-01
IU.ANMO.00.BHZ.M,2001-01-01,2002-01-01
IU.ANMO.00.BHZ.M,2002-01-01,2003-01-01
IU.ANMO.00.BHZ.M,2003-01-01,2004-01-01
IU.ANMO.00.BHZ.M,2004-01-01,2005-01-01
IU.ANMO.00.BHZ.M,2005-01-01,2006-01-01
IU.ANMO.00.BHZ.M,2006-01-01,2007-01-01
IU.ANMO.00.BHZ.M,2007-01-01,2008-01-01
IU.ANMO.00.BHZ.M,2008-01-01,2009-01-01
IU.ANMO.00.BHZ.M,2009-01-01,2010-01-01
IU.ANMO.00.BHZ.M,2010-01-01,2011-01-01
IU.ANMO.00.BHZ.M,2011-01-01,2012-01-01
IU.ANMO.00.BHZ.M,2012-01-01,2013-01-01
IU.ANMO.00.BHZ.M,2013-01-01,2014-01-01
IU.ANMO.00.BHZ.M,2014-01-01,2015-01-01
IU.ANMO.00.BHZ.M,2015-01-01,2016-01-01
IU.ANMO.00.BHZ.M,2016-01-01,2017-01-01
IU.ANMO.00.BHZ.M,2017-01-01,2018-01-01
IU.ANMO.00.BHZ.M,2018-01-01,2019-01-01
IU.ANMO.00.BHZ.M,2019-01-01,2020-01-01

With the start and end options the availability information can be limited to a given time epoch.

Example: IU BHZ data for the year 1998

../availability?target=target=IU...BHZ.M&start=1998-01-01&end=1999-01-01

IU.ADK..BHZ.M,1998-01-01,1999-01-01
IU.AFI..BHZ.M,1998-01-09,1999-01-01
IU.ANMO..BHZ.M,1998-01-01,1998-10-27
IU.ANMO.00.BHZ.M,1998-10-26,1999-01-01
IU.ANMO.10.BHZ.M,1998-10-26,1999-01-01
IU.ANTO..BHZ.M,1998-01-05,1998-06-09
IU.BILL..BHZ.M,1998-01-01,1999-01-01
IU.CASY..BHZ.M,1998-01-01,1999-01-01
...

Using /availability method to automate PSD, PDF, downloads

Combined with simple UNIX shell scripts, the /availability service makes it easy to download large batches of PSD, PDF, Spectrogram and daily noise-mode timeseries data.

For example, suppose you wanted to download all of the BH channel PSDs originating from the IU, ANMO station for the years 2005 and 2006. The PSD service limits how many individual PSDs can be returned in a single request call. The limit is around a year’s worth of data for one channel. Making month sized requests is more efficient due to memory requirements on the backend servers.

In this illustrative example we make use of the wget and awk utilities to download PSDs in month long batches.

Step 1: Download a list of months per channel and save to file

$ wget -O ANMO_BHZ_2005_to_2007.txt "https://service.earthscope.org/mustang/noise-pdf-browser/1/availability?network=IU&station=ANMO&channel=BH?&start=2005-01-01&end=2007-01-01&interval=month"
$ head ANMO_BHZ_2005_to_2007.txt
IU.ANMO.00.BH1.M,2005-01-01,2005-02-01
IU.ANMO.00.BH1.M,2005-02-01,2005-03-01
IU.ANMO.00.BH1.M,2005-03-01,2005-04-01
IU.ANMO.00.BH1.M,2005-04-01,2005-05-01
IU.ANMO.00.BH1.M,2005-05-01,2005-06-01
IU.ANMO.00.BH1.M,2005-06-01,2005-07-01
IU.ANMO.00.BH1.M,2005-07-01,2005-08-01
IU.ANMO.00.BH1.M,2005-08-01,2005-09-01
IU.ANMO.00.BH1.M,2005-09-01,2005-10-01
IU.ANMO.00.BH1.M,2005-10-01,2005-11-01

Step 2: Use unix awk command and convert list into download script

$ awk -F "," '{print "wget -O "$1"-"$2".xml \"http://service.iris.edu/mustang/noise-psd/1/query?target="$1"&start="$2"&end="$3"&format=xml\""}' ANMO_BHZ_2005_to_2007.txt > fetch_psds.sh
$ head fetch_psds.sh
wget -O IU.ANMO.00.BH1.M-2005-01-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-01-01&end=2005-02-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-02-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-02-01&end=2005-03-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-03-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-03-01&end=2005-04-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-04-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-04-01&end=2005-05-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-05-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-05-01&end=2005-06-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-06-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-06-01&end=2005-07-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-07-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-07-01&end=2005-08-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-08-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-08-01&end=2005-09-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-09-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-09-01&end=2005-10-01&format=xml"
wget -O IU.ANMO.00.BH1.M-2005-10-01.xml "https://service.earthscope.org/mustang/noise-psd/1/query?target=IU.ANMO.00.BH1.M&start=2005-10-01&end=2005-11-01&format=xml"

Step 3: Execute the fetch_psds.sh script and download the xml files

$ . fetch_psds.sh
$ ls
IU.ANMO.00.BH1.M-2005-01-01.xml	IU.ANMO.00.BH2.M-2006-02-01.xml	
IU.ANMO.10.BH1.M-2005-03-01.xml	IU.ANMO.10.BH2.M-2006-04-01.xml
IU.ANMO.00.BH1.M-2005-02-01.xml	IU.ANMO.00.BH2.M-2006-03-01.xml	
IU.ANMO.10.BH1.M-2005-04-01.xml	IU.ANMO.10.BH2.M-2006-05-01.xml .....

In this example PSDs were downloaded. Similar techniques can be applied to the other Mustang noise services

• noise-pdf
• noise-spectrogram
• noise-mode-timeseries

target selection

There are two ways to select targets.

• Use the target variable
• Use network, station, location, channel, quality variables

Target selection for /breakout method

The /breakout method allows for only one target to be selected. Therefore, wildcarding is not allowed.

Two ways to select the same target in the breakout method:

• …/breakout?target=IU.ANMO.00.BHZ.M
• …/breakout?network=IU&station=ANMO&location=00&channel=BHZ&quality=M

Target selection for /gallery and /summary methods

Target selection is flexible. Any combination of the target, network, station, location, channel and quality parameters can be used.

The /gallery and /summary methods allow for multiple targets to be selected.

Wild carding and CSV lists:

The target and network, station, location, channel and quality parameters take glob style wild carding characters * and ? as well as comma separated values CSV list.

Examples:

• select all channels starting with BH: channel=BH? …/gallery?network=IU&station=ANMO&location=00&channel=BH?
• select BH1 BH2 and BHZ channels: channel=BH1,BH2,BHZ …/gallery?network=IU&station=ANMO&location=00&channel=BH1,BH2,BHZ
• select two targets: target=IU.ANMO.00.BHZ.M,IU.ADK.10.BH1.M …/gallery?target=IU.ANMO.00.BHZ.M,IU.ADK.10.BH1.M

Automatic wild carding:

If any of the network, station, location, channel and quality parameters are selected, any of the non-selected parameters will be wild carded.

For example network=IU&channel=BHZ will select all of the targets with channel names BHZ in the IU network:

…/gallery?network=IU&channel=BHZ

Simultaneous Request Limitations, Lazy Image Loading and Printing

If the Mustang PDF plotting service (https://service.earthscope.org/mustang/noise-pdf/1/) receives too many simultaneous requests from any given client, the offending client will be shut out for 60 seconds. The /gallery and /breakout services are capable of producing HTML output with image links to this service that will cause web-browsers to reach this limit. In order to prevent this from happening, the HTML output made by the /gallery and /breakout services uses a Javascript lazy-image-loading library. This library causes images to not be loaded until they are scrolled into view thus avoiding the simultaneous request limit.

When printing from a web browser, it may be necessary to scroll all of the images into view before printing.

See Also:

• Usage guidelines for IRIS DMC services
• IRIS DMC MUSTANG noise-pdf Web Service Documentation
• Data Services Newsletter Vol 18 No 1: Mustang PDF/PSD Noise Analysis System
• IRIS DMC MUSTANG noise-spectrogram Web Service Documentation

---

Problems with this service?