Read Delayed-Mode SeaExplorer Glider Data

Reads delayed-mode CSV files produced by a SeaExplorer glider, as detected by the presence of ".raw." in their names. Such delayed-mode data are the full resolution data stored on the glider and downloaded after recovery. (Use read.glider.seaexplorer.realtime() instead of this, to read data as transmitted by the glider while it is in the field.)

Usage

read.glider.seaexplorer.delayed(
  directory,
  yo,
  level = 1,
  interpolateToCTD = TRUE,
  removeTimeSincePowerOn = 0,
  progressBar = interactive(),
  debug = getOption("gliderDebug", default = 0)
)

Arguments

directory: The directory in which the delayed-mode SeaExplorer files are located.
yo: A numeric value (or vector) specifying the yo numbers to read. If this is not provided, read.glider.seaexplorer.delayed() will read all yo numbers for which files are present in dir.
level: A numeric value specifying the processing level, 0 or 1. See Details.
interpolateToCTD: A logical indicating whether all sensors should be interpolated to the CTD times to obtain a common time base, or whether all sensors should simply be interpolated for all time stamps (which was the default behaviour before 2019-12-08)
removeTimeSincePowerOn: numeric value indicating the number of seconds of data to trim, after the glider sensors are powered on. One way to determine this is to read the whole sequence, and then plot say the first 10 minutes of salinity and temperature data, looking for a transition between aphysical values, which might take the form of zero salinity, followed by a relatively rapid ramp-up to values that seem more oceanographic.
progressBar: either a logical or character value that controls whether/how to indicate the progress made in reading and interpreting the data. This can be useful, since the work can be slow. If progressBar is a logical value, then it indicates whether to show textual progress with txtProgressBar(). If progressBar is the character value "shiny", then shiny::setProgress() and shiny::incProgress() will be used, on the assumption that the call to read.glider.seaexplorer.realtime() was made within the context of a call to shiny::withProgress(). The default is to use the value returned by interactive(), i.e. to use a textual progress indicator, but only in interactive mode.
debug: an integer specifying whether debugging information is to be printed during processing. If this is not provided, then the value of getOption("gliderDebug",0) is used. The printing is done by a call to gliderDebug. Setting debug=0 turns off this form of debugging, while higher values may yield more information, depending on the function. If one glider function calls another, it passes the value of debug but decreased by 1, which means that the value of debug controls not just the breadth of debugging, but also the depth.

Details

This function can output either "Level 0" or "Level 1" type data. Level 0 is simply the raw data as written in the CSV files with no processing done, other than to remove longitude and latitude for samples where the glider wasn't actually communicating with satellites and then interpolate between surface values.

Level 1 processing performs a number of steps to give an "analysis ready" dataset, including

Interpolation of the surface longitude and latitude to give an estimate of the subsurface positions. This is a crude estimate of subsurface location and should be taken only as a first guess.
Removal of the first few sensor values from when the glider is in navState=118 (inflecting up) or navState=110 (inflecting down). The reason for this is that when the glider is set to sample on alternating profiles, when the CTD is powered up the first sample output to the payload computer is the last sample recorded before power down.
Interpolation, depending on the value of interpolateToCTD. If interpolateToCTD is TRUE, then any "extra" sensors are interpolated to the times for which there is CTD data. Otherwise, NAs for all the sensors are interpolated to a common time, corresponding to the raw time stamps output from the various sensors. A caution – this will produce an apparent "upsampling" of each sensor, so that the apparent sample rate is higher. For example, if a Wetlabs FLBBCD sensor sampled, but there is no corresponding GP-CTD sample from the same time, the CTD parameters will be interpolated from the ones before and after. This has the disadvantage of interpolating values that were not measured, but has the advantage of assigning pressures to values measured by sensors not integrated into the CTD (e.g. Wetlabs FLBBCD, Rinko O2). Following the interpolation, any rows with duplicated times are removed.
Calculate Practical salinity from conductivity, temperature and pressure using oce::swSCTp().

In any case, a flag scheme is set up according to the IOOS classification system (see Table 2 of reference 1), as follows.

Name	Value	IOOS Name	Description
`pass`	1	Pass	Data has passed quality control (QC) tests
`not_evaluated`	2	Not Evaluated	Data has not been QC tested
`suspect`	3	Suspect or of High Interest	Data is considered to be of suspect or high interest
`fail`	4	Fail	Data is considered to have failed on one or more QC tests
`missing`	9	Missing Data	Data are missing; using a placeholder

Renaming of data in seaexplorer files

FIXME: are the original names the same in both "raw" and "sub" datasets?

Data in the gli files are stored in the glider item within the data slot of the returned object, renamed as follows. (Note that there is also an empty column in the seaexplorer data files, caused by a semicolon at the ends of the lines. This is read by R, but then discarded and not stored in the glider object.)

Data in the gli files are stored in the glider item within the data slot of the returned object, renamed as follows. (If the new name is listed as . that means that the old name is retained, but bear in mind that a new name will likely be assigned at some later point in the development of this package.)

OriginalName	Name	Notes
`Timestamp`	`time`	Converted to POSIXt time.
`NavState`	`navState`	-
`SecurityLevel`	`alarm`	-
`Heading`	`heading`	-
`Pitch`	`pitch`	-
`Roll`	`roll`	-
`Depth`	`pressureNav`	-
`Temperature`	`temperatureInternal`	-
`Pa`	`pressureInternal`	-
`Lat`	`latitude`	Converted to decimal degrees.
`Lon`	`longitude`	Converted to decimal degrees.
`DesiredH`	`headingDesired`	-
`BallastCmd`	`ballastCmd`	-
`BallastPos`	`ballastPos`	-
`LinCmd`	`linCmd`	-
`LinPos`	`linPos`	-
`AngCmd`	`angCmd`	-
`AngPos`	`angPos`	-
`Voltage`	`voltage`	-
`Altitude`	`altitude`	-

Data in the pld1 files are stored in the payload1 item within the data slot of the returned object, renamed as follows. Where possible, the corresponding IOOS NetCDF file variable names are also listed [see Integrated Ocean Observing System (U.S.). “NGDAC NetCDF File Format Version 2,” March 27, 2019. https://github.com/ioos/ioosngdac/wiki/NGDAC-NetCDF-File-Format-Version-2].

OriginalName	Name	IOOS name	Notes
`PLD_REALTIMECLOCK`	`time`	`time`	Converted to POSIXt time.
`NAV_RESOURCE`	`navState`	-	-
`NAV_LONGITUDE`	`longitude`	`lon`	Converted to decimal degrees.
`NAV_LATITUDE`	`latitude`	`lat`	Converted to decimal degrees.
`NAV_DEPTH`	`pressureNav`	`depth`	-
`FLBBCD_CHL_COUNT`	`chlorophyllCount`	-	-
`FLBBCD_CHL_SCALED`	`chlorophyll`	-	-
`FLBBCD_BB_700_COUNT`	`backscatterCount`	-	-
`FLBBCD_BB_700_SCALED`	`backscatter`	-	-
`FLBBCD_CDOM_COUNT`	`cdomCount`	-	-
`FLBBCD_CDOM_SCALED`	`cdom`	-	-
`GPCTD_CONDUCTIVITY`	`conductivity`	-	-
`GPCTD_TEMPERATURE`	`temperature`	`temperature`	degC
`GPCTD_PRESSURE`	`pressure`	`pressure`	dbar
`GPCTD_DOF`	`oxygenFrequency`	-	-
-	`salinity`	`salinity`	Practical Salinity (computed)

Data in the pld2 files (or others for additional payloads) are ignored in this version of the package. Please contact the authors, if you need to handle such files.

Author

Clark Richards and Dan Kelley

Examples

library(oceglider)
directory <- system.file("extdata/sea_explorer/delayed_raw", package = "oceglider")
g <- read.glider.seaexplorer.delayed(directory, progressBar = FALSE)
plot(g, which = "p")
#> [1] "2019-07-19 15:12:15 UTC" "2019-07-19 18:02:07 UTC"
#> w: 2549
#> t[w]: 2019-07-19 18:02:07.28
#> p[w]: 3.54