Read Raw SeaExplorer Data

Reads raw CSV files produced by a SeaExplorer glider. This function can output either "Level 0" or "Level 1" type data. Level 0 is simply the raw data as written in the CSV files with no processing done except renaming variables to the oce convention (e.g. "GPCTD_CONDUCTIVITY" is renamed as "conductivity").

Usage

read.glider.seaexplorer.raw(
  directory,
  pattern = "pld1.raw",
  yo,
  level = 1,
  interpolateToCTD = TRUE,
  rename = TRUE,
  progressBar = interactive(),
  missingValue = 9999,
  debug = getOption("gliderDebug", default = 0)
)

Arguments

directory

The directory in which the delayed-mode SeaExplorer files are located.

pattern

a character value used to find files to read. The default, "pld1", will match files with that string in the name. The best choices for this depends on the setup of files. The search for files is done with list.files(directory, pattern), and so it can be a good idea to try that first, to learn how to specify the desired files. Also, try calling with debug=1 to get some indication of the files that get read or with debug=2 to get a full listing.

yo

A numeric value (or vector) specifying the yo numbers to read. If this is not provided, read.glider.seaexplorer.raw() will read all yo numbers for which files are present in dir.

level

A numeric value specifying the processing level, 0 or

See Details.

interpolateToCTD

A logical indicating whether all sensors should be interpolated to the CTD times to obtain a common time base, or whether all sensors should simply be interpolated for all time stamps (which was the default behaviour before 2019-12-08)

rename

an indication of how (or if) to rename variables. This is needed for most practical work in the oce package, which expects standardized names, such as "temperature", as opposed to the names stored in glider files. There are three choices for rename. (a) It can be logical, with TRUE (the default) meaning to use names as defined in system.file("extdata/dictionaries/seaexplorerDict.csv",package="oceglider") or FALSE, meaning not to rename variables. (b) It can be the name of a CSV file that is in the same format as the file above-named file. (c) It can be a data frame with columns named gliderName and oceName.

progressBar

a logical value that controls whether to indicate the progress made in reading and interpreting the data. This can be useful, since the work can be slow. The default is to show progress in interactive sessions, but not in scripts.

missingValue

numeric value that indicates bad data. Any data items equalling this value are converted to NA. The default is 9999. To avoid changing values to NA, call the function with missingValue=NULL.

debug

an integer specifying whether debugging information is to be printed during processing. If this is not provided, then the value of getOption("gliderDebug",0) is used. The printing is done by a call to gliderDebug. Setting debug=0 turns off this form of debugging, while higher values may yield more information, depending on the function. If one glider function calls another, it passes the value of debug but decreased by 1, which means that the value of debug controls not just the breadth of debugging, but also the depth.

Details

Level 1 processing performs a number of steps to give an "analysis ready" dataset, including

Interpolation of the surface longitude and latitude to give an estimate of the subsurface positions. This is a crude estimate of subsurface location and should be taken only as a first guess.
Removal of the first few sensor values from when the glider is in navState=118 (inflecting up) or navState=110 (inflecting down). The reason for this is that when the glider is set to sample on alternating profiles, when the CTD is powered up the first sample output to the payload computer is the last sample recorded before power down.
Interpolation, depending on the value of interpolateToCTD. If interpolateToCTD is TRUE, then any "extra" sensors are interpolated to the times for which there is CTD data. Otherwise, NAs for all the sensors are interpolated to a common time, corresponding to the raw time stamps output from the various sensors. A caution – this will produce an apparent "upsampling" of each sensor, so that the apparent sample rate is higher. For example, if a Wetlabs FLBBCD sensor sampled, but there is no corresponding GP-CTD sample from the same time, the CTD parameters will be interpolated from the ones before and after. This has the disadvantage of interpolating values that were not measured, but has the advantage of assigning pressures to values measured by sensors not integrated into the CTD (e.g. Wetlabs FLBBCD, Rinko O2). Following the interpolation, any rows with duplicated times are removed.
Calculate Practical salinity from conductivity, temperature and pressure using oce::swSCTp().

In any case, a flag scheme is set up according to the IOOS classification system (see Table 2 of reference 1), as follows.

Name	Value	IOOS Name	Description
`pass`	1	Pass	Data has passed quality control (QC) tests
`not_evaluated`	2	Not Evaluated	Data has not been QC tested
`suspect`	3	Suspect or of High Interest	Data is considered to be of suspect or high interest
`fail`	4	Fail	Data is considered to have failed on one or more QC tests
`missing`	9	Missing Data	Data are missing; using a placeholder

Historical notes

Until package varsion 0.1-14, released on 2025-02-10, longitude and latitude were interpolated between surface values for the case where level is given as 0. This behaviour was changed for issue 127, at https://github.com/dankelley/oceglider/issues/127).
Until package varsion 0.1-16, released on 2025-02-15, data could be erased for a while after power-on events, as controlled by a parameter named removeTimeSincePowerOn. Now, a similar action may be accomplished by calling deleteStartupData() on the return object from the present function.

Renaming of data in seaexplorer files

FIXME: are the original names the same in both "raw" and "sub" datasets?

Data in the gli files are stored in the glider item within the data slot of the returned object, renamed as follows. (Note that there is also an empty column in the seaexplorer data files, caused by a semicolon at the ends of the lines. This is read by R, but then discarded and not stored in the glider object.)

Data in the gli files are stored in the glider item within the data slot of the returned object, renamed as follows. (If the new name is listed as . that means that the old name is retained, but bear in mind that a new name will likely be assigned at some later point in the development of this package.)

OriginalName	Name	Notes
`Timestamp`	`time`	Converted to POSIXt time.
`NavState`	`navState`	-
`SecurityLevel`	`alarm`	-
`Heading`	`heading`	-
`Pitch`	`pitch`	-
`Roll`	`roll`	-
`Depth`	`pressureNav`	-
`Temperature`	`temperatureInternal`	-
`Pa`	`pressureInternal`	-
`Lat`	`latitude`	Converted to decimal degrees.
`Lon`	`longitude`	Converted to decimal degrees.
`DesiredH`	`headingDesired`	-
`BallastCmd`	`ballastCmd`	-
`BallastPos`	`ballastPos`	-
`LinCmd`	`linCmd`	-
`LinPos`	`linPos`	-
`AngCmd`	`angCmd`	-
`AngPos`	`angPos`	-
`Voltage`	`voltage`	-
`Altitude`	`altitude`	-

Data in the pld1 files are stored in the payload1 item within the data slot of the returned object, renamed as follows. Where possible, the corresponding IOOS NetCDF file variable names are also listed [see Integrated Ocean Observing System (U.S.). “NGDAC NetCDF File Format Version 2,” March 27, 2019. https://github.com/ioos/ioosngdac/wiki/NGDAC-NetCDF-File-Format-Version-2].

OriginalName	Name	IOOS name	Notes
`PLD_REALTIMECLOCK`	`time`	`time`	Converted to POSIXt time.
`NAV_RESOURCE`	`navState`	-	-
`NAV_LONGITUDE`	`longitude`	`lon`	Converted to decimal degrees.
`NAV_LATITUDE`	`latitude`	`lat`	Converted to decimal degrees.
`NAV_DEPTH`	`pressureNav`	`depth`	-
`FLBBCD_CHL_COUNT`	`chlorophyllCount`	-	-
`FLBBCD_CHL_SCALED`	`chlorophyll`	-	-
`FLBBCD_BB_700_COUNT`	`backscatterCount`	-	-
`FLBBCD_BB_700_SCALED`	`backscatter`	-	-
`FLBBCD_CDOM_COUNT`	`cdomCount`	-	-
`FLBBCD_CDOM_SCALED`	`cdom`	-	-
`GPCTD_CONDUCTIVITY`	`conductivity`	-	-
`GPCTD_TEMPERATURE`	`temperature`	`temperature`	degC
`GPCTD_PRESSURE`	`pressure`	`pressure`	dbar
`GPCTD_DOF`	`oxygenFrequency`	-	-
-	`salinity`	`salinity`	Practical Salinity (computed)

Data in the pld2 files (or others for additional payloads) are ignored in this version of the package. Please contact the authors, if you need to handle such files.

History of changes to this function

In version 0.1.16, the removeTimeSincePowerOn parameter was removed. The new scheme is for the user to call deleteStartupData() on the return value, to accomplish a similar thing.

Author

Clark Richards, Chantelle Layton and Dan Kelley

Examples

library(oceglider)

directory <- system.file("extdata/sea_explorer/delayed_raw",
    package =
        "oceglider"
)
g <- read.glider.seaexplorer.raw(directory, progressBar = FALSE)
plot(g, which = "p")