trimming CTD files with shiny
Oct 18, 2015 oce · R
The shiny R library is demonstrated with an application to trim CTD data.

Introduction

The shiny library [1] provides a graphical user interface (GUI) for the R language that may open up new possibilities for human judgement in data filtering, as well as for the use of R by those who choose not to learn its syntax. The first category is the intended audience here, in this first of a series of blog postings about using shiny for Oceanography.

Consider the task of finding the downcast portion of a CTD cast. This must be done because raw CTD data typically include measurements that are of limited value. At the start of a dataset, it is typical to have measurements made during a sensor-equilibration phase, during which the instrument is held just below the water surface for a minute or so. This is followed by a descent through the water column, ideally at almost uniform speed, and, after that, by an ascent phase. In most cases, only the descent phase is of direct interest, so a first step in processing is usually to isolate this phase.

The ctdTrim() function in the oce library [2] often does a good job of locating the descent phase, and trimming data recovered before and after. However, this function is somewhat limited, employing an ad-hoc algorithm that has parameters that were calibrated on a limited dataset, guided by the eye of a single analyst. Since deep-sea CTD casts take an hour or two to acquire (i.e. cost several thousand dollars of ship time), it is entirely reasonable to pay a technician for a minute or two, to check the results, or to supplant them, based on visual inspection of the data.

This suggests that CTD trimming might be good demonstration of shiny. As I’m just learning the system, the methodology is crude. I wanted to learn how to use slider bars, so I use sliderInput() to select the downcast. I wanted to learn how to use a file-choice dialog, so I used file.choose() for that.

I am not going to explain the code in any detail. Readers unfamiliar with R will understand very little, I fear, but my purpose is not to replace the dozens of textbooks and online materials that teach R. Readers unfamiliar with shiny should start by doing. Just copy the code into two files as named below. Then, in rstudio, load one of the files, and click the “runApp” button that you should see. If that button does not appear, or if you’re using something other than rstudio, type following into the R console.

1
2

library(shiny)
runApp() # exit by striking ESC on the keyboard

Below is the contents of the ui.R file

1
2
3
4
5
6
7
8
9

library(shiny)
shinyUI(fluidPage(verticalLayout(
                                 plotOutput("ctdTrimPlot"),
                                 wellPanel(
                                           sliderInput("top", "top fraction percent:",
                                                       min=0, max=100, value=0, step=0.1),
                                           sliderInput("bottom", "bottom fraction percent:",
                                                       min=0, max=100, value=100, step=0.1))
                                 )))

This will look a bit mysterious, but anyone who spends 20 minutes with the shiny documentation will get the gist: two sliders will be shown below a plot that is create with a user-created function named ctdTrimPlot().

Below is the contents of the server.R file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

library(oce)
file <- file.choose()
ctdRaw <- read.oce(file)
## data(ctdRaw) # try this if you have no .CNV files to test
shinyServer(function(input, output) {
            top <- reactive({as.numeric(input$top)})
            bottom <- reactive({as.numeric(input$bottom)})
            trimmed <- ctdRaw
            output$ctdTrimPlot <- renderPlot({
                nf <- layout(matrix(c(1, 2, 3, 4, 4, 4), nrow=2, byrow=TRUE))
                index <- seq_along(ctdRaw[["pressure"]])
                indexStart <- index[1] + 0.01*top() * diff(range(index))
                indexEnd <- index[1] + 0.01*bottom() * diff(range(index))
                trimmed <- ctdTrim(ctdRaw, method="index", parameters=c(indexStart, indexEnd))
                save(trimmed, file="trimmed.rda")
                plotProfile(trimmed, xtype="temperature",
                            mar=c(0.2, 2.2, 2.5, 0.8), mgp=c(1.2, 0.3, 0))
                plotProfile(trimmed, xtype="salinity",
                            mar=c(0.2, 2.2, 2.5, 0.8), mgp=c(1.2, 0.3, 0))
                plotTS(trimmed,
                       mar=c(2.2, 2.2, 1.0, 0.8), mgp=c(1.2, 0.3, 0))
                plotScan(ctdRaw,
                         mar=c(2.5, 2.5, 0.8, 0.8), mgp=c(1.2, 0.3, 0))
                suggested <- range(ctdTrim(ctdRaw)[["scan"]])
                abline(v=suggested, lty=2, col=c('red', 'blue'))
                abline(v=c(indexStart, indexEnd), col=c('red', 'blue'))
                legend("topright", c(suggested[1], indexStart,
                                     suggested[2], indexEnd),
                       col=c("red", "red", "blue", "blue"), 
                       lty=c(2, 1, 2, 1),
                       legend=c("Start (suggested)",
                                sprintf("Start (user): %.0f", indexStart),
                                "End (suggested)",
                                sprintf("End (user): %.0f", indexEnd)))
            }, pointsize=20)
})

Here, notice the use of shinyServer(). Again, this will seem mysterious at first, but a quick glance shows that a major task here is the creation of the ctdTrimPlot() function. (Readers familiar with oce [2] will note that the margins are made very tight here, mainly to devote more space to the data.)

Interested users should simply copy these files, and try them. If there are no .CNV files handy, comment out the file.choose() line and uncomment the data(ctdRaw) line.

Note that the code saves the trimmed data as an rda file in the local directory. A more sophisticated application would use a tailored file name. Another useful addition might be to use mouse drags on the scan-pressure plot, instead of a slider. But these things are for another day. For now, the goal has been met: the reader can see that shiny permits user interaction in a way that is practical, if not elegant. Those who try this in action will find that it is a bit slow, but this is not so much an issue with shiny as it is with plotting. Also, bear in mind the calculation of the cost of acquiring the data … is a 1/4 second lag in an interface an issue for a dataset that cost an hour to acquire that that might yield great benefits to science?

Below is a screenshot of the initial view of the application. Anyone who has looked at CTD data will note the wildly unphysical salinity and temperature characteristics. The dotted lines in the scan-pressure plot show the trimming that ctdTrim() would do, and the solid lines are the values as set at the moment (which are 0 and 100 percent, at the start). The user should adjust those sliders to narrow in on the profile.

ctd_trim_fig_1.png

Below is a screenshot of a view after the downcast has been selected. It should be noted that I selected a different range than was selected automatically by ctdTrim(), because I thought the automatic cutoff at the bottom of the profile came too late, i.e. during a time when the instrument was not moving through the water column.

ctd_trim_fig_2.png

References and resources

  1. Shiny website: http://shiny.rstudio.com

  2. Oce website: https://dankelley.github.io/oce/

  3. Jekyll source code for this blog entry: 2015-10-18-shiny-ctd-trim.Rmd

This website is written in Jekyll, and the source is available on GitHub.