A test of the framework designed to evaluate compliance monitoring devices for ballast water discharge

With the entry of the Ballast Water Management Convention into force, ballast water discharged from ships must meet standards limiting the concentrations of living organisms. Monitoring devices to confirm compliance with these standards would ideally be portable, easy-to-use instruments capable of rapid and accurate shipboard analysis of ballast water. Following a framework established for the validation of such potential devices, six devices were evaluated in a series of laboratory and field tests at three contrasting coastal locations. Devices were designed to quantify organisms in the ≥ 10 and < 50 μm size class. In all cases, the compliance monitoring devices were compared to the agreed-upon performance standard for quantifying living organisms: microscopy-based, vital fluorophore approach. Specific results from these validations are available elsewhere, although examples are shown to demonstrate the analytical and statistical approaches used for gauging—with data analysis and statistical approaches—each device’s performance. Each metric used to evaluate devices (e.g., linearity, precision, and accuracy) was informative. However, linearity between the microscopy-based method and the compliance devices, especially along a large range of organism concentrations, would not be suitable for establishing performance criteria. Concentrations well below or above the limit for this size class (10 living organisms mL-1) would be easily categorized as meeting or exceeding discharge standard and their values do not need to be well constrained and pinpointed. Precision, when measured as the coefficient of variation, was sensitive to the dimensions and scale of the devices’ measurements, as certain devices calculated and reported cell concentrations, whereas other devices reported non-dimensional values along a wide dynamic range. Accuracy, defined as the agreement between the compliance device and the standard approach as to whether the sample met or exceeded the discharge standard, was measured by logistic regression analysis. Following this analysis, the likelihoods of detecting exceedances based upon cell concentration were calculated for each field site and cultured test organism. Accuracy was useful in defining the likelihood of correctly identifying an exceedance, and these likelihoods could be calculated for a range of cell concentrations. The concurrent testing of multiple compliance devices minimized the analysis burden as well as the logistical hurdles associated with field testing at multiple—for this study, three—locations. Eventually, the test procedures could be modified to measure variation among different units of the same device or applied to actual measurements of ballast water rather than communities of ambient organisms or cultured microalgae.