Restoring lost water level modeling data

Extensive time series of measurements are often essential to evaluate and model long term changes and averages such as tidal datums and sea level rises. As such, gaps, due to data acquisition loses, in time series data restrict the type and extent of modeling and research which may be accomplished. The Texas A&M University Corpus Christi Division of Nearshore Research (A&M-CC-DNR) has developed and compared various methods based on forward and backward linear regression to interpolate gaps in time series of water level data [1]. Our time series consist of water level data collected at six-minute intervals for about 60 stations along the coast of Texas for up to 15 years depending on the station. A&M-CC-DNR collects, archives and makes available through the World Wide Web such time series. Our program retrieves actual and harmonic water level data based upon user provided parameters. The actual water level data is searched for missing data points and the location of these gaps are recorded. The difference between the corresponding actual water level data and the harmonic water level data is then calculated. The harmonic component of the water level data has been calculated using several years of time series and is available for most of the stations. Forward and backward linear regression are applied in relation to the location of the gaps in the remaining data. After this process is complete, one of three combinations of the forward and backward regression is used to fit the results. The methods of combination are convex linear combination, convex trigonometric combination and combination by intersection. Finally, the harmonic component is added back into the newly supplemented time series and the results are graphed. The software system created to implement this process of linear regression is written in Perl along with a Perl module called PDL (Perl Data Language). Perl was chosen as the data language due to its ease and power of data extraction, manipulation and formatting. In addition, the PDL module allows the user to store and manipulate large amounts of data in a time and memory efficient manner. The computational efficiency of these algorithms will allow for a real-time web based implementation where the gaps are filled at the time of request. Generally, this process has demonstrated excellent results in filling gaps in our water level time series. The program was tested on existing data under three types of typical weather conditions: calm summers, frontal passages and extreme weather conditions, such as hurricanes. The parameters varied in order to test the accuracy of the methodology included the number of coefficients utilized in the linear regression processes as well as the size of the gaps to be filled. United States National Ocean Service (NOS) standards such as the Root Mean Square Error and the Central Frequency are used to assess the quality of the interpolation [2], Results will be presented for the different weather conditions and the different gap size and coefficient combinations.