ACP Atmospheric Chemistry and Physics ACP 1680-7324 Copernicus GmbH Göttingen, Germany 10.5194/acp-11-7905-2011 Technical Note: Comparing the effectiveness of recent algorithms to fill and smooth incomplete and noisy time series Musial J. P. 1 2 Verstraete M. M. 1 Gobron N. 1 European Commission, Joint Research Centre, Institute for Environment and Sustainability, 21027 Ispra (VA), Italy current address: Geographisches Institut der Universität Bern (GIUB), 3012 Bern, Switzerland 04 08 2011 11 15 7905 7923 This is an open-access article ditributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. This article is available from http://www.atmos-chem-phys.net/11/7905/2011/acp-11-7905-2011.html The full text article is available as a PDF file from http://www.atmos-chem-phys.net/11/7905/2011/acp-11-7905-2011.pdf

Geophysical time series often feature missing data or data acquired at irregular times. Procedures are needed to either resample these series at systematic time intervals or to generate reasonable estimates at specified times in order to meet specific user requirements or to facilitate subsequent analyses. Interpolation methods have long been used to address this problem, taking into account the fact that available measurements also include errors of measurement or uncertainties. This paper inspects some of the currently used approaches to fill gaps and smooth time series (smoothing splines, Singular Spectrum Analysis and Lomb-Scargle) by comparing their performance in either reconstructing the original record or in minimizing the Mean Absolute Error (MAE), Mean Bias Error (MBE), chi-squared test statistics and autocorrelation of residuals between the underlying model and the available data, using both artificially-generated series or well-known publicly available records. Some methods make no assumption on the type of variability in the data while others hypothesize the presence of at least some dominant frequencies. It will be seen that each method exhibits advantages and drawbacks, and that the choice of an approach largely depends on the properties of the underlying time series and the objective of the research.

 References Box, G. E. P. and Jenkins, G. M.: Time Series Analysis Forecasting and Control, Holden-Day, Oakland California, 1976. Broomhead, D. S. and King, G. P.: Extracting qualitative dynamics from experimental data, Phys. D, 20, 217–236, 1986. Broomhead, D. S., Jones, R., and King, G. P.: Topological dimension and local coordinates from time series data, J. Phys. A, 20, 563–569, 1987. Craven, P. and Wahba, G.: Smoothing noisy data with spline functions estimating the correct degree of smoothing by the method of generalized cross validation, Numerische Mathematik, 31, 377–403, 1979. Fourier, J.: Theorie Analytique de la Chaleur, Firmin Didot (reissued by Cambridge University Press, 2009; ISBN 978-1-108-00180-9), 1822. Ghil, M. Allen, M. R., Dettinger, M. D., Ide, K., Kondrashov, D., Mann, M. E., Robertson, A. W., Saunders, A., Tian, Y., Varadi, F., and Yiou, P.: Advanced Spectral Methods for Climatic Time Series, Rev. Geophys., 40, 1–41, 2002. Gobron, N., Pinty, B., Aussedat, O., Chen, J. M., Cohen, W. B., Fensholt, R., Gond, V., Lavergne, T., Mélin, F., Privette, J. L., Sandholt, I., Taberner, M., Turner, D.P., Verstraete, M. M., and Widlowski, J.-L.: Evaluation of Fraction of Absorbed Photosynthetically Active Radiation Products for Different Canopy Radiation Transfer Regimes: Methodology and Results Using Joint Research Center Products Derived from SeaWiFS Against Ground-Based Estimations, J. Geophys. Res.-Atmos., 111, D13110, http://dx.doi.org/10.1029/2005JD006511doi:10.1029/2005JD006511, 2006. Golyandina, N., Nekrutkin V., and Zhigljavsky A.: Analysis of Time Series Structure: SSA and Related Techniques, Chapman and Hall, Washington DC, USA, 2001. Harris, F. J.: On the use of windows for Harmonic Analysis with the Discrete Fourier Transform, Proceedings of the IEEE, 66, 51–83, 1978. Hocke, K. and Kämpfer, N.: Gap filling and noise reduction of unevenly sampled data by means of the Lomb-Scargle periodogram, Atmos. Chem. Phys., 9, 4197–4206, http://dx.doi.org/10.5194/acp-9-4197-2009doi:10.5194/acp-9-4197-2009, 2009. Hutchinson, M. F. and de Hoog, F. R.: Smoothing Noisy Data with Spline Functions, Numerische Mathematik, 47, 99–106, 1985. Hutchinson, M. F.: Interpolation of Rainfall Data with Thin Plate Smoothing Splines – Part I: Two Dimensional Smoothing of Data with Short Range Correlation, J. Geogr. Info. Dec. Anal., 2, 139–151, 1998. Keeling, C. D., Chin, J. F. S., and Whorf, T. P.: Increased activity of northern vegetation inferred from atmospheric CO<sub>2</sub> measurements, Nature, 382(6587), 146–49, 1996. Kondrashov, D. and Ghil, M.: Spatio-temporal filling of missing points in geophysical data sets, Nonlinear Proc. Geophys., 13, 151–159, 2006. Lomb, N. R.: Least-squares frequency analysis of unequally spaced data, Astrophys. Space Sci., 39, 447–462, 1976. Mahecha, M. D., Fürst, L. M., Gobron, N. and Lange, H.: Identifying Multiple Spatiotemporal Patterns: A Refined View on Terrestrial Photosynthetic Activity, Patt. Recog. Lett., 31, 2309–2317, 2010. Moffat, A., Papale, D., Reichstein, M., Hollinger, D. Y., Richardson, A. D., Barr, A. G., Beckstein, C., Braswell, B. H., Churkina, G., Desai, A. R., Falge, E., Gove, J. H., Heimann, M., Hui, D., Jarvis, A. J., Kattge, J., Noormets, A., and Stauch, V. J.: Comprehensive comparison of gap-filling techniques for eddy covariance net carbon fluxes, Agr. For. Meteorol., 147, 209–232, 2007. Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P.: Numerical recipes in Fortran, Cambridge University Press, Cambridge, USA, 2 edn., 1992. Reinsch, C. M.: Smoothing by spline functions, Numerische Mathematik, 10, 177–183, 1967. Vautard, R., Yiou, P., and Ghil, M.: Singular-spectrum analysis: A toolkit for short, noisy chaotic signals, Physica D, 58, 95–126, 1992. Verstraete, M. M., Gobron, N., Aussedat, O., Robustelli, M., Pinty, B., Widlowski, J.-L., and Taberner, M.: An automatic procedure to identify key vegetation phenology events using the JRC-FAPAR products, Adv. Space Res., 41, 1773–1783, 2008. Wahba, G.: Spline Models for Observational Data, SIAM, Philadelphia, 1990. Wang, D. and Liang, S.: Singular Spectrum Analysis for Filling Gaps and Reducing Uncertainties of MODIS Land Products, International Geoscience and Remote Sensing Symposium (IGARSS), 5, 558–561, 2008. Whittaker, E. T.: On a new method of graduation, Proc. Edinburgh Math. Soc., 41, 63–75, 1923. Willmott, C. J. and Matsuura, K.: Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance, Clim. Res., 30, 79–82, 2005.