Atmospheric Chemistry and Physics www.atmos-chem-phys.net 1680-7316 1680-7324 8 18 2008 10.5194/acp-8-5699-2008 http://www.atmos-chem-phys.net/8/5699/2008/ http://www.atmos-chem-phys.net/8/5699/2008/acp-8-5699-2008.html http://www.atmos-chem-phys.net/8/5699/2008/acp-8-5699-2008.pdf 5699 5713 2008-09-29 Quantitative performance metrics for stratospheric-resolving chemistry-climate models D. W. Waugh V. Eyring Department of Earth and Planetary Science, Johns Hopkins University, Baltimore, MD, USA Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany A set of performance metrics is applied to stratospheric-resolving chemistry-climate models (CCMs) to quantify their ability to reproduce key processes relevant for stratospheric ozone. The same metrics are used to assign a quantitative measure of performance ("grade") to each model-observations comparison shown in Eyring et al. (2006). A wide range of grades is obtained, both for different diagnostics applied to a single model and for the same diagnostic applied to different models, highlighting the wide range in ability of the CCMs to simulate key processes in the stratosphere. No model scores high or low on all tests, but differences in the performance of models can be seen, especially for processes that are mainly determined by transport where several models get low grades on multiple tests. The grades are used to assign relative weights to the CCM projections of 21st century total ozone. For the diagnostics used here there are generally only small differences between weighted and unweighted multi-model mean and variances of total ozone projections. This study raises several issues with the grading and weighting of CCMs that need further examination. However, it does provide a framework and benchmarks that will enable quantification of model improvements and assignment of relative weights to the model projections. Akiyoshi, H., Sugita, T., Kanzawa, H., and Kawamoto, N.: Ozone perturbations in the Arctic summer lower stratosphere as a reflection of NO_x chemistry and planetary scale wave activity, J. Geophys. Res., 109, D03304, doi:10.1029/2003JD003632, 2004. Austin, J.: A three-dimensional coupled chemistry-climate model simulation of past stratospheric trends, J. Atmos. Sci., 59, 218–232, 2002. Austin, J., Shindell, D., Beagley, S. R., Brühl, C., Dameris, M., Manzini, E., Nagashima, T., Newman, P., Pawson, S., Pitari, G., Rozanov, E., Schnadt, C., and Shepherd, T. G.: Uncertainties and assessments of chemistry-climate models of the stratosphere, Atmos. Chem. Phys., 3, 1–27, 2003. Austin, J., Wilson, R. J., Li, F., and Vomel, H.: Evolution of water vapor concentrations and stratospheric age of air in coupled chemistry-climate model simulations, J. Atmos. Sci., 64, 905–921, 2006. Brunner, D., Staehlin, J., Rogers, H. L., et al.: An evaluation of the performace of chemistry transport models by comparison with research aircraft observations, Part 1: Concepts and overall model performance, Atmos. Chem. Phys., 3, 1609–1631, 2003. Connolley, W. M. and Bracegirdle, T. J.: An Antarctic assessment of IPCC AR4 climate models, Geophys. Res. Lett., 34, doi:10.1029/2007GL031648, 2007. Dameris, M., Grewe, V., Ponater, M., Deckert, R., Eyring, V., Mager, F., Matthes, S., Schnadt, C., Stenke, A., Steil, B., Brühl, C., and Giorgetta, M.: Long-term changes and variability in a transient simulation with a chemistry-climate model employing realistic forcings, Atmos. Chem. Phys., 5, 2121–2145, 2005. Douglass, A. R., Prather, M. J., Hall, T. M., Strahan, S. E., Rasch,P. J., Sparling, L. C., Coy, L., and Rodriguez, J. M.: Choosing meteorological input for the global modeling initiative assessment of high-speed aircraft, J. Geophys. Res., 104, 27 545–27 564, 1999. Egorova, T., Rozanov, E., Zubov, V., Manzini, E., Schmutz, W., and Peter, T.: Chemistry-climate model SOCOL: a validation of the present-day climatology, Atmos. Chem. Phys., 5, 1557–1576, 2005. Eyring, V., Harris, N. R. P., Rex, M., et al.: A strategy for process-oriented validation of coupled chemistry-climate models, B. Am. Meteorol. Soc., 86, 1117–1133, 2005. Eyring, V., Butchart, N., Waugh, D. W., et al.: Assessment of temperature, trace species, and ozone in chemistry-climate model simulations of the recent past, J. Geophys. Res., 111, D22308, doi:10.1029/2006JD007327, 2006. Eyring, V., Waugh, D. W., Bodeker, G. E., et al.: Multimodel projections of stratospheric ozone in the 21st century, J. Geophys. Res., 112, D16303, doi:10.1029/2006JD008332, 2007. Fomichev, V. I., Jonsson, A. I., de Grandpré, J., Beagley, S. R., et al.: Response of the middle atmosphere to CO₂ doubling: Results from the Canadian Middle Atmosphere Model, J. Climate, 20, 1121–1144, 2007. Garcia, R. R., Marsh, D., Kinnison, D., Boville, B., and Sassi, F.: Simulations of secular trends in the middle atmosphere, 1950–2003, J. Geophys. Res., 112, D09301, doi:10.1029/2006JD007485, 2007. Gelman, M. E., Miller, A. J., Johnson, K. W., and Nagatani, R.: Detection of long-term trends in global stratospheric temperature from NMC analyses derived from NOAA satellite data, Adv. Space Res., 6, 17–26, 1996. Gettelman, A., Birner, T., Eyring, V., Akiyoshi, H., Plummer, D. A., Dameris, M., Bekki, S., Lefevre, F., Lott, F., Brühl, C., Shibata, K., Rozanov, E., Mancini, E., Pitari, G., Struthers, H., Tian, W., and Kinnison, D. E.: The Tropical Tropopause Layer 1960–2100, Atmos. Chem. Phys. Discuss., 8, 1367–1413, 2008. Gleckler, P. J., Taylor, K. E., and Doutriaux, C.: Performance Metrics for Climate Models, J. Geophys. Res., 113, D06104, doi:10.1029/2007JD008972, 2008. Grooß, J.-U. and Russell III, J. M.: Technical note: A stratospheric climatology for O₃, H₂O, CH₄, NO_x, HCl and HF derived from HALOE measurements, Atmos. Chem. Phys., 5, 2797–2807, 2005. Hall, T. M., Waugh, D. W., Boering, K. A., and Plumb, R. A.: Evaluation of transport in stratospheric models, J. Geophys. Res., 104, 18 815–18 840, 1999. Lott, F., Fairhead, L., Hourdin, F., and Levan, P.: The stratospheric version of LMDz: Dynamical Climatologies, Arctic Oscillation, and Impact on the Surface Climate, Clim. Dynam., 25, 851–868, 2005. Mote, P. W., Rosenlof, K. H., McIntyre, M. E., Carr, E. S., Gille, J. C., Holton, J. R., Kinnersley, J. S., Pumphrey, H. C., Russell III, J., and Waters, J. W.: An atmospheric tape recorder: The imprint of tropical tropopause temperatures on stratospheric water vapor, J. Geophys. Res., 101, 3989–4006, doi:10.1029/95JD03422, 1996. Newman, P. A., Nash, E. R., and Rosenfield, J. E.: What controls the temperature of the Arctic stratosphere during spring?, J. Geophys. Res., 106, 19 999–20 010, 2001. Pawson, S., Stolarski, R. S., Douglass, A. R., Newman, P. A., Nielsen, J. E., Frith, S. M., and Gupta, M. L.: Goddard Earth Observing System Chemistry-Climate Model Simulations of Stratospheric Ozone-Temperature Coupling Between 1950 and 2005, J. Geophys. Res, 113, D12103, doi:10.1029/2007JD009511, 2008. Pitari, G., Mancini, E., Rizi, V., and Shindell, D.: Feedback of future climate and sulfur emission changes an stratospheric aerosols and ozone, J. Atmos. Sci., 59, 414–440, 2002. Randel, W., Udelhofen, P., Fleming, E., et al.: The SPARC Intercomparison of Middle-Atmosphere Climatologies, J. Climate, 17, 986–1003, 2004. Reichler, T. and Kim, J.: How well do coupled models simulate today's climate?, B. Am. Meteorol. Soc., 89, 303–311, 2008. Shibata, K. and Deushi, M.: Partitioning between resolved wave forcing and unresolved gravity wave forcing to the quasi-biennial oscillation as revealed with a coupled chemistry-climate model, Geophys. Res. Lett., L12820, doi:10.1029/2005GL022885, 2005. Schmittner A., Latif, M., and Schneider, B.: Model projections of the North Atlantic thermohaline circulation for the 21st century, Geophys. Res. Lett., 32, doi:10.1029/2005GL024368, 2005. Shepherd, T. G.: Dynamics, Stratospheric Ozone and Climate Change, Atmos.-Ocean, 46, 117–138, 2008. Steil, B., Brühl, C., Manzini, E., Crutzen, P. J., Lelieveld, J., Rasch, P. J., Roeckner, E., and Krüger, K.: A new interactive chemistry climate model, 1: Present day climatology and interannual variability of the middle atmosphere using the model and 9 years of HALOE/UARS data, J. Geophys. Res., 108, 4290, doi:10.1029/2002JD002971, 2003. Stevenson, D. S., Dentener, F. J., Schultz, M. G., et al.: Multimodel ensemble simulations of present-day and near-future tropospheric ozone, J. Geophys. Res., 111, D08301, doi:10.1029/2005JD006338, 2006. Strahan, S. E. and Douglass, A. R.: Evaluating the credibility of transport processes in simulations of ozone recovery using the Global Modeling Initiative three-dimensional model, J. Geophys. Res., 109, D05110, doi:10.1029/2003JD004238, 2004. Swinbank, R. and O'Neill, A.: A stratosphere-troposphere data assimilation system, Mon. Weather Rev., 122, 686–702, 1994. Taylor, K. E.: Summarizing multiple aspects of model performance in a single diagram, J. Geophys. Res., 106, 7183–7192, 2001. Tian, W. and Chipperfield, M. P.: A new coupled chemistry-climate model for the stratosphere: The importance of coupling for future O3-climate predictions, Q. J. Roy. Meteor. Soc., 131, 281–304, 2005. Uppala, S. M., Kållberg, P. W., Simmons, A. J., et al.: The ERA-40 reanalysis, Q. J. Roy. Meteor. Soc., 131, 2961–3012, 2005. Waugh, D. W. and Hall, T. M.: Age of stratospheric air: theory, observations, and models, Rev. Geophys., 40, 1010, doi:10.1029/2000RG000101, 2002. Wilks, D. S.: Statistical methods in the atmospheric sciences, Academic Press, London, UK, 467 pp., 1995. World Meteorological Organization (WMO)/United Nations Environment Programme (UNEP): Scientific Assessment of Ozone Depletion: 2006, World Meteorological Organization, Global Ozone Research and Monitoring Project, Report No. 50, Geneva, Switzerland, 2007.