Articles | Volume 19, issue 5
https://doi.org/10.5194/acp-19-2881-2019
https://doi.org/10.5194/acp-19-2881-2019
Research article
 | 
07 Mar 2019
Research article |  | 07 Mar 2019

Advanced methods for uncertainty assessment and global sensitivity analysis of an Eulerian atmospheric chemistry transport model

Ksenia Aleksankina, Stefan Reis, Massimo Vieno, and Mathew R. Heal
Abstract

Atmospheric chemistry transport models (ACTMs) are extensively used to provide scientific support for the development of policies to mitigate the detrimental effects of air pollution on human health and ecosystems. Therefore, it is essential to quantitatively assess the level of model uncertainty and to identify the model input parameters that contribute the most to the uncertainty. For complex process-based models, such as ACTMs, uncertainty and global sensitivity analyses are still challenging and are often limited by computational constraints due to the requirement of a large number of model runs. In this work, we demonstrate an emulator-based approach to uncertainty quantification and variance-based sensitivity analysis for the EMEP4UK model (regional application of the European Monitoring and Evaluation Programme Meteorological Synthesizing Centre-West). A separate Gaussian process emulator was used to estimate model predictions at unsampled points in the space of the uncertain model inputs for every modelled grid cell. The training points for the emulator were chosen using an optimised Latin hypercube sampling design. The uncertainties in surface concentrations of O3, NO2, and PM2.5 were propagated from the uncertainties in the anthropogenic emissions of NOx, SO2, NH3, VOC, and primary PM2.5 reported by the UK National Atmospheric Emissions Inventory. The results of the EMEP4UK uncertainty analysis for the annually averaged model predictions indicate that modelled surface concentrations of O3, NO2, and PM2.5 have the highest level of uncertainty in the grid cells comprising urban areas (up to ±7 %, ±9 %, and ±9 %, respectively). The uncertainty in the surface concentrations of O3 and NO2 were dominated by uncertainties in NOx emissions combined from non-dominant sectors (i.e. all sectors excluding energy production and road transport) and shipping emissions. Additionally, uncertainty in O3 was driven by uncertainty in VOC emissions combined from sectors excluding solvent use. Uncertainties in the modelled PM2.5 concentrations were mainly driven by uncertainties in primary PM2.5 emissions and NH3 emissions from the agricultural sector. Uncertainty and sensitivity analyses were also performed for five selected grid cells for monthly averaged model predictions to illustrate the seasonal change in the magnitude of uncertainty and change in the contribution of different model inputs to the overall uncertainty. Our study demonstrates the viability of a Gaussian process emulator-based approach for uncertainty and global sensitivity analyses, which can be applied to other ACTMs. Conducting these analyses helps to increase the confidence in model predictions. Additionally, the emulators created for these analyses can be used to predict the ACTM response for any other combination of perturbed input emissions within the ranges set for the original Latin hypercube sampling design without the need to rerun the ACTM, thus allowing for fast exploratory assessments at significantly reduced computational costs.

Dates
1 Introduction

Air pollution has a wide range of detrimental impacts. Exposure to air pollutants such as nitrogen dioxide (NO2), ozone (O3), and particulate matter (PM2.5) is associated with increased risk of stroke, cardiovascular disease, and chronic and acute respiratory diseases (WHO, 2006, 2013). Additionally, particulate matter and O3 contribute to climate change through radiative forcing and aerosol–cloud interactions (for PM) (IPCC, 2013; Stevenson et al., 2013), and O3 has an adverse impact on natural and semi-natural vegetation and crop yields (Teixeira et al., 2011).

To reduce the harmful impact of air pollution, various policies and directives have been implemented. For example, in the European Union, the Ambient Air Quality Directive (EC Directive, 2008) sets limit values on ambient concentrations of air pollutants, whilst other directives set source-specific emissions limits. Atmospheric chemistry transport models (ACTMs) play an essential role in the evaluation of the potential outcomes of different management options aimed at the improvement of future air quality.

The majority of existing ACTMs are deterministic, meaning that the output variables are presented as a single value without any indication of the expected uncertainty around this value. The uncertainty estimate for the modelled value is critical because it provides an assessment of confidence in the model predictions and the confidence range may encompass different recommendations that can be drawn from the model (Rypdal and Winiwarter, 2001; Frost et al., 2013). There are various sources of uncertainty in a model; the sources range from structural or conceptual uncertainties about how well a given model represents reality to uncertainties in the model input data and physical and chemical constants, which have an effect on the calculation results of the model. It has been previously found that uncertainties in input emissions are major contributors to the uncertainty in ACTM outputs (Sax and Isakov, 2003; Hanna et al., 2007; Rodriguez et al., 2007). Therefore, this study concentrates on implementing a systematic approach for ACTM output uncertainty quantification and on determining the extent to which different input emissions drive the uncertainty in the output variables.

Analytical uncertainty propagation is not feasible for complex models such as ACTMs because it requires an exact function for input–output mapping. Consequently, Monte Carlo-based methods for uncertainty assessment have to be used. Uncertainty analysis should be performed in tandem with sensitivity analysis to maximise the knowledge gained. The main distinction between uncertainty and sensitivity analysis is that uncertainty analysis is performed to quantify model output uncertainty arising from the uncertainty in a single or multiple inputs, whilst sensitivity analysis is performed to investigate input–output relationships and to apportion the variation in model output to the different inputs. Hence, the sensitivity analysis allows conclusions to be drawn on the extent to which the overall variation in the modelled values is driven by variation in different inputs (Saltelli, 2002).

For computationally demanding models, such as ACTMs, a local one-at-a-time (OAT) sensitivity analysis is the most commonly used approach (Ferretti et al., 2015). However, unlike global sensitivity analysis, the local OAT approach does not take into account the non-linearities in the model response and the interactions between the input parameters (Saltelli and Annoni, 2010; Aleksankina et al., 2018).

The computational cost of running ACTMs to explore the entire parameter space of the uncertain inputs using Monte Carlo-based uncertainty and sensitivity analyses is typically prohibitively high because the analyses require a large number of points in parameter space, which translates to thousands of model simulations. To tackle this issue, the use of meta-models has been increasing in recent years (Yang, 2011; Ratto et al., 2012; Iooss and Lemaître, 2015; Gladish et al., 2017). A meta-model (or emulator) is a statistical approximation of the original simulation model that can be evaluated many times at a lower computational cost relative to the original model (O'Hagan, 2006; Castelletti et al., 2012). This approach allows the output of an ACTM for a large number of points in parameter space to be estimated efficiently, making uncertainty and sensitivity analyses feasible.

Different meta-modelling approaches have been used for uncertainty and sensitivity analysis; these techniques include regression smoothers (Storlie and Helton, 2008; Storlie et al., 2009), Gaussian process emulator (Oakley and O'Hagan, 2004), high-dimensional model representation (Rabitz and Alış, 1999; Ziehn and Tomlin, 2009), and polynomial chaos expansion (Sudret, 2008). Meta-models have been applied for uncertainty and sensitivity analyses in earth science fields such as ecological modelling (Luo et al., 2013; Parry et al., 2013), hydrological modelling (Asher et al., 2015; Gladish et al., 2017), and atmospheric aerosol modelling (Lee et al., 2011; Carslaw et al., 2013; Chen et al., 2013; Christian et al., 2017).

In this study, a Gaussian process is used for emulation because of its desirable properties and available implementations (i.e. MATLAB-based software UQLab or R package DiceKriging). Gaussian process emulators are non-parametric statistical models that use the principles of conditional probability to estimate model outputs. The beneficial properties are the curve that fits through the training points (for deterministic models) and a measure of the uncertainty for the estimated points when using an emulator in place of the original model for the estimation of new points.

The efficiency of the emulator compared to the original model is determined by how smooth and continuous the model response is to input perturbations. For a smooth and continuous input–output relationship, the high correlation between the inputs and the simulated points means a lower uncertainty in predictions made using the emulator further away from the training points (i.e. resulting in a good emulator performance with a small number of training points) (Lee et al., 2011).

The design of computer experiments for deterministic models differs from designs for physical experiments. As there is no random error involved in computer experiments, replication is not required (Jones and Johnson, 2009). Hence, sampling techniques that have good space-filling properties and the ability to maintain uniform spacing when projected into a lower-dimensional space are used (Jones and Johnson, 2009; Dean et al., 2015). Latin hypercube sampling (LHS) introduced by McKay et al. (1979) meets these desirable criteria. Additionally, advances have been made to optimise the space-filling properties of LHS including maximin sampling (Johnson et al., 1990; Morris and Mitchell, 1995) and the ability to add extra design points to the parameter space if necessary (Sheikholeslami and Razavi, 2017), which makes it well suited for multidimensional designs that may require the addition of extra points.

The aim of this study is to demonstrate the method for uncertainty assessment and global sensitivity analysis for computationally demanding ACTMs. The ACTM to which the method is applied here is the WRF-EMEP4UK model (Vieno et al., 2010, 2014, 2016a), and the outputs of interest are the modelled surface concentrations of O3, NO2, and PM2.5, but the methodology is generic for model and output variables. The analyses described here investigated sensitivities and uncertainties of model output to emissions from UK land-based sources and from surrounding shipping. Additionally, we identify which model inputs drive uncertainty in the output variables and to what extent, as well as discussing how the uncertainty ranges that are obtained affect current predictions and scenario analysis outcomes (i.e. confidence in model outputs).

2 Methods

2.1 Model description

The EMEP4UK model is a regional application of the EMEP MSC-W (European Monitoring and Evaluation Programme Meteorological Synthesizing Centre-West) open-source ACTM (https://www.github.com/metno/emep-ctm, version rv4.8, last access: 11 June 2018). The detailed description of EMEP MSC-W is available from Simpson et al. (2012), and the EMEP4UK model is described by Vieno et al. (2010, 2014, 2016a).

EMEP4UK is a 3-D one-way nested Eulerian model with a horizontal resolution of 5 km × 5 km over the British Isles nested within an extended European domain with 50 km × 50 km resolution. The extent of the inner domain is shown in Fig. 1. The model has 20 vertical levels, extending from the ground to 100 hPa with the lowest vertical layer at ∼90 m. The model time step is 20 s for chemistry, 5 min for the advection in the inner domain, and 20 min for the advection in the outer domain.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f01

Figure 1The inner shaded box illustrates the EMEP4UK model British Isles domain, which is modelled at 5 km × 5 km horizontal resolution. The locations of five grid cells used for uncertainty quantification and sensitivity analysis for monthly average modelled concentrations of O3, NO2, and PM2.5 are shown.

Download

The meteorological fields were computed using Weather Research and Forecast model version 3.1.1 (http://www.wrf-model.org/, last access: 15 November 2017) (Skamarock et al., 2008). The WRF model initial and boundary conditions are derived from the US National Center for Environmental Prediction (NCEP)/National Center for Atmospheric Research (NCAR) Global Forecast System (GFS) at 1 resolution, including Newtonian nudging every 6 h (NCEP, 2000).

The anthropogenic emissions of sulfur dioxide (SO2), nitrogen oxides (NOx), ammonia (NH3), fine and coarse primary particulate matter (PM2.5, PMcoarse), carbon monoxide (CO), and non-methane volatile organic compounds (NMVOC) for the UK were derived from the National Atmospheric Emissions Inventory (http://naei.beis.gov.uk/, last access: 15 October 2015). For the outer domain, the emissions are provided by the Centre for Emission Inventories and Projections (CEIP; http://www.ceip.at/, last access: 15 October 2017). All emissions are split across a set of emission source sectors defined by the Selected Nomenclature for Air Pollutants (SNAP) described in Table 1. The hour-of-day, day-of-week, and monthly emission factors are used to distribute the annual total emissions temporally to hourly resolution as described in Simpson et al. (2012). The international shipping emissions were derived from ENTEC UK Ltd. (now Amec Foster Wheeler) (Entec, 2010). Biogenic emissions of dimethyl sulfide in addition to monthly in-flight aircraft, soil, and lightning NOx emissions are included as described in Simpson et al. (2012). Biogenic emissions of monoterpenes and isoprene are calculated by the model for every grid cell and time step according to the methodology of Guenther et al. (1993, 1995) using near-surface air temperature and photosynthetically active radiation as well as aggregated land-cover categorisations, as described in Simpson et al. (2012). The emissions of sea salt and wind-blown dust are also included. The details of the sea salt generation parameterisation scheme used in the model are presented in Monahan et al. (1986) and Mårtensson et al. (2003). The boundary condition monthly average concentrations of fine and coarse dust are calculated with the global chemical transport model of the University of Oslo (Grini et al., 2005); the detailed parameterisation of dust mobilisation is presented in Simpson et al. (2012).

Table 1SNAP source sectors (Eurostat, 2004).

Download Print Version | Download XLSX

The chemistry, aerosol formation, and wet and dry deposition schemes are as described in Simpson et al. (2012). The chemistry scheme has 72 species and 137 reactions, and the gas–aerosol partitioning is described by the MARS formulation. A detailed evaluation of model performance is discussed elsewhere (Vieno et al., 2010, 2016b; Dore et al., 2015; Lin et al., 2017). In our study, all model runs were executed using meteorology and emissions data for the year 2012.

2.2 Input variables and their uncertainty ranges

In this study, emissions of all of the major primary anthropogenic pollutant compounds were investigated. The decision to concentrate on anthropogenic emissions was made based on the fact that one of the main applications of the EMEP4UK model is providing scientific support for policy-making regarding the impacts of interventions leading to anthropogenic emissions reductions. Hence, the potential future changes in emissions driven by environmental and climate change policies are not likely to affect biogenic emissions as much as anthropogenic emissions. Therefore, it was decided to investigate model response to the changes in anthropogenic emissions.

For this study, emissions of five pollutants (NOx, SOx, VOC, NH3, primary PM2.5) were split into 13 model input variables based on the contributions from different emission source sectors to total annual emissions; the emissions from the dominant sector (the sector with the highest relative contribution to total emissions) for every pollutant were treated as a separate variable, while the emissions from the rest of the sectors were grouped and treated as another input variable. Shipping emissions were treated as a separate variable and were not split by the pollutant type. The description of the SNAP sectors is shown in Table 1, and the definitions of the input variables for the uncertainty and sensitivity analyses in this work are presented in Table 2, where variables marked with D represent emissions from a single dominant sector (D1 and D2 in the case of multiple dominant sectors) and variables marked with O indicate the grouped “other” emissions from the rest of the sectors. Emissions from “natural” sources (e.g. lightning, soil, ocean) were not part of the uncertainty and sensitivity analyses.

Table 2Input variable definitions for the EMEP4UK uncertainty propagation and apportionment. The quoted uncertainties for emission sources are for UK annual totals. See main text for information on the sources of these values.

n/a: not applicable.

Download Print Version | Download XLSX

Uncertainty ranges for the input emissions from UK anthropogenic land-based sources were assigned according to data in the UK Informative Inventory Report (IIR) (Wakeling et al., 2017). In the IIR, uncertainties are defined as upper and lower limits of the 95 % confidence interval relative to the central estimate. There is no information on uncertainty ranges for different source sectors available for the emissions for 2012 because uncertainties split by the emission source sector were first presented in the IIR that included 2014 emissions (Wakeling et al., 2016). Hence, for this study, the most recently published data for the uncertainty ranges of pollutants split by source sector were used.

Equation (1) was used to aggregate uncertainties for multiple emission source sectors for the grouped-source input variables, where x is the quantity of interest and U is the uncertainty of that quantity, taken from the EMEP/EEA air pollutant emission inventory guidebook (Pulles and Kuenen, 2016).

(1) U total = ( U 1 x 1 ) 2 + ( U 2 x 2 ) 2 + + ( U n x n ) 2 x 1 + x 2 + + x n

The shipping emission variable in this study combines all emissions of all relevant pollutants, and hence a “best estimate” range for the uncertainty was chosen. The range was estimated based on the available published information. The shipping emissions were not split by pollutant because the inclusion of the split for this source comprised too great a computational cost for the analyses. Most shipping emissions do not impact land-based population and ecosystem exposure, which was our focus, compared with terrestrial emissions.

Some recently published sources (Corbett, 2003; Scarbrough et al., 2017) state that the uncertainty in shipping emissions is significant, but do not provide quantitative estimates. The most recent source of quantitative information on the uncertainty in shipping emissions is the report for the European Commission (Entec, 2002) which presents the estimates of uncertainties for emission factors of NOx, SO2, PM, and VOC for shipping emissions “at sea”, “manoeuvring”, and “in port”. The uncertainties are presented for emissions in the year 2000 as 95 % confidence intervals (CIs) with the lowest values of uncertainty presented for “at sea” emission factors (±10 %–20 %) and the highest values for “manoeuvring” emission factors (±30 %–50 %). For the total pollutant emissions for the year 2000 the percentage uncertainties around the estimates are ±21 % for NOx, ±11 % for SO2, ±11 % for CO2, ±28 % for VOC, and ±45 % for PM. Additionally, in Moreno-Gutiérrez et al. (2015) the uncertainty in the emission factors for all pollutant compounds was estimated to be ±20 %. Using the above data, an overall uncertainty of ±30 % was assigned to the shipping emissions variable in this study (Table 2). It was applied to all shipping emissions within the inner British Isles domain of the EMEP4UK model.

2.3 Gaussian process emulator for EMEP4UK

A Gaussian process emulator was used to estimate model predictions at unsampled points in the space of the uncertain model inputs. The UQLab, a MATLAB-based software framework for uncertainty quantification (Marelli and Sudret, 2014; Lataniotis et al., 2017), was implemented to build the emulators for the uncertainty propagation and the following sensitivity analysis. A comprehensive description of the statistical theory of Gaussian process applied to uncertainty and sensitivity analysis with full mathematical details can be found in O'Hagan (2006) and Oakley and O'Hagan (2002, 2004).

The uncertainty values and sensitivity indices were calculated for three EMEP4UK model outputs (O3, NO2, and PM2.5 surface concentrations) with annual and monthly temporal resolution. For the annually averaged outputs, an emulator was created for each modelled grid cell in the EMEP4UK domain (n= 59 400). The first- and total-order sensitivity indices were calculated for the land-based grid cells only (n > 10 000). For the monthly mean model outputs, uncertainty and sensitivity analyses were performed for five selected grid cells. The five grid cells were selected to contain a UK national-network air pollution monitoring station to aid classification according to the environment (i.e. rural background, urban background, and urban traffic) and also to provide geographically representative coverage across the UK.

LHS maximin design, which maximises the minimum distance between the points in the parameter space to provide the optimum space-filling properties, was used. The design was previously demonstrated as suitable for Gaussian process emulators by Jones and Johnson (2009). A design with 84 data points was created for the scaling coefficients that were subsequently applied to the input emissions. This means that emissions corresponding to a particular input variable were perturbed homogeneously throughout the whole of the UK model domain. The ranges of scaling coefficient used for the sampling design are presented in Table 2. In this study, 84 sampling points were found to be sufficient to create an adequately performing emulator, as the input–output response function for the EMEP4UK model was expected to be smooth on monthly and annually averaged timescales. More generally, however, for the case in which model runs are computationally expensive and the input–output relationship is less predictable a sequential sampling technique can be applied to track the improvement of emulator performance with an increase in the sample size.

In this study, the surface concentration of O3, NO2, and PM2.5 for every grid cell is defined as a scalar output Y=f(X), where X is the vector of input values {X1, … , X13}.

A Gaussian process emulator utilises a Bayesian approach; training data are used to update the selected prior to produce posterior mean and covariance functions. The Gaussian process is specified by its mean function and covariance function. The mean function is given by Eq. (2):

(2) E [ f ( x ) | β ] = h ( x ) T β ,

where h() is a vector of regression functions and β is a vector of unknown coefficients. The choice of h() incorporates any prior beliefs about the form of f(⋅). In this study, the mean function was chosen to have a linear form βo+i=113βixi on the basis that the response of the surface concentration to changes in input emissions is expected to be smooth with no discontinuities or fluctuations.

The covariance function between f(x) and f(x) is given by Eq. (3):

(3) cov f x , f x | σ 2 = σ 2 c ( x , x ) ,

where σ2 is the hyperparameter that represents the variance of the Gaussian process and c(x,x) is the correlation function. The correlation function increases as the distance between x and x decreases and equals 1 when x=x. In this study Matérn 5/2 (Eq. 4) was used, where h is the absolute distance between x and x, and θ is a vector of range parameters or length scales, which define how far one needs to move along a particular axis in the input space for the function values to become uncorrelated.

(4) c x , x = 1 + 5 h θ + 5 h 2 3 θ 2 exp - 5 h θ

A number of emulators were built with the EMEP4UK simulation data using other available covariance functions; however, little difference was found in the performance of the emulators. The hyperparameters β, σ2, and θ were estimated using a cross-validation approach.

The emulator error was estimated by implementing k-fold cross-validation (Urban and Fricker, 2010; Gladish et al., 2017). The original sample was randomly partitioned into k=10 subsamples, which allowed approximately 90 % of the data to be used as a training set and 10 % as a validation set. The spatial distribution of cross-validation errors is presented in the Supplement (Fig. S1).

2.4 Uncertainty and sensitivity analysis

2.4.1 Uncertainty propagation

The uncertainties for the EMEP4UK output variables were estimated using a Monte Carlo approach (also described in the IPCC guidelines (IPCC, 2006) as a Tier 2 approach). The specific uncertainty ranges assigned to the input emission variables were used to constrain the input sampling space. All inputs were assigned normal distributions with a baseline value as the mean and the standard deviation derived from the corresponding confidence interval (Table 1). For every grid cell, the emulator was used to predict model values of surface concentrations of O3, NO2, and PM2.5 at the new set of input points (n=5000). The resulting probability distributions for each grid cell were evaluated, and the resulting uncertainty was estimated as half of the 95 % confidence interval relative to the central estimate (i.e. the mean for normally distributed values) of the output value, as described in the EMEP/EEA and IPCC guidebooks (IPCC, 2006; Pulles and Kuenen, 2016). The uncertainty for the monthly average modelled surface concentrations of O3, NO2, and PM2.5 was calculated for five grid cells using the same approach as above. The locations of the grid cells within the UK are shown in Fig. 1. The five grid cells selected were assigned the following environment types (the names and environment type reflect those of the national-network monitoring site within that grid cell): Auchencorth Moss and Harwell – rural background; Birmingham Acocks Green and London N. Kensington – urban background; and London Marylebone Road – urban traffic.

2.4.2 Global sensitivity analysis; first- and total-order indices

A variance-based global sensitivity analysis was conducted to apportion overall uncertainty in modelled variables to the uncertainty in the input emissions. Sobol' first- and total-order sensitivity indices were estimated (Sobol', 1993, 2001; Homma and Saltelli, 1996; Janon et al., 2014). The first-order indices represent the fraction of total variance of the output (i.e. the proportion of the overall uncertainty in Y) explained by the variance in an input Xi, while total-order indices show the sum of the effects due to an input Xi and all of its interactions with other inputs (X∼i). Therefore, the values of first- and total-order indices can be compared to identify the presence of interactions between input Xi and all other model inputs.

Unlike an OAT sensitivity coefficient, a first-order sensitivity index accounts for the non-linear response of a model output to a parameter across the specified parameter variation range. Sensitivity indices in this context are also indicators of importance for the input variables.

The first-order sensitivity index is defined as the ratio of the variance of the mean of Y when one input variable is fixed, VXi(EXi(Y|Xi)), to the unconditional variance of Y, V(Y) (Eq. 5).

(5) S i = V X i ( E X i ( Y | X i ) ) V ( Y )

The total-order sensitivity index measures the total effect of a variable, which includes its first-order effect and interactions with any other variables (Eq. 6).

(6) S T i = 1 - V X i E X i Y | X i V Y = E X i V X i Y | X i V Y ,

where X∼i denotes the matrix of all variables but Xi. In EXi(VXi(Y|Xi)) the inner variance of Y is taken over all possible values of Xi while keeping X∼i fixed, while the output expectation E is taken over all possible values X∼i (Ghanem et al., 2017).

The first- and total-order sensitivity indices were estimated following the methods described by Sobol' (1993) and Janon et al. (2014), respectively.

For the annual average modelled surface concentrations of O3, NO2, and PM2.5, the sensitivity indices were calculated for the UK land-based grid cells for the whole domain. For the monthly average modelled concentrations, sensitivity indices for five selected grid cells (discussed above) were estimated to determine whether seasonality affects the magnitude of the sensitivity indices.

3 Results and discussion

3.1 Uncertainty propagation

Figure 2 shows the spatial distribution of annual average surface concentrations of O3, NO2, and PM2.5 modelled with EMEP4UK and their absolute and relative uncertainties given the uncertainties in UK pollutant emissions for each source sector shown in Table 2. The uncertainties are presented as a range ± the baseline value and represent the 95 % confidence interval. The maps represent the uncertainty in surface concentrations propagated from the uncertainties reported in the UK emissions (Wakeling et al., 2017) and estimated uncertainties in shipping emissions in the EMEP4UK model domain (Entec, 2002; Moreno-Gutiérrez et al., 2015). The uncertainties in surface concentration do not incorporate any uncertainties in the spatial and temporal aspects of the input emissions because no data on these aspects of uncertainty are provided by the compilers of the emissions inventories.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f02

Figure 2Baseline surface concentrations of O3, NO2, and PM2.5 and their respective spatial distributions of absolute and relative uncertainties (at the 5 km × 5 km model grid resolution, year 2012) for the specified uncertainties in UK emissions. The uncertainty values are represented as a range ± the baseline value and represent the 95 % confidence interval.

Download

For O3 and NO2 the areas with the highest uncertainty coincide with the locations of shipping lanes. This is due to assigning all shipping emissions an uncertainty of ±30 %, which causes high variability in the corresponding NOx emissions. The uncertainty in O3 surface concentrations for the land-based grid cells is generally low (median relative uncertainty is ±0.6 %) with values of uncertainty up to ±7 % or ±1.4 ppb occurring in the grid cells containing major UK cities. The overall low uncertainty in the modelled O3 concentrations can be attributed to the combination of a low uncertainty in precursor emissions and the substantial contribution of hemispheric background O3 to UK ambient concentrations, which are not part of this analysis of uncertainty with respect to the UK-only emissions (Simpson et al., 2012).

The relative uncertainty of NO2 has a homogeneous spatial pattern (median relative uncertainty for all land-based grid cells is ±7.4 %), while absolute uncertainty is found to be higher (up to ±3.5µg m−3 or ±9 %) in areas with major UK cities. The magnitude of uncertainty in NO2 is determined by the combination of two factors: (i) NO2 uncertainty is driven by NOx emission inputs, which have low levels of uncertainty associated with them, and (ii) low overall variation in O3 surface concentrations affects the reactions between NO, NO2, and O3 that are linked through the photolysis of NO2 to give NO and the reaction of NO with O3 to produce NO2.

The spatial pattern of PM2.5 surface concentrations and the corresponding absolute and relative uncertainties differ from those for O3 and NO2. The concentration gradient indicates the presence of transboundary PM2.5 transport into the UK. This is consistent with findings reported by AQEG (2013) that only about half of the PM2.5 annual average concentrations have a UK origin. The spatial pattern of uncertainty in PM2.5 concentrations shows higher uncertainty, both relative and absolute, in the grid cells with major cities; median relative uncertainty for all land-based grid cells is ±4.6 % with up to ±9 % (±0.9µg m−3) in the grid cells with major cities. The surface concentrations of PM2.5 are dominantly comprised of primary PM2.5 emissions and inorganic aerosols resulting from chemical reactions between SO2, NOx, and NH3. Hence, the spatial pattern of uncertainty can be explained by the fact that the main contribution to primary PM2.5 comes from emissions from sources such as stationary combustion (e.g. residential heating) and road transport. The pattern of decreasing uncertainty from the land-based grid cells (centre) towards the edges of the domain indicates a change in variation due to the transport of PM2.5 away from the sources of emitted pollutants.

The overall uncertainty in the output variables (O3, NO2, and PM2.5) was found to be lower compared to the uncertainty of the model input emissions. This can be explained by the overall weak response of surface concentrations to changes in emissions originating from the UK, which leads to the conclusion that surface concentrations are affected by the transport of pollutants from elsewhere. Another explanation is the so-called “compensation of errors” whereby a positive effect of one or multiple input variables on the output is compensated for by a negative effect of other input variables. This leads to the narrower confidence intervals associated with EMEP4UK outputs (Skeffington et al., 2007).

An important observation from this uncertainty analysis is that the areas with the highest uncertainty coincide with the most populated areas. Given that O3, NO2, and PM2.5 are associated with adverse health effects, it is particularly important to have an estimate for the confidence level of the modelled values in more densely populated regions. This work has shown that the highest uncertainty is precisely in these regions. The reason for the increased levels of uncertainty in the grid cells coinciding with urban areas is discussed below.

3.2 Sensitivity analysis

In addition to quantitative uncertainty estimates, it is of interest to know how the uncertainty of each input contributes to the overall uncertainty and whether there are interactions between inputs that potentially affect the magnitude of overall uncertainty. This was achieved by conducting a variance-based sensitivity analysis.

Figures 3, 4, and 5 show the spatial distribution of the first-order sensitivity indices that represent the fractional contribution of the uncertainty of each input variable to the overall uncertainty in the output. Only the variables with Si>0.03 are presented here. First-order indices with values less than 0.03 were omitted as the method used for computation of sensitivity indices is prone to numerical errors when the analytical sensitivity index values are close to zero (Saltelli et al., 2006). The threshold was estimated by examining the noise in first-order sensitivity indices calculated for unimportant input variables. Excluding Si<0.03 does not have an effect on the results presented because a relative contribution of less than 3 % to the overall uncertainty can be considered negligible.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f03

Figure 3Spatial distributions (at the 5 km × 5 km model grid resolution) of the first-order sensitivity indices for modelled surface concentrations of O3. D indicates emissions from a dominant sector and O indicates grouped emissions from the rest of the sectors. For NOx emissions the dominant sectors are energy production (D1) and road transport (D2); for VOC emissions solvent use is dominant, and for NH3 agriculture is dominant. The shipping emissions variable combines emissions of all relevant pollutants.

Download

The difference between total- and first-order sensitivity is used to highlight interactions between the variable Xi and all other input variables. For the sensitivity coefficients computed for the annual-averaged model outputs, there was no substantial difference found between first- and total-order sensitivity indices, and hence no between-input interactions were identified on the annual timescale (Fig. S2).

Figure 3 shows the spatial distribution of first-order sensitivity indices for the input variables affecting modelled O3 concentrations. It is predominantly the NOx input emissions that drive the uncertainty in modelled O3 surface concentrations.

The greatest contribution to O3 surface concentration uncertainty in areas with higher levels of overall uncertainty is from the input variable NOx_O, which represents NOx emissions from all the other SNAP sectors apart from SNAP 1 (combustion in the energy and transformation industries) and SNAP 7 (road transport). The NOx emissions combined into this input variable account for 27 % of total NOx emissions, and the uncertainty range for this variable is ±19 %. The input variable NOx_D1 (emissions from combustion in the energy and transformation industries) does not contribute substantially to output uncertainty despite making up 41 % of total NOx emissions, with a relative uncertainty of ±7 %. This is explained by the height at which these emissions occur; the emissions are injected into the vertical layers at heights of > 184 m above ground level. This leads to NOx being dispersed and transported away from these elevated sources without affecting ground-level O3 concentrations locally. The NOx emissions from the input variable NOx_D2 (road transport) account for the remaining 32 % of total NOx emissions. The spatial distribution of corresponding sensitivity indices indicates that uncertainty in road transport emissions affects overall uncertainty in O3 surface concentrations in the grid cells closest to the emission sources (i.e. major roads). A large proportion (> 80 %) of overall uncertainty in O3 concentrations in areas adjacent to the south and south-east coasts of England is apportioned to the uncertainty in shipping emissions.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f04

Figure 4Spatial distributions (at the 5 km × 5 km model grid resolution) of the first-order sensitivity indices for modelled surface concentrations of NO2. D indicates emissions from a dominant sector and O indicates grouped emissions from the rest of the sectors. For NOx emissions the dominant sectors are energy production (D1) and road transport (D2). The shipping emissions variable combines emissions of all relevant pollutants.

Download

In Scotland, most of the overall uncertainty in the O3 surface concentration is apportioned to the variables VOC_D and VOC_O that respectively represent VOC input emissions from the dominant VOC source sector (solvent and other product use) and emissions from the rest of the source sectors grouped into a single input. A small proportion is apportioned to the variable NH3_D that represents NH3 emissions from agricultural sources. The effect of these input variables manifests in Scotland because of low levels of locally emitted NOx. The overall uncertainty in this area is very low.

In summary, the uncertainty in modelled surface concentrations of O3 in densely populated areas can be apportioned to the uncertainty in NOx emissions from non-dominant sources and uncertainty in shipping emissions.

The uncertainty in the surface concentration of NO2 was found to be driven mostly by uncertainty in NOx emissions (variables NOx_D1, NOx_D2, NOx_O) and shipping emissions (Fig. 4). Similarly to O3, NO2 is most sensitive to NOx emissions combined from all SNAP sectors apart from SNAP 1 (combustion in energy and transformation industries) and SNAP 7 (road transport). There is almost no sensitivity to NOx emissions from SNAP 1 for the same reason given above that these are elevated emissions. The sensitivity to NOx emissions from SNAP 7 is most pronounced close to the source of emissions (i.e. major roads and cities).

The similarity in the spatial distribution of sensitivity indices for O3 and NO2 model outputs results from the concentrations of these pollutants being inversely correlated, as their chemical transformation reactions are interlinked. In the same way as for O3, uncertainty in the NO2 concentrations along the south and south-east coasts of England is mostly driven by uncertainty in shipping emissions. In fact, uncertainty in shipping emissions contributes approximately 30 % of uncertainty in NO2 concentrations, even well inland in areas away from major roads and cities.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f05

Figure 5Spatial distributions (at the 5 km × 5 km model grid resolution) of the first-order sensitivity indices for modelled surface concentrations of PM2.5. D indicates emissions from a dominant sector and O indicates grouped emissions from the rest of the sectors. For NH3 emissions the dominant sector is agriculture, and for PM2.5 the dominant sectors are residential and non-industrial combustion (D1) and road transport (D2). The shipping emissions variable combines emissions of all relevant pollutants.

Download

Figure 5 shows the spatial distribution of first-order sensitivity indexes for the model inputs that contribute to the uncertainty in modelled surface concentrations of PM2.5. Modelled PM2.5 is sensitive to all emissions of NH3 (the dominant sector is agriculture), to primary PM2.5 (the dominant sector D1 is residential combustion and D2 is road transport), and to shipping emissions. In areas with lower surface PM2.5 concentrations such as Scotland, Wales, northern England, and south-west England, the uncertainty is mainly driven by NH3 emissions from agriculture (NH3_D). The spatial pattern of emission sensitivity indices for PM2.5 mirrors the spatial distribution of PM2.5 emission sources. From Figs. 2 and 5 it can be seen that in areas with the highest levels of uncertainty the model output is most sensitive to the emissions of primary PM2.5. Similar to the results for O3 and NO2, the areas with the highest uncertainty coincide with the most populated areas.

The pattern in calculated sensitivity indices partially agrees with a previous study of changes in PM2.5 surface concentrations in response to a 30 % reduction in emissions of PM2.5, NH3, SOx, NOx, and VOC by Vieno et al. (2016). In the study by Vieno et al. (2016) surface concentrations of PM2.5 were found to be sensitive to reductions in each of the five pollutants individually (the same reduction was applied to a pollutant's emissions from all SNAP sectors simultaneously), with the highest sensitivity to NH3 and PM2.5 emissions (up to an approximately 6 % reduction in surface concentration in response to 30 % reduction in emissions). In comparison, in our study the uncertainty in PM2.5 surface concentrations is not affected by the perturbations of SOx, NOx, and VOC. This is likely due to (i) the difference in ranges of variation (i.e. uncertainty ranges) in this study (SOx, NOx, and VOC input variables have narrower ranges of variation compared to PM2.5 and NH3) and (ii) the presence of non-additivity and non-linearity in the model response to perturbations in the inputs.

3.3 Uncertainty propagation and sensitivity analysis for monthly averaged model outputs

Uncertainty assessment and sensitivity analyses for monthly averaged surface concentrations of NO2, O3, and PM2.5 were performed for five different grid cells that were assigned the following environment types based on the site type attributed to the national-network monitoring site within that grid cell: Auchencorth Moss and Harwell – rural background; Birmingham Acocks Green and London N. Kensington – urban background; and London Marylebone Road – urban traffic.

Monthly average concentrations with error bars representing the absolute uncertainty values (as a 95 % CI) are presented in Fig. 6. Figure 7 shows corresponding values of the relative uncertainty. Figure 8 shows how the magnitude of first-order sensitivity indices estimated for five different grid cells changes on a monthly timescale. If all first-order sensitivity coefficients add up to 1 then there are no interactions between inputs and all model variance can be apportioned to the variance in the individual inputs.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f06

Figure 6Monthly average surface concentrations of NO2, O3, and PM2.5, with error bars showing (absolute) uncertainty, for five grid cells across the UK representing a spread of geographical locations and environment types. The environment types are assigned as follows: Auchencorth Moss and Harwell – rural background; Birmingham Acocks Green and London N. Kensington – urban background; and London Marylebone Road – urban traffic.

Download

The NO2 surface concentrations show a seasonal trend of lower concentrations occurring during summer months with the exception of the Auchencorth Moss grid cell in which NO2 concentrations are low throughout the year. The magnitude of uncertainty in NO2 is proportional to the modelled concentration and changes relative to the concentration, which can be seen from the monthly relative uncertainty values (Fig. 7). The first-order sensitivity indices for NO2 show that only NOx emissions (across all sectors) and shipping emissions influence the modelled surface NO2 concentrations. Hence, it can be concluded that the uncertainty in modelled concentrations of NO2 directly depends on the uncertainty in NOx emissions and is not affected by the uncertainties in the emissions of any other pollutant. The change in the magnitude of sensitivity coefficients for the Harwell grid cell indicates increasing influence of shipping emissions on NO2 concentrations during the summer months. A potential explanation for this is a seasonal change in the wind direction, which results in more NOx from shipping emissions being transported to the grid cell during the summer months (the wind rose is presented in Fig. S3).

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f07

Figure 7Magnitude of relative uncertainty in monthly average surface concentrations of NO2, O3, and PM2.5 for five grid cells across the UK representing a spread of geographical locations and environment types. The environment types are assigned as follows: Auchencorth Moss and Harwell – rural background; Birmingham Acocks Green and London N. Kensington – urban background; and London Marylebone Road – urban traffic.

Download

The uncertainties in the O3 modelled surface concentrations show an inverse seasonal trend compared to the uncertainties in modelled NO2. Unlike the uncertainty in the NO2 concentration, the uncertainty in the O3 concentration is influenced by the grid cell environment type; the highest level of uncertainty is observed for the London Marylebone Road grid cell (urban traffic). The relative uncertainty in O3 concentrations for the Auchencorth Moss grid cell (rural background) is small and close to the median relative uncertainty in O3 for annual average concentrations, which as discussed above is ±0.6 %. This indicates that perturbations in the input emissions do not substantially affect the O3 concentration in this grid cell. Although the magnitude of uncertainty in O3 is very small in this grid cell, the inputs that drive it differ noticeably throughout the year; during May–August the variance is mostly explained by VOC emissions (explains 77 % of uncertainty for July) and during November–February NOx emissions drive the uncertainty. The magnitude of O3 concentrations and corresponding uncertainties in the Birmingham Acocks Green and Harwell grid cells is very similar. The trends in sensitivity indices are also similar; during the April–September period some variance in the model output is explained by uncertainty in VOC emissions. However, in the Harwell grid cell shipping emissions play a more important role. For the London-based grid cells, the level of uncertainty is the highest and is mainly driven by the uncertainty in NOx and shipping emissions.

For the PM2.5 monthly average concentrations, London-based grid cells show the highest values of absolute uncertainty and Auchencorth Moss the lowest. The relative uncertainty in London-based grid cells is also the highest. From Fig. 7 it can be seen that the contribution to the overall uncertainty from the uncertainty due to NH3 emissions for these grid cells is not as important as for other three; the majority of uncertainty is explained by the uncertainty in the primary PM2.5 emissions with PM2.5 from road transport being the dominating variable. In Birmingham Acocks Green and Harwell, the effect of NH3 emissions from agricultural sources is more pronounced; from 30 % to 70 % of overall uncertainty in PM2.5 can be apportioned to uncertainty coming from agricultural emissions of NH3 during spring and summer months.

https://www.atmos-chem-phys.net/19/2881/2019/acp-19-2881-2019-f08

Figure 8Monthly variation in the first-order sensitivity indices for five grid cells across the UK representing a spread of geographical locations and environment types. Based on the monitoring station classification, grid squares are assigned the following environment types: Auchencorth Moss and Harwell – rural background; Birmingham Acocks Green and London N. Kensington – urban background; and London Marylebone Road – urban traffic.

Download

3.4 Wider implications of our study

There are published studies that apply global sampling-based uncertainty and sensitivity analyses as well as derivative-based methods (methods that do not have limitations of local OAT, i.e. linearity assumption) to ACTMs. However, the results reported by these studies are mostly of use for model development and calibration purposes and not the assessment of confidence in the model predictions and outputs. This is mainly because the simulations are performed for a short period ranging from days (Rodriguez et al., 2007; Chen and Brune, 2012; Beddows et al., 2017) to weeks (Cohan et al., 2010; Shrivastava et al., 2016).

Additionally, in some studies, commercial software or packages with a graphical user interface (GUI) are used for global sensitivity and uncertainty analysis (Lee et al., 2011; Chen and Brune, 2012; Christian et al., 2017). These tools are well designed for a specific purpose but lack the option to scale up and to automate the analysis, i.e. the ability to calculate sensitivity indices and uncertainty ranges for thousands of grid squares automatically.

Our study addresses both of the shortcomings. We demonstrate sensitivity and uncertainty analyses for the ACTM for a whole year for the UK domain as well as investigate variations in sensitivity and uncertainty on the monthly timescale for multiple locations with different environmental characteristics. Additionally, the package used to create Gaussian process emulators and to conduct uncertainty and sensitivity calculations is fully customisable and can be adapted for any application.

The model runs generated for the global sensitivity and uncertainty analysis can be utilised for other purposes provided that the sampling range for all inputs of interest is wide enough. For example, in our study the training points for the Gaussian emulator were selected to cover a wider range of input perturbations compared to the corresponding uncertainty range (Table 2). For all input emissions of SOx, NOx, VOC, and NH3 the ranges of variation for the LHS design were set to ±40 % of their baseline value; for primary PM2.5 emissions the range was set to ±75 % and for shipping emissions from −40 % to +100 %. Hence, the emulators created in this study using the model runs within the aforementioned input space can be used to investigate other scenarios of the model response to input emission perturbations with no extra computational cost. Hence, alternative ranges and probability distributions can be assigned to the model inputs to estimate the resulting output uncertainty or the emulator can be used for various emission reduction scenario analyses.

Finally, in this study the overall model output uncertainty is likely to be lower than the theoretical total model output uncertainty, as in addition to the input emissions there are a variety of other uncertain model inputs. Assessing the effect of variation in every model input and parameter on the model output is a laborious task; hence, ideally sensitivity analysis should be incorporated as a part of the model development process. By using this approach, the effect of all uncertain inputs and parameters could be assessed without having to do it retrospectively.

4 Conclusions

In this study, we have conducted global sensitivity and uncertainty analyses for the EMEP4UK Eulerian atmospheric chemistry transport model to quantify the uncertainty in surface concentrations of O3, NO2, and PM2.5 and to identify the input emission variables that contribute the most to the uncertainty in each of the outputs. The uncertainty for model outputs was estimated from the uncertainties assigned to the UK emissions of SO2, NOx, NH3, VOC, and primary PM2.5 and documented in the UK National Atmospheric Emissions Inventory. The benefit of conducting global sensitivity analysis in addition to uncertainty assessment is that it allows us to determine how a model responds to input perturbations within the ranges set by the input uncertainty estimates and consequently to identify the inputs which cause variation in the model outputs (i.e. drive the uncertainty). The median values of the overall uncertainty calculated for the UK land-based grid cells for annual average surface concentrations of O3, NO2, and PM2.5 were found to be in the ranges of ±0.6 %, ±7.4 %, and ±4.6 %, respectively. This indicates that variation in the input data (i.e. emissions) does not cause a substantial variation in outputs. Our results indicate that this can likely be explained by variations in the other model input parameters such as chemical reaction rates, deposition velocities, or physical constant values which might cause more variation in the model outputs. Alternatively, the surface concentrations of the modelled pollutants in the UK may be dominated by precursor emissions and long-range transport from outside the UK and are therefore relatively insensitive to changes in the UK emissions.

As a consequence, our results can provide more clarity about the confidence in modelled surface concentrations of pollutants that affect human health, especially in densely populated urban areas. The results of our analysis indicate that modelled surface concentrations of O3, NO2, and PM2.5 have the highest level of uncertainty in the grid cells comprising dense urban areas. The uncertainties of O3, NO2, and PM2.5 in these grid cells reach ±7 %, ±9 %, and ±9 %, respectively.

In addition to obtaining a quantitative estimate of the overall uncertainty, the input emissions that have the greatest influence on the uncertainty in the modelled outputs were identified by performing a global variance-based sensitivity analysis. It was found that in urban areas uncertainty in PM2.5 concentrations is driven by the uncertainty in primary PM2.5 emissions. In contrast, in more remote areas NH3 emissions had a stronger influence. Emissions of NOx combined from non-dominant sectors (i.e. all sectors excluding energy production and road transport) were found to contribute the most to the uncertainty in both O3 and NO2 surface concentrations. Along the south and east coasts of England the uncertainty in shipping emissions contributed the most to the overall uncertainty in O3 and NO2 concentrations.

The comparison between first- and total-order sensitivity indices did not indicate substantial interactions between the input variables for the model response on the annual timescale.

In our study we also demonstrated how the degree of uncertainty changes throughout the year by calculating uncertainty ranges for monthly averaged surface concentrations of O3, NO2, and PM2.5 for five selected grid cells. The global sensitivity conducted for monthly averaged values showed seasonal trends in the type of input emissions that drive uncertainty in the surface concentrations.

The ability to estimate uncertainty in the predictions produced by a model is vital because even low levels of uncertainty could be important in areas where the model yields predictions of surface concentrations that are close to limit values. This can lead to instances of exceedance due to the binary nature of limit value exceedance calculations, i.e. the concentration is either over or under the limit. Sensitivity analysis should be an integral part of the assessment process applied ex ante for the implementation of policy interventions, as it is also important to know which of the inputs contribute to the uncertainty in model outputs the most.

This work has demonstrated a global sensitivity and uncertainty analysis application for an Eulerian ACTM. The emulator-based approach used here is applicable to any other complex model and any type of model input such as emissions, physical constants, or chemical reaction rate constants. The results of the analyses provide useful insights into the level of confidence in modelled predictions. Additionally, the Gaussian process emulators created for this analysis can be used with very little computational cost for any other scenario exploration purposes or assessment of overall uncertainty given different uncertainty ranges and probability distributions assigned to the model inputs.

Code and data availability

The EMEP MSC-W model code is available from https://github.com/metno/emep-ctm (last access: 13 February 2019) (Simpson et al., 2012). The uncertainty and sensitivity data presented in this paper are available from https://doi.org/10.5281/zenodo.2213633 (last access: 28 February 2019) (Aleksankina, 2018) together with the analysis scripts.

Supplement

The supplement related to this article is available online at: https://doi.org/10.5194/acp-19-2881-2019-supplement.

Author contributions

KA created the experimental design, performed the model runs and the data analysis, and drafted the initial paper. MRH and SR assisted with data analysis and interpretation and contributed to the discussion of the results and writing the paper. MV assisted with the initial model set-up and contributed to paper editing. All authors approved the final paper.

Competing interests

The authors declare that they have no conflict of interest.

Acknowledgements

Ksenia Aleksankina acknowledges studentship funding from the University of Edinburgh and the NERC Centre for Ecology & Hydrology. This work was supported by the Natural Environment Research Council award number NE/R016429/1 as part of the UK-SCaPE programme for National Capability.

Edited by: Robert Harley
Reviewed by: two anonymous referees

References

Air Quality Expert Group: Mitigation of United Kingdom PM 2.5 Concentrations, available at: https://uk-air.defra.gov.uk/assets/documents/reports/cat11/1508060903_DEF-PB14161_Mitigation_of_UK_PM25.pdf (last access: 15 May 2018), 2013. 

Aleksankina, K.: Advanced methods for uncertainty assessment and global sensitivity analysis of a Eulerian atmospheric chemistry transport model [Data set], Zenodo, available at: https://doi.org/10.5281/zenodo.2213633, 2018. 

Aleksankina, K., Heal, M. R., Dore, A. J., Van Oijen, M., and Reis, S.: Global sensitivity and uncertainty analysis of an atmospheric chemistry transport model: the FRAME model (version 9.15.0) as a case study, Geosci. Model Dev., 11, 1653–1664, https://doi.org/10.5194/gmd-11-1653-2018, 2018. 

Asher, M. J., Croke, B. F. W., Jakeman, A. J., and Peeters, L. J. M.: A review of surrogate models and their application to groundwater modeling, Water Resour. Res., 51, 5957–5973, https://doi.org/10.1002/2015WR016967, 2015. 

Beddows, A. V., Kitwiroon, N., Williams, M. L., and Beevers, S. D.: Emulation and Sensitivity Analysis of the Community Multiscale Air Quality Model for a UK Ozone Pollution Episode, Environ. Sci. Technol., 51, 6229–6236, https://doi.org/10.1021/acs.est.6b05873, 2017. 

Carslaw, K. S., Lee, L. A., Reddington, C. L., Pringle, K. J., Rap, A., Forster, P. M., Mann, G. W., Spracklen, D. V., Woodhouse, M. T., Regayre, L. A., and Pierce, J. R.: Large contribution of natural aerosols to uncertainty in indirect forcing, Nature, 503, 67–71, https://doi.org/10.1038/nature12674, 2013. 

Castelletti, A., Galelli, S., Ratto, M., Soncini-Sessa, R., and Young, P. C.: A general framework for Dynamic Emulation Modelling in environmental problems, Environ. Modell. Softw., 34, 5–18, https://doi.org/10.1016/j.envsoft.2012.01.002, 2012. 

Chen, S. and Brune, W. H.: Global sensitivity analysis of ozone production and O3NOx–VOC limitation based on field data, Atmos. Environ., 55, 288–296, https://doi.org/10.1016/j.atmosenv.2012.03.061, 2012. 

Chen, S., Brune, W. H., Lambe, A. T., Davidovits, P., and Onasch, T. B.: Modeling organic aerosol from the oxidation of α-pinene in a Potential Aerosol Mass (PAM) chamber, Atmos. Chem. Phys., 13, 5017–5031, https://doi.org/10.5194/acp-13-5017-2013, 2013. 

Christian, K. E., Brune, W. H., and Mao, J.: Global sensitivity analysis of the GEOS-Chem chemical transport model: ozone and hydrogen oxides during ARCTAS (2008), Atmos. Chem. Phys., 17, 3769–3784, https://doi.org/10.5194/acp-17-3769-2017, 2017. 

Cohan, D. S., Koo, B., and Yarwood, G.: Influence of uncertain reaction rates on ozone sensitivity to emissions, Atmos. Environ., 44, 3101–3109, https://doi.org/10.1016/j.atmosenv.2010.05.034, 2010. 

Corbett, J. J.: Updated emissions from ocean shipping, J. Geophys. Res., 108, 4650, https://doi.org/10.1029/2003JD003751, 2003. 

Dean, A., Morris, M., Stufken, J., and Bingham, D.: Handbook of Design and Analysis of Experiments, Chapman and Hall/CRC, New York, USA, 2015. 

Dore, A. J., Carslaw, D. C., Braban, C., Cain, M., Chemel, C., Conolly, C., Derwent, R. G., Griffiths, S. J., Hall, J., Hayman, G., Lawrence, S., Metcalfe, S. E., Redington, A., Simpson, D., Sutton, M. A., Sutton, P., Tang, Y. S., Vieno, M., Werner, M., and Whyatt, J. D.: Evaluation of the performance of different atmospheric chemical transport models and inter-comparison of nitrogen and sulphur deposition estimates for the UK, Atmos. Environ., 119, 131–143, https://doi.org/10.1016/j.atmosenv.2015.08.008, 2015. 

EC Directive: Directive 2008/50/EC of the European Parliament and of the Council of 21 May 2008 on ambient air quality and cleaner air for Europe, available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32008L0050&from=en (last access: 1 May 2018), 2008. 

Entec: Quantification of emissions from ships associated with ship movements between ports in the European Community, available at: http://ec.europa.eu/environment/air/pdf/chapter2_ship_emissions.pdf (last access: 2 May 2018), 2002. 

Entec: UK Ship Emissions Inventory, available at: http://uk-air.defra.gov.uk/reports/cat15/1012131459_21897_Final_Report_291110.pdf (last access: 1 July 2018), 2010. 

Eurostat: NAMEA for Air Emissions Compilation Guide, Publisher-European Commission, EUROSTAT, 2004. 

Ferretti, F., Saltelli, A., and Tarantola, S.: Trends in sensitivity analysis practice in the last decade, Sci. Total Environ., 568, 666–670, https://doi.org/10.1016/j.scitotenv.2016.02.133, 2015. 

Frost, G. J., Middleton, P., Tarrasón, L., Granier, C., Guenther, A., Cardenas, B., Denier van der Gon, H., Janssens-Maenhout, G., Kaiser, J. W., Keating, T., Klimont, Z., Lamarque, J. F., Liousse, C., Nickovic, S., Ohara, T., Schultz, M. G., Skiba, U., Van Aardenne, J., and Wang, Y.: New Directions: GEIA's 2020 vision for better air emissions information, Atmos. Environ., 81, 710–712, https://doi.org/10.1016/j.atmosenv.2013.08.063, 2013. 

Ghanem, R., Higdon, D., and Owhadi, H.: Handbook of Uncertainty Quantification, edited by: Ghanem, R., Higdon, D., and Owhadi, H., Springer International Publishing, Switzerland, 2017. 

Gladish, D. W., Pagendam, D. E., Peeters, L. J. M., Kuhnert, P. M., and Vaze, J.: Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models, J. Agric. Biol. Envir. S., 23, 39–62, https://doi.org/10.1007/s13253-017-0308-3, 2017. 

Grini, A., Myhre, G., Zender, C. S., and Isaksen, I. S. A.: Model simulations of dust sources and transport in the global atmosphere: Effects of soil erodibility and wind speed variability, J. Geophys. Res., 110, D02205, https://doi.org/10.1029/2004JD005037, 2005. 

Guenther, A., Hewitt, C. N., Erickson, D., Fall, R., Geron, C., Graedel, T., Harley, P., Klinger, L., Lerdau, M., Mckay, W. A., Pierce, T., Scholes, B., Steinbrecher, R., Tallamraju, R., Taylor, J., and Zimmerman, P.: A global model of natural volatile organic compound emissions, J. Geophys. Res., 100, 8873, https://doi.org/10.1029/94JD02950, 1995. 

Guenther, A. B., Zimmerman, P. R., Harley, P. C., Monson, R. K., and Fall, R.: Isoprene and monoterpene emission rate variability: Model evaluations and sensitivity analyses, J. Geophys. Res., 98, 12609, https://doi.org/10.1029/93JD00527, 1993. 

Hanna, S. R., Paine, R., Heinold, D., Kintigh, E. and Baker, D.: Uncertainties in air toxics calculated by the dispersion models AERMOD and ISCST3 in the Houston ship channel area, J. Appl. Meteorol. Clim., 46, 1372–1382, https://doi.org/10.1175/JAM2540.1, 2007. 

Homma, T. and Saltelli, A.: Importance measures in global sensitivity analysis of nonlinear models, Reliab. Eng. Syst. Safe., 52, 1–17, https://doi.org/10.1016/0951-8320(96)00002-6, 1996. 

Iooss, B. and Lemaître, P.: A Review on Global Sensitivity Analysis Methods, in: Uncertainty management in Simulation-Optimization of Complex Systems: Algorithms and Applications, Springer US, New York, 101–122, 2015. 

IPCC: IPCC Guidelines for National Greenhouse Gas Inventories, General Guidance and Reporting, available at: https://www.ipcc-nggip.iges.or.jp/public/2006gl/pdf/1_Volume1/V1_3_Ch3_Uncertainties.pdf (last access: 2 February 2018), 2006. 

IPCC: Climate Change 2013: The Physical Science Basis, Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge, UK and New York, NY, USA, 2013. 

Janon, A., Klein, T., Lagnoux, A., Nodet, M., and Prieur, C.: Asymptotic normality and efficiency of two Sobol index estimators, ESAIM-Probab. Stat., 18, 342–364, https://doi.org/10.1051/ps/2013040, 2014. 

Johnson, M. E., Moore, L. M., and Ylvisaker, D.: Minimax and maximin distance designs, J. Stat. Plan. Infer., 26, 131–148, https://doi.org/10.1016/0378-3758(90)90122-B, 1990. 

Jones, B. and Johnson, R. T.: Design and analysis for the Gaussian process model, Qual. Reliab. Eng. Int., 25, 515–524, https://doi.org/10.1002/qre.1044, 2009. 

Lataniotis, C., Marelli, S., and Sudret, B.: Gaussian process modelling using UQLab, available at: http://arxiv.org/abs/1709.09382 (last access: 15 February 2019), 2017. 

Lee, L. A., Carslaw, K. S., Pringle, K. J., Mann, G. W., and Spracklen, D. V.: Emulation of a complex global aerosol model to quantify sensitivity to uncertain parameters, Atmos. Chem. Phys., 11, 12253–12273, https://doi.org/10.5194/acp-11-12253-2011, 2011. 

Lin, C., Heal, M. R., Vieno, M., MacKenzie, I. A., Armstrong, B. G., Butland, B. K., Milojevic, A., Chalabi, Z., Atkinson, R. W., Stevenson, D. S., Doherty, R. M., and Wilkinson, P.: Spatiotemporal evaluation of EMEP4UK-WRF v4.3 atmospheric chemistry transport simulations of health-related metrics for NO2, O3, PM10, and PM2.5 for 2001–2010, Geosci. Model Dev., 10, 1767–1787, https://doi.org/10.5194/gmd-10-1767-2017, 2017. 

Luo, Z., Wang, E., Bryan, B. A., King, D., Zhao, G., Pan, X., and Bende-Michl, U.: Meta-modeling soil organic carbon sequestration potential and its application at regional scale, Ecol. Appl., 23, 408–420, https://doi.org/10.1890/12-0672.1, 2013. 

Marelli, S. and Sudret, B.: UQLab: A Framework for Uncertainty Quantification in Matlab, in Vulnerability, Uncertainty, and Risk, American Society of Civil Engineers, Reston, VA, USA, 2554–2563, 2014. 

Mårtensson, E. M., Nilsson, E. D., de Leeuw, G., Cohen, L. H., and Hansson, H.-C.: Laboratory simulations and parameterization of the primary marine aerosol production, J. Geophys. Res.-Atmos., 108, https://doi.org/10.1029/2002JD002263, 2003. 

McKay, M. D., Beckman, R. J., and Conover, W. J.: Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code, Technometrics, 21, 239–245, https://doi.org/10.1080/00401706.1979.10489755, 1979. 

Monahan, E. C., Spiel, D. E., and Davidson, K. L.: A Model of Marine Aerosol Generation Via Whitecaps and Wave Disruption, Springer, Dordrecht, the Netherlands, 167–174, 1986. 

Moreno-Gutiérrez, J., Calderay, F., Saborido, N., Boile, M., Rodríguez Valero, R., and Durán-Grados, V.: Methodologies for estimating shipping emissions and energy consumption: A comparative analysis of current methods, Energy, 86, 603–616, https://doi.org/10.1016/j.energy.2015.04.083, 2015. 

Morris, M. D. and Mitchell, T. J.: Exploratory designs for computational experiments, J. Stat. Plan. Infer., 43, 381–402, https://doi.org/10.1016/0378-3758(94)00035-T, 1995. 

NCEP: NCEP FNL Operational Model Global Tropospheric Analyses, continuing from July 1999, Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, Boulder, CO, USA, 2000. 

Oakley, J. and O'Hagan, A.: Bayesian inference for the uncertainty distribution of computer model outputs, Biometrika, 89, 769–784, 2002. 

Oakley, J. E. and O'Hagan, A.: Probabilistic sensitivity analysis of complex models: A Bayesian approach, J. R. Stat. Soc. B, 66, 751–769, https://doi.org/10.1111/j.1467-9868.2004.05304.x, 2004. 

O'Hagan, A.: Bayesian analysis of computer code outputs: A tutorial, Reliab. Eng. Syst. Safe., 91, 1290–1300, https://doi.org/10.1016/j.ress.2005.11.025, 2006. 

Parry, H. R., Topping, C. J., Kennedy, M. C., Boatman, N. D., and Murray, A. W. A.: A Bayesian sensitivity analysis applied to an Agent-based model of bird population response to landscape change, Environ. Modell. Softw., 45, 104–115, https://doi.org/10.1016/j.envsoft.2012.08.006, 2013. 

Pulles, T. and Kuenen, J.: EMEP/EEA air pollutant emission inventory guidebook, available at: https://www.eea.europa.eu/publications/emep-eea-guidebook-2016 (last access: 1 November 2018), 2016. 

Rabitz, H. and Alış, Ö. F.: General foundations of high-dimensional model representations, J. Math. Chem., 25, 197–233, https://doi.org/10.1023/A:1019188517934, 1999. 

Ratto, M., Castelletti, A., and Pagano, A.: Emulation techniques for the reduction and sensitivity analysis of complex environmental models, Environ. Modell. Softw., 34, 1–4, https://doi.org/10.1016/j.envsoft.2011.11.003, 2012. 

Rodriguez, M. A., Brouwer, J., Samuelsen, G. S., and Dabdub, D.: Air quality impacts of distributed power generation in the South Coast Air Basin of California 2: Model uncertainty and sensitivity analysis, Atmos. Environ., 41, 5618–5635, https://doi.org/10.1016/j.atmosenv.2007.02.049, 2007. 

Rypdal, K. and Winiwarter, W.: Uncertainties in greenhouse gas emission inventories – evaluation, comparability and implications, Environ. Sci. Policy, 4, 107–116, https://doi.org/10.1016/S1462-9011(00)00113-1, 2001. 

Saltelli, A.: Making best use of model evaluations to compute sensitivity indices, Comput. Phys. Commun., 145, 280–297, https://doi.org/10.1016/S0010-4655(02)00280-1, 2002. 

Saltelli, A. and Annoni, P.: How to avoid a perfunctory sensitivity analysis, Environ. Modell. Softw., 25, 1508–1517, https://doi.org/10.1016/j.envsoft.2010.04.012, 2010. 

Saltelli, A., Ratto, M., Tarantola, S. and Campolongo, F.: Sensitivity analysis practice: A guide to scientific models, John Wiley & Sons Ltd., Chichester, England, 109–151, 2006. 

Sax, T. and Isakov, V.: A case study for assessing uncertainty in local-scale regulatory air quality modeling applications, Atmos. Environ., 37, 3481–3489, https://doi.org/10.1016/S1352-2310(03)00411-4, 2003. 

Scarbrough, T., Tsagatakis, I., Smith, K., Wakeling, D., Smith, T., Hauerhoff, E., and Murrells, T.: A review of the NAEI shipping emissions methodology, available at: https://uk-air.defra.gov.uk/assets/documents/reports/cat07/1712140936_ED61406_NAEI_shipping_report_12Dec2017.pdf (last access: 15 March 2018), 2017. 

Sheikholeslami, R. and Razavi, S.: Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models, Environ. Modell. Softw., 93, 109–126, https://doi.org/10.1016/j.envsoft.2017.03.010, 2017. 

Shrivastava, M., Zhao, C., Easter, R. C., Qian, Y., Zelenyuk, A., Fast, J. D., Liu, Y., Zhang, Q., and Guenther, A.: Sensitivity analysis of simulated SOA loadings using a variance-based statistical approach, J. Adv. Model. Earth Sy., 8, 499–519, https://doi.org/10.1002/2015MS000554, 2016. 

Simpson, D., Benedictow, A., Berge, H., Bergström, R., Emberson, L. D., Fagerli, H., Flechard, C. R., Hayman, G. D., Gauss, M., Jonson, J. E., Jenkin, M. E., Nyíri, A., Richter, C., Semeena, V. S., Tsyro, S., Tuovinen, J.-P., Valdebenito, Á., and Wind, P.: The EMEP MSC-W chemical transport model – technical description, Atmos. Chem. Phys., 12, 7825–7865, https://doi.org/10.5194/acp-12-7825-2012, 2012 (data available at: https://github.com/metno/emep-ctm, last access: 13 February 2019). 

Skamarock, W., Klemp, J., Dudhia, J., Gill, D., Barker, D., Duda, M., Huang, X., Wang, W., and Powers, J.: A Description of the Advanced Research WRF Version 3, NCAR technical note NCAR/TN-475+STR., 2008. 

Skeffington, R. A., Whitehead, P. G., Heywood, E., Hall, J. R., Wadsworth, R. A., and Reynolds, B.: Estimating uncertainty in terrestrial critical loads and their exceedances at four sites in the UK, Sci. Total Environ., 382, 199–213, https://doi.org/10.1016/j.scitotenv.2007.05.001, 2007. 

Sobol', I.: Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates, Math. Comput. Simulat., 55, 271–280, https://doi.org/10.1016/S0378-4754(00)00270-6, 2001. 

Sobol', I. M.: Sensitivity estimates for nonlinear mathematical models, Math. Model. Comput. Exp., 1, 407–414, 1993. 

Stevenson, D. S., Young, P. J., Naik, V., Lamarque, J.-F., Shindell, D. T., Voulgarakis, A., Skeie, R. B., Dalsoren, S. B., Myhre, G., Berntsen, T. K., Folberth, G. A., Rumbold, S. T., Collins, W. J., MacKenzie, I. A., Doherty, R. M., Zeng, G., van Noije, T. P. C., Strunk, A., Bergmann, D., Cameron-Smith, P., Plummer, D. A., Strode, S. A., Horowitz, L., Lee, Y. H., Szopa, S., Sudo, K., Nagashima, T., Josse, B., Cionni, I., Righi, M., Eyring, V., Conley, A., Bowman, K. W., Wild, O., and Archibald, A.: Tropospheric ozone changes, radiative forcing and attribution to emissions in the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP), Atmos. Chem. Phys., 13, 3063–3085, https://doi.org/10.5194/acp-13-3063-2013, 2013. 

Storlie, C. B. and Helton, J. C.: Multiple predictor smoothing methods for sensitivity analysis: Description of techniques, Reliab. Eng. Syst. Safe., 93, 28–54, https://doi.org/10.1016/J.RESS.2006.10.012, 2008. 

Storlie, C. B., Swiler, L. P., Helton, J. C., and Sallaberry, C. J.: Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models, Reliab. Eng. Syst. Safe., 94, 1735–1763, https://doi.org/10.1016/j.ress.2009.05.007, 2009. 

Sudret, B.: Global sensitivity analysis using polynomial chaos expansions, Reliab. Eng. Syst. Safe., 93, 964–979, https://doi.org/10.1016/j.ress.2007.04.002, 2008. 

Teixeira, E., Fischer, G., van Velthuizen, H., van Dingenen, R., Dentener, F., Mills, G., Walter, C., and Ewert, F.: Limited potential of crop management for mitigating surface ozone impacts on global food supply, Atmos. Environ., 45, 2569–2576, https://doi.org/10.1016/j.atmosenv.2011.02.002, 2011. 

Urban, N. M. and Fricker, T. E.: A comparison of Latin hypercube and grid ensemble designs for the multivariate emulation of an Earth system model, Comput. Geosci., 36, 746–755, https://doi.org/10.1016/j.cageo.2009.11.004, 2010. 

Vieno, M., Dore, A. J., Stevenson, D. S., Doherty, R., Heal, M. R., Reis, S., Hallsworth, S., Tarrason, L., Wind, P., Fowler, D., Simpson, D., and Sutton, M. A.: Modelling surface ozone during the 2003 heat-wave in the UK, Atmos. Chem. Phys., 10, 7963–7978, https://doi.org/10.5194/acp-10-7963-2010, 2010. 

Vieno, M., Heal, M. R., Hallsworth, S., Famulari, D., Doherty, R. M., Dore, A. J., Tang, Y. S., Braban, C. F., Leaver, D., Sutton, M. A., and Reis, S.: The role of long-range transport and domestic emissions in determining atmospheric secondary inorganic particle concentrations across the UK, Atmos. Chem. Phys., 14, 8435–8447, https://doi.org/10.5194/acp-14-8435-2014, 2014. 

Vieno, M., Heal, M. R., Williams, M. L., Carnell, E. J., Nemitz, E., Stedman, J. R., and Reis, S.: The sensitivities of emissions reductions for the mitigation of UK PM2.5, Atmos. Chem. Phys., 16, 265–276, https://doi.org/10.5194/acp-16-265-2016, 2016a. 

Vieno, M., Heal, M. R., Twigg, M. M., MacKenzie, I. A., Braban, C. F., Lingard, J. J. N., Ritchie, S., Beck, R. C., Móring, A., Ots, R., Di Marco, C. F., Nemitz, E., Sutton, M. A., and Reis, S.: The UK particulate matter air pollution episode of March–April 2014: more than Saharan dust, Environ. Res. Lett., 11, 044004, https://doi.org/10.1088/1748-9326/11/4/044004, 2016b. 

Wakeling, D., Passant, N. R., Murrells, T. P., Pang, Y., Thistlethwaite, G., Walker, C., Garcia, J. M. R., Webb, J., Brown, P., Vento, S. del, Misra, A., Hobson, M., Pridmore, A., Dore, C., and Misselbrook, T.: UK Informative Inventory Report (1990 to 2014), DEFRA, UK, 65–73, 2016. 

Wakeling, D., Passant, N., Murrells, T., Pang, Y., Thistlethwaite, G., Walker, C., Brown, P., Vento, S. del, Hunter, R., Wiltshire, J., Broomfield, M., Watterson, J., Pearson, B., Rushton, K., Hobson, M., Smith, H., and Misselbrook, T.: UK Informative Inventory Report (1990 to 2015), DEFRA, UK, 67–76, 2017. 

WHO: Air quality guidelines, Global update 2005, Particulate matter, ozone, nitrogen dioxide and sulfur dioxide, World Health Organisation Regional Office for Europe, Copenhagen, Denmark, 2006. 

WHO: Health risks of air pollution in Europe – HRAPIE project, Copenhagen, available at: http://www.euro.who.int/data/assets/pdf_file/0006/238956/Health_risks_air_pollution_HRAPIE_project.pdf (last access: 3 March 2019), 2013. 

Yang, J.: Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis, Environ. Modell. Softw., 26, 444–457, https://doi.org/10.1016/j.envsoft.2010.10.007, 2011. 

Ziehn, T. and Tomlin, A. S.: GUI–HDMR – A software tool for global sensitivity analysis of complex models, Environ. Modell. Softw., 24, 775–785, https://doi.org/10.1016/j.envsoft.2008.12.002, 2009. 

Download
Short summary
Atmospheric chemistry transport models are widely used to underpin policies to mitigate the detrimental effects of air pollution on human health and ecosystems. Understanding the level of confidence in model predictions is thus vital. We present a comprehensive approach for uncertainty assessment and global variance-based sensitivity analysis to propagate uncertainty from model input data and identify the extent to which uncertainty in different emissions drives the model output uncertainty.
Altmetrics
Final-revised paper
Preprint