ACPAtmospheric Chemistry and PhysicsACPAtmos. Chem. Phys.1680-7324Copernicus PublicationsGöttingen, Germany10.5194/acp-17-9035-2017An improved hydrometeor detection method for millimeter-wavelength cloud
radarGeJinmingZhuZeenZhengChuangXieHailingZhouTianHuangJianpinghttps://orcid.org/0000-0003-2845-797XFuQiangqfu@atmos.washington.eduKey Laboratory for Semi-Arid Climate Change of the Ministry of
Education and College of Atmospheric Sciences, Lanzhou University, Lanzhou,
730000, ChinaDepartment of Atmospheric Sciences, University of Washington, Seattle,
WA 98105, USAQiang Fu (qfu@atmos.washington.edu)27July201717149035904721November20168December20166May201728June2017This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/3.0/This article is available from https://acp.copernicus.org/articles/17/9035/2017/acp-17-9035-2017.htmlThe full text article is available as a PDF file from https://acp.copernicus.org/articles/17/9035/2017/acp-17-9035-2017.pdf
A modified method with a new noise reduction scheme that can reduce
the noise distribution to a narrow range is proposed to distinguish clouds
and other hydrometeors from noise and recognize more features with weak
signal in cloud radar observations. A spatial filter with central weighting,
which is widely used in cloud radar hydrometeor detection algorithms, is also
applied in our method to examine radar return for significant levels of
signals. “Square clouds” were constructed to test our algorithm and the
method used for the US Department of Energy Atmospheric Radiation
Measurements Program millimeter-wavelength cloud radar. We also applied both
the methods to 6 months of cloud radar observations at the Semi-Arid
Climate and Environment Observatory of Lanzhou University and compared the
results. It was found that our method has significant advantages in reducing
the rates of both failed negative and false positive hydrometeor
identifications in simulated clouds and recognizing clouds with weak signal
from our cloud radar observations.
Introduction
Clouds, which are composed of liquid water droplets, ice crystals or both,
cover about two-thirds of the Earth surface at any time
(e.g., King et al., 2013). By reflecting solar radiation back to the
space (the albedo effect) and trapping thermal radiation emitted by the
Earth surface and the lower troposphere (the greenhouse effect), clouds
strongly modulate the radiative energy budget in the climate system
(e.g., Fu et al., 2002; Huang et al., 2006a, b, 2007; Ramanathan et al., 1989; Jing Su et al., 2008). Clouds are also
a vital component of water cycle by connecting the water-vapor condensation
and precipitation. Despite the importance of clouds in the climate system,
they are difficult to represent in climate models (e.g., Williams and Webb, 2009),
which causes the largest uncertainty in the predictions of climate change by
general circulation models (GCMs; e.g., Randall, 2007; Stephens, 2005; Williams and Webb, 2009).
Cloud formation, evolution and distribution are governed by complex physical
and dynamical processes on a wide range of scales from synoptic motions to
turbulence (Bony et al., 2015). Unfortunately, the processes that occur on
smaller spatial scales than a GCM grid box cannot be resolved by current
climate models, and the coupling between large-scale fluctuations and cloud
microphysical processes is not well understood (e.g., Huang et al., 2006b; Mace et al., 1998;
Yan et al., 2015; Yuan et al., 2006). Moreover, the cloud horizontal inhomogeneity and vertical
overlap are not resolved by GCMs (Barker, 2000; Barker and Fu, 2000; Fu et al., 2000a, b;
Huang et al., 2005; Li et al., 2015). To better understand cloud processes to improve their
parameterization in climate models and reveal their evolution in response
to climate change, long-term continuous observations of cloud fields in
terms of both macro- and microphysical properties are essential (e.g.,
Ackerman and Stokes, 2003; Sassen and Benson, 2001; Thorsen et al., 2011; Wang and Sassen, 2001).
Millimeter-wavelength cloud radars (MMCRs) can resolve cloud vertical
structure for their occurrences and microphysical properties (e.g.,
Clothiaux et al., 1995; Kollias et al., 2007a; Mace et al., 2001). The wavelengths of MMCRs are shorter than those of
weather radars, making them sensitive to cloud droplets and ice crystals
and able to penetrate multiple cloud layers (e.g., Kollias et al., 2007a).
Because of their outstanding advantages for cloud research,
millimeter-wavelength radars have been deployed on various research
platforms including the first space-borne millimeter-wavelength Cloud
Profiling Radar (CPR) onboard the CloudSat
(Stephens et al., 2002). Ground-based cloud radars are operated
at the US Department of Energy's Atmospheric Radiation Program (ARM)
observational sites (formerly MMCRs, now replaced with a new
generation of Ka-band zenith radar; KAZR; e.g., Ackerman and Stokes, 2003; Clothiaux et al., 1999, 2000;
Kollias et al., 2007b; Protat et al., 2011) and in Europe (Illingworth et al., 2007; Protat et al., 2009). In July 2013, a
KAZR was deployed in China at the Semi-Arid Climate and Environment
Observatory of Lanzhou University (SACOL) site (latitude of 35.946∘ N, longitude of 104.137∘ E; altitude of 1.97 km; Huang et al.,
2008), providing an opportunity to observe and reveal the detailed structure
and process of the midlatitude clouds over the semi-arid regions of East
Asia.
Before characterizing the cloud physical properties from the cloud radar
return signal, we first need to distinguish and extract the hydrometeor
signals from the background noise (i.e., cloud mask). A classical cloud mask
method was developed in Clothiaux et al. (2000, 1995) by analyzing the
strength and significance of returned signals. This method consists of two
main steps. First any power in a range gate that is greater than a mean
value of noise plus 1 standard deviation is selected as a bin containing
potential hydrometer signal. Second, a space–time coherent filter is
created to estimate the significance level of the potential hydrometer bin
signal to be real. This cloud mask algorithm is operationally used for the
ARM MMCRs data analysis and was later adopted to the CPR onboard the
CloudSat (Marchand et al., 2008).
It is recognized that by visually examining a cloud radar return image, one
can easily tell where the return power is likely to be caused by
hydrometeors and where the power is just from noise. This ability of the human
eye to extract and analyze information from an image has been broadly
studied in image processing and computer vision. A number of mathematical
methods for acquiring and processing information from images have been
developed, including some novel algorithms for noise reduction and edge
detection (Canny, 1986; He et al., 2013; Marr and Hildreth, 1980; Perona and Malik, 1990). In this paper, we propose a
modified cloud mask method for cloud radar by noticing that removing noise
from signal and identifying cloud boundaries are the essential goals of
cloud masking. This method reduces the radar noise while preserving cloud edges
by employing the bilateral filtering that is widely used in the image
processing (Tomasi and Manduchi, 1998). The power weighting probability method proposed
by Marchand et al. (2008) is also adopted in our method to prevent the
cloud corners from being removed. It is found that our improved hydrometeor
detection algorithm is efficient in terms of reducing false positives and
negatives as well as identifying cloud features with weak signals such as
thin cirrus clouds.
The KAZR deployed at the SACOL is described in Sect. 2 and the modified
cloud mask algorithm is introduced in Sect. 3. The applications of the new
scheme to both hypothetical and observed cloud fields including a comparison
with previous schemes are shown in Sect. 4. Summary and conclusions are
given in Sect. 5.
(a) Probability distribution function (PDF) of the noise power and
SNR from the KAZR observations on a clear day, 21 January 2014.
(b) Cumulative distribution function (CDF) of original and convolved SNR of the
noise on the clear day. (c) and (d) CDF of original and convolved SNR
of
a cloudy case on 4 January 2014 for range gates inside and outside the
cloud adjacent to the cloud boundary, respectively. The converted SNR is
obtained by using a 2-D Gaussian distribution kernel (Eq. 2).
(a) Comparison of original noise, reduced noise and hydrometeor
signal distributions. σo and σn are 1
standard deviation of the original and reduced background noise,
respectively. (b) Illustration of the bilateral filtering process: (b1) Gaussian kernel distribution in space, (b2) δ
function and
(b3) bilateral kernel by combining Gaussian kernel with δ
function.
The KAZR radar
The SACOL KAZR, built by ProSensing Inc. of Amherst, MA, is a
zenith-pointing cloud radar operating at approximately 35 GHz for the
dual-polarization measurements of Doppler spectra. The main purpose of the
KAZR is to provide vertical profiles of clouds by measuring the first three
Doppler moments: reflectivity, radial Doppler velocity and spectra width.
The linear depolarization ratio (LDR; Marr and Hildreth, 1980) can be computed from the
ratio of cross-polarized reflectivity to co-polarized reflectivity.
The SACOL KAZR has a transmitter with a peak power of 2.2 kw and two modes
working at separate frequencies. One is called “chirp” mode that uses a
linear frequency-modulation pulse compression to achieve high radar
sensitivity of about -65 dBZ at 5 km altitude. The minimum altitude (or
range) that can be detected in chirp mode is approximately 1 km a.g.l. To view
clouds below 1 km, a short pulse or “burst mode” pulse is transmitted at a
separate frequency just after transmission of the chirp pulse. This burst
mode pulse allows clouds as low as 200 m to be measured. The chirp pulse is
transmitted at 34.890 GHz while the burst pulse is transmitted at 34.830 GHz. These two waveforms are separated in the receiver and processed
separately.
The pulse length is approximately 300 ns (giving a range resolution of about
45 m), while the digital receiver samples the return signal every 30 m. The
inter-pulse period is 208.8 µs, the number of coherent averages is 1
and the number of the fast Fourier transform points is currently set to 512.
An unambiguous range is thus 31.29 km, an unambiguous velocity is 10.29 m s-1
and a velocity resolution is 0.04 m s-1. The signal dwell time is 4.27 s. These
operational parameters are set for the purpose of having enough radar
sensitivity and accurately acquiring reflectivities of hydrometeors. In this
study, we mainly use radar-observed reflectivity (dBZ) data to test our new
hydrometeor detection method.
Improved hydrometeor detection algorithm
The basic assumption in the former cloud mask algorithms (e.g.,
Clothiaux et al., 1995; Marchand et al., 2008) is that the random noise power follows the normal
distribution. Here clear-sky cases in all seasons from the KAZR observations
were first analyzed for its background noise power distributions. Figure 1a
shows an example of a clear-sky case from 00:00 to 12:00 UTC on 21 January 2014. The noise power is estimated from the top 30 range gates,
which includes both internal and external sources (Fukao and Hamazu, 2014). It has an
apparent non-Gaussian distribution with a positive skewness of 1.40
(Fig. 1a). The signal-to-noise ratio (SNR) is defined as
SNR=10logPsPn,
where Ps is the power received at each range gate in a profile and Pn
is the mean noise power that is estimated by averaging the return power in
the top 30 range gates, which are between 16.8 and 17.7 km a.g.l. Since this
layer is well above the tropopause, few atmospheric hydrometeors existing in
this layer can scatter enough power back to achieve the radar sensitivity.
Figure 1a shows that the SNRs for clear skies closely follow a Gaussian
distribution. Instead of using radar-received power, the SNR is used as the
input in our cloud mask algorithm including estimating the background noise
level. This is because in our method the chance of a central range gate being noise or a potential feature relies on the probability of a given
range of SNR values following the Gaussian distribution. Note that the mean
value of the SNR for the noise power is not zero, but a small negative value
of about -0.3. This is because the mean of the noise power is larger than
its the median due to its positive skewed distribution. It is further noted
that, for the noise, the distribution of SNR and its mean for the top 30 range
gates are the same as those from the lower atmosphere.
Schematic flow diagram for hydrometeor detection method. So
and Sn are the mean SNR for the original and reduced noise,
respectively.
The SNR value is treated as the brightness of a pixel in an image
f(x,y) in our hydrometeor detection method. In image
processing, the random noise can be smoothed out by using a low-pass filter,
which gives a new value for a pixel of an image by averaging with
neighboring pixels (Tomasi and Manduchi, 1998). The cloud signals are highly correlated
in both space and time and have more similar values in near pixels while the
random noise values are not correlated. Figure 2a shows a schematic
comparison of the original noise, reduced noise and hydrometeor signal
distributions: the low-pass filter could efficiently reduce the original
radar noise represented by the green line to a narrow bandwidth (blue line)
while keeping the signal preserved. By reducing the standard deviations of
noise, which shrinks the overlap region of signal and noise and enhances
their contrast, the weak signals (yellow area) that cannot be detected based
on original noise level may become distinguished.
Following this idea, we develop a non-iterative hydrometeor detection
algorithm by applying a noise reduction and a central-pixel weighting
schemes. Figure 3 shows the schematic flow diagram of our method. For given
mean SNR values (So) and 1 standard deviation (σo) of the
original background noise, the input SNR data set is first separated into
two groups. The group with values greater than So+3σo is
considered to be the cloud features that can be confidently identified. Another
group with values between So and So+3σo may potentially
contain moderate (So+σo<SNR≤So+3σo) to weak
(So<SNR≤So+σo) cloud signals, which will further go
through a noise reduction process. Here So and σo are
estimated from the top 30 range gates of each five successive profiles.
The noise reduction process is performed by convolving radar SNR time–height
data with a low-pass filter. The Gaussian filter, which outputs a “weighted
average” of each pixel and its neighborhood with the average weighted more
towards the value of the central pixel (v0), is one of the most common
functions of the noise reduction filter. A 2-D Gaussian distribution kernel,
shown in Fig. 2b1, can be expressed as
Gi,j=12πσ2exp-i2+j22σ2,
where i and j are the indexes in a filter window and are 0 for the central
pixel, and σ is the standard deviation of the Gaussian
distribution for the window size of the kernel. Equation (2) is used in our
study to filter the radar SNR image. Note that the convolution kernel is
truncated at about 3 standard deviations away from the mean in order to
accurately represent the Gaussian distribution. Figure 1b is the cumulative
distribution functions of clear-sky SNR by convolving the same data
in Fig. 1a with filters that have different kernel sizes (3 × 3, 5 × 5, 7 × 7 and 9 × 9
pixels), corresponding to the σ ranging from 0.5 to 2. The
original SNR values are distributed from about -5 to 5. After convolving the
image with the Gaussian filter, the SNR distribution can be constrained to a
much narrower range. It is clear that the filter with a larger kernel size
is more effective in suppressing the noise. Shown in Fig. 1c are results for
a cloudy case on 4 January 2014 by applying the filter to the range
gates inside the cloud but adjacent to the boundary. It is shown that a
larger kernel size shifts the SNR farther away from the noise region. It
therefore appears that increasing the standard deviation (i.e., the window
size) would reduce the noise and enhance the contrast between signal and
noise more effectively. At the same time, however, a larger kernel can also
attenuate or blur the high-frequency components of an image (e.g., the
boundary of clouds) more. As shown in Fig. 1d, when the window size is
increased from 3 × 3 (σ= 0.5) to 9 × 9 (σ= 2), the SNR distribution of the range gates that are
outside the cloud but adjacent to the boundary gradually move toward larger
values. This will consequently raise the risk of misidentifying cloud
boundaries. To solve this problem, a bilateral filtering idea proposed by Tomasi and Manduchi (1998) is adopted here.
Considering a sharp edge between cloudy and
clear region as shown in Fig. 2b2, we define a δi,j function that, when the central pixel is on the cloudy or clear
side, gives a weighting of 1 to the similar neighboring pixels (i.e., on the
same side) and 0 to the other side. After combining this δ
function to the Gaussian kernel in Fig. 2b1, we can get a new
nonlinear function called bilateral kernel as shown in Fig. 2b3. It
can be written as
Bi,j=12πσ2exp-i2+j22σ2⋅δi,j.
Thus the bilateral kernel will reduce averaging noises with signals, and
vice versa. The noise-reduced imagehx,y is produced by convolving the bilateral kernel with the original
input image fx,y as
hx,y=k-1x,y∑j=-wj=w∑i=-wi=wfx+i,y+j⋅Bi,j,
where ±w is the bounds of the finite filter window, and
k-1x,y is defined as
1/∑j=-wj=w∑i=-wi=wBi,j, which is used to normalize the weighting. Since the bilateral
kernel function only averages the central pixel with neighbors on the same
side (Fig. 2b), ideally it will preserve sharp edges of a target. We will
discuss how to construct the δ function in order to group
the central pixel with its neighbors later in this section. In the noise
reduction process, a 5 × 5 window size (i.e., 25 bins in
total) is specified for the low-pass filter, which is empirically determined
by visually comparing the cloud masks with original images. We should keep
in mind that a small window size is less effective in noise reduction but a
large window is not suitable for recognizing weak signals.
For performing the noise reduction with Eq. (4) in a 5 × 5 filter window, the
number of range bins (Ns) with signal greater than So+3σo are
first counted. These Ns range bins are then subtracted from the total
25 of the range bins in the filter window. Note that a noise reduction is
only applied when the central pixel is among the 25-Ns bins, and the δ function is set to be zero for the Ns range bins. If the remaining
25-Ns range bins are all noises, the range bin number (Nm) with SNR
greater than So+σo should be about equal to an integral number
(Nt) of 0.16 × (25-Ns) where 0.16 is the probability
for a remaining range bin to have a value greater than So+σo
for a Gaussian noise. Thus when Nm is equal to or smaller than Nt,
all the 25-Ns range bins could only contain pure noise and/or some weak
cloud signals. In this case, the δ function is set to 1 for
all the 25-Ns bins. When Nm is found to be larger than Nt,
the 25-Ns range bins might contain a combination of moderate signal,
noise and/or some weak clouds. In this case, So+σo is selected
as a threshold to determine whether the pixels are on the same side of the
central pixel. If the central pixel has a value greater than So+σo, the δ function is assigned to 1 for the 25-Ns
pixels with SNR≥So+σo, but 0 for the bins with
SNR<So+σo. If the central pixel is less than
So+σo, the δ function is assigned to 1 for
the pixels with SNR<So+σo, but 0 for the 25-Ns bins
with SNR≥So+σo.
After picking out the strong return signals and applying the noise reduction
scheme, the new background noise Sn and its standard deviation σn are estimated. While Sn is the same as So, the σn
is significantly reduced, which is a half of σo. This will make
it possible to identify more hydrometeors as exhibited in Fig. 2a. We assign
different confidence level values (which is called the mask value in this
study) to the following initial cloud mask according to the SNR; 40 is first
assigned to the mask of any range bins with SNR >So+3σo in the original input data. For the rest of the range bins, after
applying the noise reduction, if the SNR>Sn+3σn, the
mask is assigned a value of 30; if Sn+2σn<SNR≤Sn+3σn, the
mask is 20; if Sn+σn<SNR≤Sn+2σn, the mask is 10; and the remaining range bin mask is
assigned a value of 0. Thus, a mask value assigned to a pixel represents the
confident level for the pixel to be a feature.
Illustration of the steps of the detection method using the real
data from 8 January 2014.
To reduce both false positives (i.e., false detections) and false negatives
(i.e., failed detections), the next step is to estimate whether a range gate
contains significant hydrometeor. Following Clothiaux et al. (2000, 1995)
and Marchand et al. (2008), a 5 × 5 spatial filter is used
to calculate the probability of clouds and noise occurring in the 25 range
gates. The probability of central-pixel weighting scheme proposed by
Marchand et al. (2008) is adopted here, and the weighting for the
central pixel is assigned according to its initial mask value. The
probability is calculated by
p=G(L)0.16NT0.84N0,
where N0 is the number of masks with zero mask value, NT is the
number of masks with non-zero mask value and N0+NT=25; G(L) is the
weighting probability of the central pixel that could be a false detection
at a given significant level of L (i.e., mask value) in the initial cloud
mask. Here G(0)= 0.84, G(10)= 0.16, G(20)= 0.028 and G(≥30)= 0.002. If
p estimated from Eq. (5) is less than a given threshold (pthresh), then
the central pixel is likely to be a hydrometeor signal. The cloud mask value
will be set to the same value as in the initial mask if it is non-zero;
otherwise it will be set to 10. Likewise, if p>pthresh, then
the central pixel is likely to be noise and the mask value will be set to 0.
This process is iterated five times for each pixel to obtain the final cloud
mask.
Following Marchand et al. (2008), who explained the logic of
choosing a proper threshold, pthresh is calculated as
pthresh=0.16Nthresh0.8425-Nthresh.
Note that a smaller pthresh will keep the false positives lower but
increase the false negative. Herein we adopt the pthresh of
5.0 × 10-12 used in Clothiaux et al. (2000),
which is approximately equivalent to Nthresh= 13.
Panels (a1), (a2) and (a3) are three “square clouds”
that have strong, moderate and weak SNR values with random Gaussian noise
used to test the detection method. Panels (b1), (b2) and (b3) are
SNR distributions after convolving the data with a bilateral kernel. Panels
(c1), (c2) and (c3) are the final cloud mask filtered by the
spatial filter.
Figure 4 illustrates the main steps of our detection method by using the data
from 8 January 2014. Figure 4a is the original SNR input. Figure 4b
shows the SNR distribution after the noise reduction process. One can see
that the SNR, after being compressed to a narrow range, becomes much smoother
than original input. This step significantly increases the contrast between
signal and noise. Figure 4c indicates the range gates that potentially
contain hydrometeors in the initial cloud mask. Figure 4d is the final
result after applying the spatial filter.
ResultsDetection test
To test the performance of our hydrometeor detection method, we create seven
squares of SNR with sides of 100, 50, 25, 15, 10 and 5 and three bins to mimic the
radar “time–height” observations as shown in Fig. 5. The background noise
is randomly given by a Gaussian distribution with a mean S0 and a
standard deviation σ0. The targets in panels a1,
a2 and a3 are set with different SNR values to represent
situations in which clouds have strong, moderate and weak signals,
respectively. In panel a1 the target signals are set to be
S0+10σ0. In panel a2, the target signals distribute from
S0+σ0 to S0+3σ0 with a mean value of
S0+2σ0. In panel a3, the target SNRs range from S0
to S0+σ0 with a mean value of S0+0.5σ0.
Summary of false positives and failed negatives for hypothetical
strong, moderate and weak cloud cases in Fig. 5a1, a2 and a3, respectively.
The three middle panels in Fig. 5 show the results after applying the noise
reduction. Again, comparing with the input signals, we can see that the
background noise is well compressed and becomes smoother. The shapes of the
square targets are all well maintained with sharp boundaries for strong and
moderate signals (see Fig. 5b1 and b2). In Fig. 5b3 for weak signals, the
three-bin square target is not obvious while the other six squares are still
distinguishable. To separate the compressed background noise from
hydrometeor signals, the 5 × 5 spatial filter is further
applied to the noise-reduced data. The three right panels in Fig. 5 show the
final mask results. Generally, the hydrometeor detection method can identify
those targets well. Six of the seven square targets can be identified for
clouds with strong and moderate SNR. The 3 × 3 square is
missed because the small targets cannot be resolved by the 5 × 5 spatial filter. Since the temporal resolution of KAZR is about 4 s, we expect that a
cloud only having three bins in horizontal would be
rare. For the targets with weak SNR values, the 3 × 3 and
5 × 5 square targets are missed, but the rest five square
targets are successfully distinguished and their boundaries are well
maintained as shown in Fig. 5c3.
Cloud mask without applying noise reduction and central-pixel
weighting. (a), (b) and (c) are for the targets with strong, moderate
and weak SNR, respectively, from Fig. 4a1, a2 and a3.
To further demonstrate the performance of our method for detecting the
hypothetical clouds in Fig. 5a1, a2 and a3, the false and failed detection
rates are listed in the Table 1. For strong signals, no background noise
pixel is misidentified as one containing hydrometeors at level 40. Although
at levels less than 40, some noise pixels around the edges of targets are
identified as signals, the false detection is within 0.05 %. The failed
detection rate is about 0.24 %. For moderate signals, the failed detection
rate is still as small as 0.23 %, while the false detection increases a
little to 0.10 % at the confidence levels below 30. The failed detection
can reach up to 9.77 % for weak signal at level 10 but more than 90 %
weak signals can be captured in our method. Note that the false positive is
less than 0.01 %; in other words, any range gate that is detected likely
as a signal bin will have extremely high likelihood to contain hydrometeors
although its backscattered signal is weak.
One-day example of radar- and lidar-observed cirrus clouds at the
SACOL on 8 January 2014. (a) KAZR reflectivity; (b) MPL normalized
backscatter intensity; (c) MPL depolarization ratio; (d) radar cloud mask
derived by the ARM MMCR algorithm; (e) radar cloud mask derived by our new
method; (f) MPL feature mask. Three windows in (d), (e) and (f) show the
zoom-in views of cirrus masks.
The upper panel shows the number of occurrences of the detected
hydrometeor range bins from the two methods. The solid line is the number of
range gates derived from our method. The dotted line from the ARM MMCR
algorithm. The lower two panels demonstrate the increased percentage of
hydrometeor bins from our method comparing to the ARM MMCR algorithm. The
solid line is calculated by applying both noise reduction and central-pixel
weighting schemes, while the dashed line is calculated by only applying the
central-pixel weighting scheme in our detection method.
The simple square clouds are also tested by using the ARM hydrometeor
detection algorithm developed for the MMCRs (Clothiaux et al., 2000, 1995), which does not
include the noise reduction and weighting schemes. As can be seen in Fig. 6,
this algorithm can only find five of the seven square targets with strong
and moderate SNR. Meanwhile, without central-pixel weighting, the corners of
the targets become rounded and more than 2.23 % of hydrometeors are missed
for strong and moderate cloud cases. More importantly, none of the weak
cloud signals can be detected. Comparing Figs. 5 and 6, it is obvious that
our hydrometeor detection method can maintain the cloud boundary well, keep
both false and failed detection rate as low as a few percent for strong and
moderate cloud cases and has a remarkable advantage in recognizing weak
signals.
It is noted that the ARM program has recently developed a new operational
cloud mask algorithm for the KAZRs by applying the Hildebrand and Sekhon (1974) technique to determine the SNR values along with the
spatial filter (Johnson, K., personal communication, 2017). It is our
future research task to compare our algorithm with the ARM's new operational
algorithm.
Application to the SACOL KAZR observations
Our hydrometeor detection method was then applied to the winter and summer
(December 2013 and January, February, June, July and August 2014) KAZR data at
the SACOL. A micropulse lidar (MPL) transmitted at 527 nm is operated near
the KAZR. Lidar is more sensitive to thin cirrus clouds and thus used to
assess the performance of our algorithm. Figure 7a, b and c show a
1-day example of radar reflectivity, normalized backscatter and
depolarization ratio of lidar, respectively. The cloud masks from our
detection method and the ARM MMCR method are shown in Fig. 7d and e. The MPL
feature mask is derived by modifying the method developed in Thorsen et al. (2015) and Thorsen and Fu (2015; see Fig. 7f). The vertical and
horizontal resolutions of the radar and lidar are different, and we map the
observed data and derived feature mask on the same height and time
coordinates for the purpose of a comparison. A distinct thin feature layer
appears at about 8 km from 15:00 to 18:30 UTC during the lidar
observation,
which is clearly identified as a cirrus cloud using the depolarization
ratio. The contrast between the cirrus layer and background from the KAZR
observation (Fig. 7a) is very weak, and only a few range gates are
identified as the hydrometeors using the method without the noise reduction
and weighting (Fig. 7d). However, our cloud mask method can find more range
gates (about 2.8 times of ARM's result). All these increased range bins from
our method are also detected as thin cirrus by the MPL (Fig. 7f). Another
apparent discrepancy exists in the low atmosphere layer. A non-negligible
number of range gates at about 2 km are recognized as hydrometeor echoes by
our method but mostly missed by former technique. This feature layer is also
apparent in lidar observations with both relative large backscatter
intensities and depolarization ratios (Fig. 7b and c). MPL recognizes this
feature as an aerosol layer. From our KAZR observations, we did find some
dust events that were detected by this millimeter-wavelength radar (see the
auxiliary Fig. 1). Those feature echoes detected by our method might
be partly caused by large dust particles. Although the dust is not desired for
cloud mask, the appearance of those particles does prove the ability of our
method to recognize weak signals.
Mean values of four quantities for increased KAZR feature and noise
pixels.
(a) A comparison of the increased detections with the MPL
observations. (b) The percentage of the cloud pixels identified by MPL but
not by KAZR in the total MPL detected cloud pixels. The solid line in Fig. 9a
is the percentage of increased detections seen by both KAZR with our method
and MPL as compared with the total increased detections. The dash line in
Fig. 9a is the number of increased detections. The solid lines in Fig. 9b
represents for the algorithm with noise reduction step. The dash line in
Fig. 9b is for the method without noise reduction scheme.
The upper two panels in Fig. 8 compare the number of occurrences of the
detected hydrometeor range bins from our methods with that from the ARM MMCR
algorithm for the 6 months of data. Generally, one can see that the
variations of the identified hydrometeor numbers with height from the two
techniques are in a good agreement. The distinct discrepancies appear at
about 2 km in winter and above 13 km in summer, when our method apparently
identifies more hydrometeors. To quantitatively evaluate the two schemes used
in our algorithm and illustrate the improvements of our method, we plot the percent
change of the increased hydrometeors from our method by comparing it to the
ARM MMCR method in the lower two panels in Fig. 8. As
expected from the results in the test square clouds, our method can identify
more signals. The remarkable feature is that the increased percentage is
over 20 % at high altitude, indicating that our method can recognize more
cirrus clouds. The increased percentage of hydrometeor derived only with the
weighting scheme (dashed line) and with both the noise reduction and
weighting schemes (solid line) varies differently with height to demonstrate the individual
contribution of the scheme to the improvement of our method. In winter, the
number of the detected hydrometeors with only the weighting scheme is almost
the same as that from the ARM method at layer from 3.5 to 9 km a.g.l., while
this number will increase by about 5 % if the noise reduction scheme is
involved, indicating that some hydrometeors with weak SNR values may exit in
this layer. Above and below this atmospheric layer, the increased percentage
is largely determined by the weighting scheme. In summer, the two lines
almost overlap each other between 3.5 and 9.5 km with values below 5 %,
revealing that the bins found by our method in the mid-atmospheric layer
are mainly around the boundaries of clouds. We may infer that in summer
season, clouds in the middle level are usually composed of large droplets with
strong SNR values. The two lines are gradually moving apart with height. This is
because hydrometeors in the upper troposphere usually have smaller size that
cause weak SNR values, which will be effectively detected by the noise
reduction scheme.
PDF of (a) MPL backscatter, (b) MPL depolarization ratio,
(c) KAZR SNR and (d) KAZR LDR for the increased KAZR detections (solid line) and
KAZR noise (dashed line) pixels.
We also analyzed the data when both KAZR and MPL observations are available
and compared our KAZR cloud mask with MPL feature detection. Figure 9a shows
the percentage of the increased detections identified by both KAZR with our
method and MPL observations as normalized to the KAZR total increased
detections. Here we should point out that MPL has difficulty
distinguishing dust from clouds (especially cirrus clouds). Unfortunately,
there exists a large amount of dust aerosols over the SACOL region. We visually
examined several cases and found that many MPL signals, which should be clouds,
are misidentified as aerosols. For this reason, we compare the increased KAZR detections with the features (i.e., cloud and aerosol) detected by
MPL above 3 km. It is obvious that more than 90 % of increased
detections are also detected as features by MPL. Below 3 km, we calculated
the percentage by comparing the KAZR detections only with the cloud pixels
detected by MPL since aerosol is always present in the lowest several
kilometers. To test whether those increased detections that are not
identified as cloud by MPL under 3 km are signal or noise, we examined the
probability distribution functions (PDFs) of MPL normalized aerosol backscatter and depolarization corresponding
to the increased KAZR feature and KAZR noise regions in Fig. 10a and b.
The PDFs of MPL backscatter for the KAZR feature and noise regions are quite
different (Fig. 10a), with mean backscatter of 0.15 for feature and 0.10
(photoelectrons km-2)/(µs µJ-1) for noise. The
mean of the MPL depolarization ratio is 0.16 for feature and 0.12 for noise
although the PDFs are similar (Fig. 10b), because dust is the main aerosol
type over this region. We also plot the PDFs of KAZR SNR and LDR for the
increased feature and noise pixels (Fig. 10c and d). The PDFs of SNR and
LDR are Gaussian-like for noise pixels and are quite different from those
for the increased detections. Table 2 shows the mean values of the four
quantities shown in Fig. 10. All the differences of these mean values between
KAZR noise and increased feature regions pass the significant test at 95 %
confidence level except for the MPL depolarization ratio. These increased
features from our feature mask could thus be dust (and/or some plankton) but
cannot be the false positive. Figure 9b shows the profile of the false negative
(i.e., the percentage of the cloud pixels identified by MPL but not by KAZR
in the total MPL-detected cloud pixels). We can see that our method with the
noise reduction has relative smaller false negatives especially in the
layers under 3 km and between 7 and 10 km. Table 3 is the confusion matrix
of the KAZR feature mask results from both our and the ARM MMCR methods
estimated by MPL cloud feature. Overall, 70.7 % of the cloud mask identified by
MPL was also recognized by the new method, while this percent is 68.9 % for
the algorithm without noise reduction. The difference of false positive
between the two methods is only 0.1 % as shown in Table 3. These numbers
show an improvement of our method of recognizing weak signals by
comparing with the results from the ARM MMCR method; however, they cannot
be used to assess the accuracy of our method due to the issue of MPL feature
detection.
Summary and discussion
Based on image noise reduction technique, we propose a modified method to
detect hydrometeors from cloud radar return signals. The basic idea is to
treat the SNR value of each range gate as a pixel brightness and suppress
the SNR distributions of noise to a narrow range by convolving with a 2-D
bilateral kernel which can effectively avoid blurring the high-frequency
components (i.e., boundaries of a target). After the noise smoothing process,
a special filter with a central-pixel weighting scheme is used to obtain the
final cloud mask. The detection of the test square clouds shows that there
are two remarkable advantages of our method. First, the noise reduction
scheme of our algorithm can enhance the contrast between signal and noise,
while keeping the cloud boundaries preserved and detecting more hydrometeors
with weak SNR values. Second, both false positive and failed negative rates
for strong and moderate clouds can be reduced to acceptably small values. A
comparison of radar and lidar observations further highlight the advantage
of our method for recognizing weak cloud signal in application.
KAZR reflectivity on 29 January 2014 at the
SACOL, indicating a dust event. The morphology and power level of the return
signal are not apparent for a cloud from the surface to the height of 5 km
between 08:00 and 16:00 UTC.
The datasets used in this paper are available by contacting the author at gejm@lzu.edu.cn.
The authors declare that they have no conflict of interest.
Acknowledgements
This work was supported by the National Science Foundation of China
(41430425, 41575016, 41521004, 41505011), China 111 project (no. B13045)
and the Fundamental Research Funds for the Central University
(lzujbky-2016-k01).
Edited by: Xiaohong Liu
Reviewed by: three anonymous referees
References
Ackerman, T. P. and Stokes, G. M.: The Atmospheric Radiation
Measurement program, Phys. Today, 56, 14–14, 2003.Barker, H. W.: Indirect aerosol forcing by homogeneous and
inhomogeneous clouds, J. Climate, 13, 4042–4049,
10.1175/1520-0442(2000)013<4042:iafbha>2.0.co;2, 2010.Barker, H. W. and Fu, Q.: Assessment and optimization of the
gamma-weighted two-stream approximation, J. Atmos. Sci., 57, 1181–1188,
10.1175/1520-0469(2000)057<1181:aaootg>2.0.co;2, 2000.Bony, S., Stevens, B., Frierson, D. M. W., Jakob, C., Kageyama, M., Pincus, R., Shepherd, T. G., Sherwood, S. C.,
Siebesma, A. P., Sobel, A. H., Watanabe M., and Webb, M. J.: Clouds, circulation and climate sensitivity,
Nat. Geosci., 8, 261–268, 10.1038/ngeo2398, 2015.
Canny, J.: A computational approach to edge-detection, IEEE T. Pattern Anal., 8,
679–698, 1986.Clothiaux, E. E., Miller, M. A., Albrecht, B. A., Ackerman, T. P., Verlinde,
J.,
Babb, D. M., Peters, R. M., and Syrett, W. J.: An evaluation of a 94-ghz
radar for remote-sensing of cloud properties, J. Atmos. Ocean. Tech., 12, 201–229,
10.1175/1520-0426(1995)012<0201:aeoagr>2.0.co;2, 1995.Clothiaux, E. E., Moran, K. P., Martner, B. E., Ackerman, T. P., Mace, G. G.,
Uttal, T., Mather, J. H., Widener, K. B., Miller, M. A., and Rodriguez, D. J.:
The atmospheric radiation measurement program cloud radars:
Operational modes, J. Atmos. Ocean. Tech., 16, 819–827, 10.1175/1520-0426(1999)016<0819:tarmpc>2.0.co;2, 1999.Clothiaux, E. E., Ackerman, T. P., Mace, G. G., Moran, K. P., Marchand, R.
T.,
Miller, M. A., and Martner, B. E.: Objective determination of cloud
heights and radar reflectivities using a combination of active remote
sensors at the ARM CART sites, J. Appl. Meteorol., 39, 645–665,
10.1175/1520-0450(2000)039<0645:odocha>2.0.co;2, 2000.Fu, Q., Carlin, B., and Mace, G.: Cirrus horizontal inhomogeneity and
OLR bias, Geophys. Res. Lett., 27, 3341–3344, 10.1029/2000gl011944, 2000a.
Fu, Q., Cribb, M. C., and Barker, H. W.: Cloud geometry effects on
atmospheric solar absorption, J. Atmos. Sci., 57, 1156–1168, 2000b.Fu, Q., Baker, M., and Hartmann, D. L.: Tropical cirrus and water vapor: an effective Earth infrared iris feedback?,
Atmos. Chem. Phys., 2, 31–37, 10.5194/acp-2-31-2002, 2002.
Fukao, S. and Hamazu, K.: Radar for Meteorological and Atmospheric
Observations, Toyko, Springer, 2014.He, K., Sun, J., and Tang, X.: Guided Image Filtering, IEEE T. Pattern Anal., 35,
1397–1409, 10.1109/tpami.2012.213, 2013.Hildebrand, P. H. and Sekhon. R. S.: Objective Determination of the
Noise Level in Doppler Spectra J. Appl. Meteorol., 13, 808–811,
10.1175/1520-0450(1974)013<0808:ODOTNL>2.0.CO;2, 1974.Huang, J. P., Minnis, P., Lin, B., Yi, Y. H., Khaiyer, M. M., Arduini, R. F.,
Fan, A., and Mace, G. G.: Advanced retrievals of multilayered cloud
properties using multispectral measurements, J. Geophys. Res.-Atmos., 110,
D15S18, 10.1029/2004jd005101, 2005.Huang, J. P., Minnis, P., Lin, B., Yi, Y. H., Fan, T. F., Sun-Mack, S., and
Ayers, J. K.: Determination of ice water path in ice-over-water cloud
systems using combined MODIS and AMSR-E measurements, Geophys. Res. Lett., 33,
L21801, 10.1029/2006gl027038, 2006a.
Huang, J. P., Wang, Y. J., Wang, T. H., and Yi, Y. H.: Dusty cloud
radiative forcing derived from satellite data for middle latitude regions of
East Asia, Prog. Nat. Sci., 16, 1084–1089, 2006b.Huang, J. P., Ge, J., and Weng, F.: Detection of Asia dust storms using
multisensor satellite measurements, Remote Sens. Environ., 110, 186–191,
10.1016/j.rse.2007.02.022, 2007.Huang, J., Zhang, W., Zuo, J., Bi, J., Shi, J., Wang, X., Chang, Z., Huang, Z., Yang, S., Zhang, B., Wang, G., Feng, G., Yuan, J.,
Zhang, L., Zuo, H., Wang, S., Fu, C., and Chou, J.: An Overview of the Semi-arid Climate and
Environment Research Observatory over the Loess Plateau, Adv. Atmos. Sci., 25, 906–921,
10.1007/s00376-008-0906-7, 2008.Illingworth, A. J., Hogan, R. J., O'Connor, E. J., Bouniol, D., Brooks, M. E., Delanoë, J., Donovan, D. P.,
Eastment, J. D., Gaussiat, N., Goddard, J. W. F., Haeffelin, M., Klein Baltink, H., Krasnov, O. A.,
Pelon, J., Piriou, J.-M., Protat, A., Russchenberg, H. W. J., Seifert, A.,
Tompkins, A. M., Van Zadelhoff, G.-J., Vinit, F., Willén, U., Wilson, D. R., and Wrench, C. L.: Cloudnet – Continuous evaluation of cloud
profiles in seven operational models using ground-based observations,
B. Am. Meteorol. Soc., 88, 883-898, 10.1175/bams-88-6-883, 2007.Jing Su, Jianping Huang, Qiang Fu, Minnis, P., Jinming Ge, and Jianrong Bi: Estimation of Asian dust aerosol effect on cloud
radiation forcing using Fu-Liou radiative model and CERES measurements, Atmos. Chem. Phys., 8, 2763–2771, 10.5194/acp-8-2763-2008, 2008.King, M. D., Platnick, S., Menzel, W. P., Ackerman, S. A., and Hubanks, P. A.:
Spatial and Temporal Distribution of Clouds Observed by MODIS
Onboard the Terra and Aqua Satellites, IEEE T. Geosci. Remote, 51, 3826–3852,
10.1109/tgrs.2012.2227333, 2013.Kollias, E., Clothiaux, E., Miller, M. A., Albrecht, B. A., Stephens, G. L., and
Ackerman, T. P.: Millimeter-wavelength radars – New frontier in
atmospheric cloud and precipitation research, B. Am. Meteorol. Soc., 88, 1608–1624,
10.1175/bams-88-10-1608, 2007a.Kollias, E., Clothiaux, E., Miller, M. A., Luke, E. P., Johnson, K. L.,
Moran, K. P., Widener, K. B., and Albrecht, B. A.: The Atmospheric Radiation
Measurement Program cloud profiling radars: Second-generation sampling
strategies, processing, and cloud data products, J. Atmos. Ocean. Tech., 24, 1199–1214,
10.1175/jtech2033.1, 2007b.Li, J., Huang, J., Stamnes, K., Wang, T., Lv, Q., and Jin, H.: A global survey of cloud overlap based on CALIPSO and CloudSat
measurements, Atmos. Chem. Phys., 15, 519–536, 10.5194/acp-15-519-2015, 2015.Mace, G. G., Ackerman, T. P., Minnis, P., and Young, D. F.: Cirrus layer
microphysical properties derived from surface-based millimeter radar and
infrared interferometer data, J. Geophys. Res.-Atmos., 103, 23207–23216, 10.1029/98jd02117, 1998.Mace, G. G., Clothiaux, E. E., and Ackerman, T. P.: The composite
characteristics of cirrus clouds: Bulk properties revealed by one year of
continuous cloud radar data, J. Climate, 14, 2185–2203,
10.1175/1520-0442(2001)014<2185:tccocc>2.0.co;2, 2001.Marchand, R., Mace, G. G., Ackerman, T., and Stephens, G.: Hydrometeor
detection using Cloudsat – An earth-orbiting 94-GHz cloud radar, J. Atmos. Ocean. Tech., 25,
519–533, 10.1175/2007jtecha1006.1, 2008.Marr, D. and Hildreth, E.: Theory of edge-detection, P. R. Soc. B, 207,
187–217, 10.1098/rspb.1980.0020, 1980.Perona, P. and Malik, J.: Scale-space and edge-detection using
anisotropic diffusion, IEEE T. Pattern Anal., 12, 629–639, 10.1109/34.56205, 1990.Protat, A., Bouniol, D., Delanoe, J., May, P. T., Plana-Fattori, A., Hasson,
A.,
O'Connor, E., Goersdorf, U., and Heymsfield, A. J.: Assessment of
Cloudsat Reflectivity Measurements and Ice Cloud Properties Using
Ground-Based and Airborne Cloud Radar Observations, J. Atmos. Ocean. Tech., 26, 1717–1741,
10.1175/2009jtecha1246.1, 2009.Protat, A., Delanoë, J., May, P. T., Haynes, J., Jakob, C., O'Connor, E., Pope, M., and
Wheeler, M. C.: The variability of tropical ice cloud properties as a function of the large-scale
context from ground-based radar-lidar observations over Darwin, Australia, Atmos. Chem. Phys., 11, 8363–8384, 10.5194/acp-11-8363-2011, 2011.Ramanathan, V., Cess, R. D., Harrison, E. F., Minnis, P., Barkstrom, B. R.,
Ahmad, E., and Hartmann, D.: Cloud-radiative forcing and climate – Results from the earth radiation budget experiment, Science, 243, 57–63,
10.1126/science.243.4887.57, 1989.
Randall, D. A., Wood, R. A., Bony, S., Colman, R., Fichefet, T., Fyfe, J.,
Kattsov, V., Pitman, A., Shukla, J., Srinivasan, J., Stouffer, R. J., Sumi, A., and
Taylor, K. E.: Cilmate Models and Their Evaluation, in: Climate Change
2007: The Physical Science Basis, Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change,
edited by: Solomon, S., Qin, D., Manning, M., Chen, Z., Marquis, M., Averyt, K. B., Tignor, M., and Miller, H. L., Cambridge University Press, Cambridge,
United Kingdom and New York, NY, USA, 2007.Sassen, K. and Benson, S.: A midlatitude cirrus cloud climatology
from the facility for atmospheric remote sensing. Part II: Microphysical
properties derived from lidar depolarization, J. Atmos. Sci., 58, 2103–2112,
10.1175/1520-0469(2001)058<2103:amcccf>2.0.co;2, 2001.Stephens, G. L.: Cloud feedbacks in the climate system: A critical
review, J. Climate, 18, 237–273, 10.1175/jcli-3243.1, 2005.Stephens, G. L., Vane, D. G., Boain, R. J., Mace, G. G., Sassen, K., Wang, Z.,
Illingworth, A. J., O'Connor, E. J., Rossow, W. B., Durden, S. L., Miller, S. D., Austin, R. T., Benedetti, A., Mitrescu, C., and
The CloudSat Science Team: The cloudsat mission and the a-train – A new
dimension of space-based observations of clouds and precipitation, B. Am. Meteorol. Soc.,
83, 1771–1790, 10.1175/bams-83-12-1771, 2002.Thorsen, T. J. and Fu, Q.: Automated Retrieval of Cloud and Aerosol
Properties from the ARM Raman Lidar. Part II: Extinction, J. Atmos. Ocean. Tech., 32, 1999–2023,
10.1175/jtech-d-14-00178.1, 2015.Thorsen, T. J., Fu, Q., and Comstock, J.: Comparison of the CALIPSO satellite
and ground-based observations of cirrus clouds at the ARM TWP sites,
J. Geophys. Res.-Atmos., 116, D21203, 10.1029/2011jd015970, 2011.Thorsen, T. J., Fu, Q. N., Rob, K., Turner, D. D., Comstock, J.
M.:
Automated Retrieval of Cloud and Aerosol Properties from the ARM Raman
Lidar. Part I: Feature Detection, J. Atmos. Ocean. Tech., 32, 1977–1998,
10.1175/JTECH-D-14-00150.1, 2015.Tomasi, C. and Manduchi, R.: Bilateral Filtering for Gray and Color
Images, IEEE International Conference on Computer Vision, Bombay, India, 10.1109/ICCV.1998.710815, 1998.Wang, Z. and Sassen, K.: Cloud type and macrophysical property
retrieval using multiple remote sensors, J. Appl. Meteorol., 40, 1665–1682,
10.1175/1520-0450(2001)040<1665:ctampr>2.0.co;2 2001.Williams, K. D. and Webb, M. J.: A quantitative performance
assessment of cloud regimes in climate models, Clim. Dynam., 33, 141–157,
10.1007/s00382-008-0443-1, 2009.Yan, H. R., Huang, J. P., Minnis, P., Yi, Y. H., Sun-Mack, S., Wang, T. H., and
Nakajima, T. Y.: Comparison of CERES-MODIS cloud microphysical
properties with surface observations over Loess Plateau, J. Quant. Spectrosc. Ra., 153, 65–76,
10.1016/j.jqsrt.2014.09.009, 2015.Yuan, J., Fu, Q., and McFarlane, N.: Tests and improvements of GCM
cloud parameterizations using the CCCMA SCM with the SHEBA data set,
Atmos. Res., 82, 222–238, 10.1016/j.atmosres.2005.10.009, 2006.