VORTEX-Southeast 2016: UMass S-Band FMCW Profiler Data, version 2.0 January 2023 Susan Belak, Robin Tanamachi Department of Earth, Atmospheric, and Planetary Sciences (EAPS) Purdue University 550 Stadium Mall Drive West Lafayette, IN 47907 (765) 496-2866 Stephen Frasier University of Massachusetts Microwave Remote Sensing Laboratory 151 Holdsworth Way Amherst, MA 01003 (413) 545-0779 Corresponding author: Robin Tanamachi, rtanamachi@purdue.edu ### Data set overview ### These data have been reprocessed from the original version (1.0). The following changes and additions were made: 1. Raw Doppler spectra (variable ("sf") were reprocessed using an in-painting method[1] to remove power spurs / spikes. (See Remark 2 in the v1.0 documentation, appended below.) These in-painted spectra are encoded as variable "sfp." Additionally, the power spur flag (flg2) from v1.0 is now integrated into each hourly file, instead of being distributed as a separate file. 2. The in-painted spectra were converted to moments data using coherent signal processing[2]. These reprocessed moments are denoted by variable names ending in the letter "c." Corresponding variables are: Variable name: Original: Coherently processed: Reflectivity Factor Ze Zec Doppler (radial) velocity vel vec Signal-to-noise ratio snr snc The variable "ncp" (normalized coherent power), a product of the coherent signal processing, was also added. Changes 1 and 2 constitute the M.S. thesis work[5] of Mrs. Susan Belak (former last name: Beveridge). 3. Automated boundary layer height detection and tracking using an Extended Kalman Filter-based method[3, 4] was applied sequentially to the Zec field. At times when the filter failed or rain was likely present (mean Zec in the BL > 5 dBZ), this variable is set to NaN. The resulting ABLH time series is encoded as variable "ablh." This addition is described in a forthcoming publication[6]. References: [1] Chan, S. H., X. Wang, and O. A. Elgendy, 2017: Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications. IEEE Transactions on Computational Imaging, 3, 84–98, https://doi.org/10.1109/TCI.2016.2629286. [2] Pazmany, A. L., and S. J. Haimov, 2017: Coherent Power Measurements with a Compact Airborne Ka-Band Precipitation Radar. Journal of Atmospheric and Oceanic Technology, 35, 3–20, https://doi.org/10.1175/JTECH-D-17-0058.1. [3] Lange, D., F. Rocadenbosch, J. Tiana-Alsina, and S. J. Frasier, 2015: Atmospheric boundary layer height estimation using a Kalman filter and a frequency-modulated continuous-wave radar. IEEE Transactions on Geoscience and Remote Sensing, 53, 3338–3349, https://doi.org/10.1109/TGRS.2014.2374233. [4] Tanamachi, R. L., S. J. Frasier, J. Waldinger, A. T. LaFleur, D. D. Turner, and F. Rocadenbosch, 2019: Progress toward characterization of the atmospheric boundary layer over northern Alabama using observations by a vertically pointing, S-band profiling radar during VORTEX-Southeast. Journal of Atmospheric and Oceanic Technology, 36, 2221–2246, https://doi.org/10.1175/JTECH-D-18-0224.1. [5] Beveridge, S. L., 2021: Quality control and verification of Doppler spectra collected from a vertically pointing FMCW radar deployed during VORTEX-Southeast. Purdue University, 87 pp. https://doi.org/10.25394/PGS.14916606.v1 [6] Belak, S. L., R. L. Tanamachi, M. Asel, G. Dennany, A. Gnanasambandam, S. H. Chan, and S. J. Frasier, 2023: Quality control of Doppler spectra from a vertically pointing, S-band profiling radar. Submitted to Journal of Atmospheric and Oceanic Technology. With the addition of several new variables, each hourly file is now approximately twice the size of the original (~500 MB). Users are advised to ensure they have adequate disk space before downloading the entire data set. We elected to retain the v1.0 variables in v2.0 for ease of comparison between old and new data. ### Data format ### netcdf { dimensions: time = 205 ; vels = 256 ; gate = 1024 ; variables: float64 time(time) ; time:units = seconds since 01-01-1970 00:00:00 ; float32 vels(vels) ; vels:long_name = Spectral Velocity Bins ; vels:units = m/s ; float32 gate(gate) ; gate:long_name = Height ; gate:units = m AGL ; int16 Zef(time, gate, vels) ; Zef:long_name = Spectral Reflectivity Factor ; Zef:units = dBZ/bin x10 ; int16 sf(time, gate, vels) ; sf:long_name = Raw Spectral Power ; sf:units = dB(uncalibrated) x10 ; int16 sn(time, gate) ; sn:long_name = Noise Power ; sn:units = dB(uncalibrated) x10 ; int16 sff(time, gate, vels) ; sff:long_name = Filtered Spectral Power ; sff:units = dB(uncalibrated) x10 ; int16 snr(time, gate) ; snr:long_name = Signal to Noise Ratio ; snr:units = dB x10 ; int16 Ze(time, gate) ; Ze:long_name = Reflectivity Factor ; Ze:units = dBZ x10 ; float32 vel(time, gate) ; vel:long_name = Mean Radial Velocity ; vel:units = m/s ; float32 wid(time, gate) ; wid:long_name = Spectrum Width ; wid:units = m/s ; int8 flg2(gate) ; flg2:long_name = Power spur contamination flag from v1.0 ; flg2:units = 0 = ok, 1 = suspect ; int16 sfp(time, gate, vels) ; sfp:long_name = In-painted spectral power ; sfp:units = dB(uncalibrated) x10 ; int16 Zec(time, gate) ; Zec:long_name = Coherent Reflectvity Factor ; Zec:units = dBZ x10 ; float32 vec(time, gate) ; vec:long_name = Coherent Mean Radial Velocity ; vec:units = m/s ; int16 snc(time, gate) ; snc:long_name = Coherent Signal to Noise Ratio ; snc:units = dB x10 ; float32 ncp(time, gate) ; ncp:long_name = Normalized Coherent Power ; ncp:units = [0-1] ; float64 ablh(time) ; ablh:units = m AGL ; ablh:long_name = Atmospheric Boundary Layer Height ; // global attributes: :NetCDFRevision = UMass MIRSL and Purdue FMCW v2.0 ; :RadarName = UMa-FMCW ; :PRF = 190.73500061035156 ; :Frequency = 2920000000.0 ; :Latitude = 34.69060134887695 ; :Longitude = -86.88150024414062 ; } ----- Original (v1.0) documentation ----- Vortex-SE 2016: UMass S-band FMCW Profiler Data Stephen Frasier, Joseph Waldinger, William Heberling University of Massachusetts Microwave Remote Sensing Laboratory 151 Holdsworth Way Amherst, MA 01003 (413) 545-0779 and Robin Tanamachi, Dan Dawson Department of Earth, Atmospheric, and Planetary Sciences (EAPS) Purdue University 550 Stadium Mall Drive West Lafayette, IN 47907 (765) 496-2866 emails: frasier@umass.edu, jwaldinger@umass.edu, rtanamachi@purdue.edu, dandawson@purdue.edu Data Contact: frasier@umass.edu 1.0 Data Set Overview Vertical profiles of radar moments and Doppler spectra over the period 3/7/2016--4/30/2016 collected by the UMass S-band FMCW profiling radar. The radar was located at the "Belle Mina site" at 34.6906N/86.8815W. 2.0 Instrument Description The S-band FMCW Radar is principally described in Ince et al.(2003). For Vortex-SE it operated with a substantially higher sampling rate to enable a wider range of vertical velocities, with some sacrifice of sensitivity to do so. Basic instrument parameters are as follows Center Frequency 2.92 GHz Bandwidth 30 MHz Sweep Interval 5.24 ms Sweep Rate (PRF) 190.735 Hz Transmit Power 250 W (continuous) Antenna Gain 34 dB Compression Gain 50.2 dB Noise level -97.4 dBm Beam width 3 deg Range bins 1024 Range resolution 5 m Max/min velocity +/-4.9 m/s 3.0 Data Collection & Processing FM sweeps are collected in blocks of 256 sweeps to produce a Doppler spectrum over a 1.34 s interval. Twelve such spectra are then accumulated yielding averaged spectra every 16.1 s. Moments are calculated from the spectra. Reflectivity is estimated from system SNR using the radar equation and nominal system parameters. 4.0 Data Format Data are stored in hourly files in NetCDF format. Below is a commented CDL description of a file. netcdf S20160307T020003 { dimensions: vels = 256 ; // number of velocity bins in Doppler spectra gate = 1024 ; // number of 5-m range bins time = UNLIMITED ; // (224 currently) variables: float vbins(vels) ; vbins:Name = "Spectral Velocity Bins" ; vbins:Units = "m/s" ; float height(gate) ; height:Name = "Height" ; height:Units = "m AGL" ; int secs(time) ; secs:Name = "Secs" ; secs:Units = "s since 1/1/1970" ; short Zef(time, gate, vels) ; Zef:Name = "Spectral Reflectivity Factor" ; Zef:Units = "dBZ/bin x10" ; short sf(time, gate, vels) ; sf:Name = "Raw Spectral Power" ; sf:Units = "dB(uncalibrated) x10" ; short sn(time, gate) ; sn:Name = "Noise Power" ; sn:Units = "dB(uncalibrated) x10" ; short sff(time, gate, vels) ; sff:Name = "Filtered Spectral Power" ; sff:Units = "dB(uncalibrated) x10" ; short snr(time, gate) ; snr:Name = "Signal to Noise Ratio" ; snr:Units = "dB x10" ; short Ze(time, gate) ; // total of Zef over Vel Ze:Name = "Reflectivity Factor" ; Ze:Units = "dBZ x10" ; float vel(time, gate) ; vel:Name = "Mean Radial Velocity" ; vel:Units = "m/s" ; float wid(time, gate) ; wid:Name = "Spectrum Width" ; wid:Units = "m/s" ; // global attributes: :NetCDFRevision = "UMass MIRSL FMCW V1.0" ; :RadarName = "UMa-FMCW" ; :PRF = 190.735f ; :Frequency = 2.92e+09f ; :Latitude = 34.6906f ; :Longitude = -86.8815f ; } 5.0 Data Remarks Remark 1: Intermittent small gaps in the data stream yield some uneven temporal sampling. All spectra consist of 12 complete processing blocks over 16.1 s, and the timestamp corresponds to the middle of the time interval. The timestamps themselves are not always evenly spaced (i.e. not every 16.1 sec). A few longer time gaps were due to power/communications outages. Remark 2: Raw power spectra (variable "sf") contain spurs/spikes due to interference from high-voltage switching power supplies in the travelling wave tube transmitter. A median filtering method has been used to eliminate most of these (variable "sff") though some strong spurs still persist. In the presence of strong scattering, these spurs are overwhelmed. An auxiliary file, flag2.nc, indicates (1/0) the range bins containing persistent spurs. Remark 3: Although the system's equivalent noise floor corresponds to a constant value reported above, the receiver gain is range dependent owing to receiver filter characteristics. Thus, the noise-floor in raw spectra also varies with range. Remark 4: The reflectivity units are reflectivity factor (Z), as that is most familiar to radar meteorologists. For clear-air scattering from refractive index turbulence (Bragg scattering), the more appropriate unit is the refractive index structure parameter, Cn2. Both of these are related to Volume Reflectivity, eta (radar cross section per unit volume), as follows: eta = pi^5/lambda^4 |Kw|^2 Z (Rayleigh Scatter) eta = 0.38 lambda^(-1/3) Cn2 (Bragg Scatter) Remark 5: During heavier precipitation episodes, the vertical velocity is aliased, yielding positive (upward) velocities. This has been left as is, and it is up to the user to de-alias velocities appropriately. 6.0 References Ince, T., S. J. Frasier, A. Muschinski, A. L. Pazmany, 2003. "An S-Band Frequency-Modulated Continuous-Wave Boundary Layer Profiler: Description and Initial Results," Radio Sci., 38(4), 1072, doi:10.1029/2002RS002753.