Reading cloud radar data

The high resolution data has about 500 MB per file, which when read in over a remote source can lead to long wait times. To reduce the wait times the data can be read in lazily using dask. Intake will this do this by default. Let’s obtain the EUREC4A intake catalog:

import eurec4a
cat = eurec4a.get_intake_catalog(use_ipfs="QmahMN2wgPauHYkkiTGoG2TpPBmj3p5FoYJAq9uE9iXT9N")

Available products

The LIMRAD94 cloud radar offers multiple products which can be accessed using names and additional parameters. Let’s see which products and parameters are available for the cloud radar. For the parameters, we are also interested in their valid range:

for key, source in cat.Meteor.LIMRAD94.items():
    desc = source.describe()
    user_parameters = desc.get("user_parameters", [])
    if len(user_parameters) > 0:
        params = " (" + ", ".join(p["name"] for p in user_parameters) + ")"
    else:
        params = ""
    print(f"{key}{params}: {desc['description']}")
    for parameter in user_parameters:
        print(f"    {parameter['name']}: {parameter['min']} ... {parameter['max']} default: {parameter['default']}")
    print()

high_res (date, version): daily heave corrected original resolution cloudradar data
    date: 2020-01-17 00:00:00 ... 2020-02-29 00:00:00 default: 2020-02-01 00:00:00
    version: 1.0 ... 1.1 default: 1.1

low_res (date): daily 30m 30s averaged radar reflectivity
    date: 2020-01-17 00:00:00 ... 2020-02-29 00:00:00 default: 2020-02-01 00:00:00

Radar reflectivity

We’ll have a look at the high_res data in version 1.1. We’ll keep the default date for simplicity:

ds = cat.Meteor.LIMRAD94.high_res(version=1.1).to_dask()
ds

/usr/share/miniconda3/envs/how_to_eurec4a/lib/python3.12/site-packages/intake_xarray/base.py:21: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  'dims': dict(self._ds.dims),

<xarray.Dataset> Size: 720MB
Dimensions:                    (chirp: 3, time: 44447, range: 367)
Coordinates:
  * range                      (range) float32 1kB 313.0 335.4 ... 1.296e+04
  * time                       (time) datetime64[ns] 356kB 2020-02-01T00:00:0...
Dimensions without coordinates: chirp
Data variables: (12/23)
    NumSpectraAveraged         (chirp) float32 12B dask.array<chunksize=(3,), meta=np.ndarray>
    Numfft                     float32 4B ...
    Nyquist_velocity           (chirp) float32 12B dask.array<chunksize=(3,), meta=np.ndarray>
    Skew                       (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
    SurfRelHum                 (time) float32 178kB dask.array<chunksize=(44447,), meta=np.ndarray>
    Zh                         (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
    ...                         ...
    range_offsets              (chirp) float64 24B dask.array<chunksize=(3,), meta=np.ndarray>
    time_shift                 (time, chirp) float32 533kB dask.array<chunksize=(22224, 3), meta=np.ndarray>
    v                          (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
    v_no_rolling_mean_applied  (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
    v_uncor                    (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
    width                      (time, range) float32 65MB dask.array<chunksize=(5556, 46), meta=np.ndarray>
Attributes: (12/20)
    Conventions:    CF-1.8
    _FillValue:     -999.0
    calibrations:   remove Precip. ghost: False\n, remove curtain ghost: True...
    campaign_id:    EUREC4A
    contact:        heike.kalesse@uni-leipzig.de
    day:            1
    ...             ...
    source:         94 GHz Cloud Radar LIMRAD94\nRadar type: Frequency Modula...
    system:         LIMRAD94
    title:          LIMRAD94 (SLDR) Doppler Cloud Radar, calibrated file
    version:        Variable names and dimensions prepared for upload to Aeri...
    version_id:     v1.1
    year:           2020

xarray.Dataset

• Dimensions:
  • chirp: 3
  • time: 44447
  • range: 367
• Coordinates: (2)
  • range
    (range)
    float32
    313.0 335.4 ... 1.292e+04 1.296e+04

    long_name :

    Range from antenna to the centre of each range gate

    units :

    m

    array([  313.0271 ,   335.3862 ,   357.74527, ..., 12878.56   , 12920.646  ,
           12962.733  ], dtype=float32)

  • time
    (time)
    datetime64[ns]
    2020-02-01T00:00:06.700000047 .....

    long_name :

    seconds since 2020-01-01 00:00:00 UTC

    array(['2020-02-01T00:00:06.700000047', '2020-02-01T00:00:08.570000171',
           '2020-02-01T00:00:10.450000047', ..., '2020-02-01T23:59:50.815000057',
           '2020-02-01T23:59:52.684999943', '2020-02-01T23:59:54.545000076'],
          dtype='datetime64[ns]')

• Data variables: (23)
  • NumSpectraAveraged
    (chirp)
    float32
    dask.array<chunksize=(3,), meta=np.ndarray>

    long_name :

    Number of spectral averages

    units :

    1

    Array Chunk Bytes 12 B 12 B Shape (3,) (3,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • Numfft
    ()
    float32
    ...

    long_name :

    Number of points in FFT

    units :

    1

    [1 values with dtype=float32]

  • Nyquist_velocity
    (chirp)
    float32
    dask.array<chunksize=(3,), meta=np.ndarray>

    long_name :

    Unambiguous Doppler velocity (+/-)

    units :

    m s-1

    Array Chunk Bytes 12 B 12 B Shape (3,) (3,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • Skew
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    long_name :

    Skewness

    plot_range :

    [-1, 1]

    units :

    1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • SurfRelHum
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Relative humidity from weather station

    units :

    %

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • Zh
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    Calibrated reflectivity. Calibration convention: in the absence of attenuation, a cloud at 273 K containing one million 100-micron droplets per cubic metre will have a reflectivity of 0 dBZ at all frequencies.

    long_name :

    Radar reflectivity factor

    plot_range :

    [-60, 20]

    plot_scale :

    linear

    units :

    dBZ

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • altitude
    ()
    float32
    ...

    long_name :

    Height of instrument above mean sea level

    units :

    m

    [1 values with dtype=float32]

  • bt
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Direct detection brightness temperature at 89 GHz

    units :

    K

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • frequency
    ()
    float32
    ...

    long_name :

    Radar frequency

    units :

    GHz

    [1 values with dtype=float32]

  • heave_cor
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This is the velocity by which the original Doppler spectrum was corrected by. The heave rate is subtracted from the Doppler velocity, meaning the spectra is shifted to the left for positive heave rates and to the right for negative heave rates.

    long_name :

    Heave rate correction

    plot_range :

    [-7, 7]

    plot_scale :

    linear

    unit_html :

    m s^-1

    units :

    m s-1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • heave_cor_bins
    (time, range)
    float64
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This is the number of bins by which the original Doppler spectrum was shifted by. Positive values shift the spectrum to the left, while negative values shift the spectrum to the right.

    long_name :

    Heave rate correction in Doppler spectra bins

    plot_scale :

    linear

    units :

    1

    Array Chunk Bytes 124.45 MiB 1.95 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float64 numpy.ndarray

  • kurt
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    long_name :

    Kurtosis

    plot_range :

    [0, 3]

    units :

    1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • latitude
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Latitude

    units :

    degrees_north

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • ldr
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This parameter is the ratio of cross-polar to co-polar reflectivity.

    long_name :

    Linear depolarisation ratio

    plot_range :

    [-30.0, 0.0]

    units :

    dB

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • longitude
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Longitude

    units :

    degrees_east

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • lwp
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Liquid water path

    units :

    g m-2

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • rain
    (time)
    float32
    dask.array<chunksize=(44447,), meta=np.ndarray>

    long_name :

    Rain rate from weather station

    units :

    mm h-1

    Array Chunk Bytes 173.62 kiB 173.62 kiB Shape (44447,) (44447,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • range_offsets
    (chirp)
    float64
    dask.array<chunksize=(3,), meta=np.ndarray>

    long_name :

    chirp sequences start index array in altitude layer array

    units :

    1

    Array Chunk Bytes 24 B 24 B Shape (3,) (3,) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • time_shift
    (time, chirp)
    float32
    dask.array<chunksize=(22224, 3), meta=np.ndarray>

    long_name :

    Time shift between radar and ship calculated for each hour and chirp

    units :

    s

    Array Chunk Bytes 520.86 kiB 260.44 kiB Shape (44447, 3) (22224, 3) Dask graph 2 chunks in 2 graph layers Data type float32 numpy.ndarray

  • v
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This parameter is the radial component of the velocity, with positive velocities are away from the radar (i.e. up) and negative velocities moving towards the radar (i.e. down). It was corrected for the ships heave motion. A rolling average over 3 time steps has been applied to it.

    folding_velocity :

    7.261648654937744

    long_name :

    Best estimate of averaged mean Doppler velocity

    plot_range :

    [-7, 7]

    plot_scale :

    linear

    unit_html :

    m s^-1

    units :

    m s-1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • v_no_rolling_mean_applied
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This parameter is the radial component of the velocity, with positive velocities are away from the radar (i.e. up) and negative velocities moving towards the radar (i.e. down). It was corrected for the ships heave motion but no rolling mean was applied.

    folding_velocity :

    7.261648654937744

    long_name :

    Heave corrected mean Doppler velocity

    plot_range :

    [-7, 7]

    plot_scale :

    linear

    unit_html :

    m s^-1

    units :

    m s-1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • v_uncor
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This parameter is the uncorrected radial component of the velocity, with positive velocities are away from the radar (i.e. up) and negative velocities moving towards the radar (i.e. down).

    folding_velocity :

    7.261648654937744

    long_name :

    Uncorrected Mean Doppler velocity

    plot_range :

    [-7.0, 7.0]

    plot_scale :

    linear

    unit_html :

    m s^-1

    units :

    m s-1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

  • width
    (time, range)
    float32
    dask.array<chunksize=(5556, 46), meta=np.ndarray>

    comment :

    This parameter is the standard deviation of the reflectivity-weighted velocities in the radar pulse volume.

    long_name :

    Spectral width

    plot_range :

    [0.01, 3.0]

    plot_scale :

    logarithmic

    unit_html :

    m s^-1

    units :

    m s-1

    Array Chunk Bytes 62.23 MiB 0.97 MiB Shape (44447, 367) (5556, 46) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (2)
  • range
    PandasIndex

    PandasIndex(Index([  313.027099609375,  335.3861999511719,  357.7452697753906,
            380.1043395996094, 402.46343994140625,   424.822509765625,
           447.18157958984375,  469.5406799316406,  491.8997497558594,
            514.2588500976562,
           ...
             12583.9521484375,      12626.0390625,   12668.1259765625,
             12710.2119140625,    12752.298828125,   12794.3857421875,
               12836.47265625,   12878.5595703125,    12920.646484375,
             12962.7333984375],
          dtype='float32', name='range', length=367))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-01 00:00:06.700000047',
                   '2020-02-01 00:00:08.570000171',
                   '2020-02-01 00:00:10.450000047',
                   '2020-02-01 00:00:12.320000171',
                   '2020-02-01 00:00:14.179999828',
                   '2020-02-01 00:00:16.299999952',
                   '2020-02-01 00:00:18.190000057',
                   '2020-02-01 00:00:20.049999952',
                   '2020-02-01 00:00:21.920000076',
                   '2020-02-01 00:00:23.779999971',
                   ...
                   '2020-02-01 23:59:37.445000171',
                   '2020-02-01 23:59:39.315000057',
                   '2020-02-01 23:59:41.184999943',
                   '2020-02-01 23:59:43.054999828',
                   '2020-02-01 23:59:44.945000171',
                   '2020-02-01 23:59:46.985000133',
                   '2020-02-01 23:59:48.865000009',
                   '2020-02-01 23:59:50.815000057',
                   '2020-02-01 23:59:52.684999943',
                   '2020-02-01 23:59:54.545000076'],
                  dtype='datetime64[ns]', name='time', length=44447, freq=None))

• Attributes: (20)

  Conventions :

  CF-1.8

  _FillValue :

  -999.0

  calibrations :

  remove Precip. ghost: False , remove curtain ghost: True despeckle: True , number of standard deviations above noise: 6.0 spectra heave corrected: True spectra heave corrected with version: ca spectra dealiased: True

  campaign_id :

  EUREC4A

  contact :

  heike.kalesse@uni-leipzig.de

  day :

  1

  description :

  Concatenated data files of LIMRAD 94GHz - FMCW Radar, filters applied: ghost-echo, despeckle, use only main peak, spectra corrected for heave motion of ship and then dealiased

  history :

  Created Thu Feb 18 23:16:55 2021

  institution :

  Leipzig Institute for Meteorology (LIM), Leipzig, Germany

  instrument_id :

  LIMRAD94

  location :

  RV-Meteor

  month :

  2

  platform_id :

  Meteor

  reference :

  W Band Cloud Radar LIMRAD94 Documentation and User Manual provided by manufacturer RPG Radiometer Physics GmbH Information about system also available at https://www.radiometer-physics.de/

  source :

  94 GHz Cloud Radar LIMRAD94 Radar type: Frequency Modulated Continuous Wave, Transmitter power 1.5 W typical (solid state amplifier) Antenna Type: Bi-static Cassegrain with 500 mm aperture Beam width: 0.48deg FWHM

  system :

  LIMRAD94

  title :

  LIMRAD94 (SLDR) Doppler Cloud Radar, calibrated file

  version :

  Variable names and dimensions prepared for upload to Aeris data center. 1.1: updated metadata

  version_id :

  v1.1

  year :

  2020

Explore the dataset and choose the variable you want to work with. The variables are loaded lazily, i.e. only when their content is really required to complete the operation. An example which forces the data to load is plotting, in this case only the radar reflectivity will be loaded.

%matplotlib inline
import matplotlib.pyplot as plt
import pathlib
plt.style.use(pathlib.Path("./mplstyle/book"))

ds.Zh.plot(x='time', cmap="Spectral_r")  # plot the variable with time as the x axis
plt.ylim(0, 3000);