Flight tracks

The P3 flew 95 hours of observations over eleven flights, many of which were coordinated with the NOAA research ship R/V Ronald H. Brown and autonomous platforms deployed from the ship. Each flight contained a mixture of sampling strategies including: high-altitude circles with frequent dropsonde deployment to characterize the large-scale environment; slow descents and ascents to measure the distribution of water vapor and its isotopic composition; stacked legs aimed at sampling the microphysical and thermodynamic state of the boundary layer; and offset straight flight legs for observing clouds and the ocean surface with remote sensing instruments and the thermal structure of the ocean with in situ sensors dropped from the plane.

As a result of this diverse sampling the flight tracks are much more variable then for most of the other aircraft.

General setup:

import xarray as xr
import numpy as np
import datetime
# Related to plotting
import matplotlib.pyplot as plt
%matplotlib inline

Now access the flight track data.

import eurec4a
cat = eurec4a.get_intake_catalog()

Mapping takes quite some setup. Maybe we’ll encapsulate this later but for now we repeat code in each notebook.

import matplotlib as mpl
import matplotlib.ticker as mticker

import cartopy.crs as ccrs
from   cartopy.feature import LAND
from   cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER
def ax_to_map(ax, lon_w = -60.5, lon_e = -49, lat_s = 10, lat_n = 16.5):
    # Defining boundaries of the plot
    ax.set_extent([lon_w,lon_e,lat_s,lat_n]) # lon west, lon east, lat south, lat north

def set_up_map(plt, lon_w = -60.5, lon_e = -49, lat_s = 10, lat_n = 16.5):
    ax = plt.axes(projection=ccrs.PlateCarree())
    ax_to_map(ax, lon_w, lon_e, lat_s, lat_n)

def add_gridlines(ax):
    # Assigning axes ticks
    xticks = np.arange(-65,0,2.5)
    yticks = np.arange(0,25,2.5)
    gl = ax.gridlines(crs=ccrs.PlateCarree(), draw_labels=True, linewidth=1,
                      color='black', alpha=0.5, linestyle='dotted')
    gl.xlocator = mticker.FixedLocator(xticks)
    gl.ylocator = mticker.FixedLocator(yticks)
    gl.xformatter = LONGITUDE_FORMATTER
    gl.yformatter = LATITUDE_FORMATTER
    gl.xlabel_style = {'size': 10, 'color': 'k'}
    gl.ylabel_style = {'size': 10, 'color': 'k'}
    gl.right_labels = False
    gl.bottom_labels = False
    gl.xlabel = {'Latitude'}

What days did the P-3 fly on? We can find out via the flight segmentation files.

# On what days did the P-3 fly? These are UTC date
all_flight_segments = eurec4a.get_flight_segments()
flight_dates = np.unique([np.datetime64(flight["takeoff"]).astype("datetime64[D]")
                          for flight in all_flight_segments["P3"].values()])

Now set up colors to code each flight date during the experiment. One could choose a categorical palette so the colors were as different from each other as possible. Here we’ll choose from a continuous set that spans the experiment so days that are close in time are also close in color.

# Like mpl.colors.Normalize but works also with datetime64 objects
def mk_norm(vmin, vmax):
    def norm(values):
        return (values - vmin) / (vmax - vmin)
    return norm
norm = mk_norm(np.datetime64("2020-01-15"),

# color map for things coded by flight date
#   Sample from a continuous color map running from start to end of experiment
def color_of_day(day):
    return plt.cm.viridis(norm(day), alpha=0.9)

For plotting purposes it’ll be handy to define a one-day time window and to convert between date/time formats

one_day = np.timedelta64(1, "D")

def to_datetime(dt64):
    epoch = np.datetime64("1970-01-01")
    second = np.timedelta64(1, "s")
    return datetime.datetime.utcfromtimestamp((dt64 - epoch) / second)

Most platforms available from the EUREC4A intake catalog have a tracks element but we’ll use the flight-level data instead. We’re using xr.concat() with one dask array per day to avoid loading the whole dataset into memory at once.

nav_data = xr.concat([entry.to_dask().chunk() for entry in cat.P3.flight_level.values()],
                     dim = "time")
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>502 Proxy Error</title></head><body><h1>Proxy Error</h1><p>The proxy server received an invalidresponse from an upstream server.<br />The proxy server could not handle the request <em><a href="/thredds-ocean/dodsC/psl/atomic/p3/meteorology/EUREC4A_ATOMIC_P3_Flight-Level_20200117_v1.1.nc.dds">GET&nbsp;/thredds-ocean/dodsC/psl/atomic/p3/meteorology/EUREC4A_ATOMIC_P3_Flight-Level_20200117_v1.1.nc.dds</a></em>.<p>Reason: <strong>Error reading from remote server</strong></p></p><p>Additionally, a 502 Bad Gatewayerror was encountered while trying to use an ErrorDocument to handle the request.</p></body></html>
KeyError                                  Traceback (most recent call last)
/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock)
    198             try:
--> 199                 file = self._cache[self._key]
    200             except KeyError:

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/lru_cache.py in __getitem__(self, key)
     52         with self._lock:
---> 53             value = self._cache[key]
     54             self._cache.move_to_end(key)

KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('https://www.ncei.noaa.gov/thredds-ocean/dodsC/psl/atomic/p3/meteorology/EUREC4A_ATOMIC_P3_Flight-Level_20200117_v1.1.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False))]

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/tmp/ipykernel_5906/1195062795.py in <module>
----> 1 nav_data = xr.concat([entry.to_dask().chunk() for entry in cat.P3.flight_level.values()],
      2                      dim = "time")

/tmp/ipykernel_5906/1195062795.py in <listcomp>(.0)
----> 1 nav_data = xr.concat([entry.to_dask().chunk() for entry in cat.P3.flight_level.values()],
      2                      dim = "time")

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake_xarray/base.py in to_dask(self)
     67     def to_dask(self):
     68         """Return xarray object where variables are dask arrays"""
---> 69         return self.read_chunked()
     71     def close(self):

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake_xarray/base.py in read_chunked(self)
     42     def read_chunked(self):
     43         """Return xarray object (which will have chunks)"""
---> 44         self._load_metadata()
     45         return self._ds

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake/source/base.py in _load_metadata(self)
    234         """load metadata only if needed"""
    235         if self._schema is None:
--> 236             self._schema = self._get_schema()
    237             self.dtype = self._schema.dtype
    238             self.shape = self._schema.shape

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake_xarray/base.py in _get_schema(self)
     17         if self._ds is None:
---> 18             self._open_dataset()
     20             metadata = {

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake_xarray/opendap.py in _open_dataset(self)
     91     def _open_dataset(self):
     92         import xarray as xr
---> 93         store = self._get_store()
     94         self._ds = xr.open_dataset(store, chunks=self.chunks, **self._kwargs)

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/intake_xarray/opendap.py in _get_store(self)
     81                     "Opendap session requires 'pydap' engine."
     82                 )
---> 83             return xr.backends.NetCDF4DataStore.open(self.urlpath)
     84         elif self.engine == "pydap":
     85             return xr.backends.PydapDataStore.open(self.urlpath, session=session)

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in open(cls, filename, mode, format, group, clobber, diskless, persist, lock, lock_maker, autoclose)
    377             netCDF4.Dataset, filename, mode=mode, kwargs=kwargs
    378         )
--> 379         return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
    381     def _acquire(self, needs_lock=True):

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in __init__(self, manager, group, mode, lock, autoclose)
    325         self._group = group
    326         self._mode = mode
--> 327         self.format = self.ds.data_model
    328         self._filename = self.ds.filepath()
    329         self.is_remote = is_remote_uri(self._filename)

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in ds(self)
    386     @property
    387     def ds(self):
--> 388         return self._acquire()
    390     def open_store_variable(self, name, var):

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in _acquire(self, needs_lock)
    381     def _acquire(self, needs_lock=True):
--> 382         with self._manager.acquire_context(needs_lock) as root:
    383             ds = _nc4_require_group(root, self._group, self._mode)
    384         return ds

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/contextlib.py in __enter__(self)
    111         del self.args, self.kwds, self.func
    112         try:
--> 113             return next(self.gen)
    114         except StopIteration:
    115             raise RuntimeError("generator didn't yield") from None

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/file_manager.py in acquire_context(self, needs_lock)
    185     def acquire_context(self, needs_lock=True):
    186         """Context manager for acquiring a file."""
--> 187         file, cached = self._acquire_with_cache_info(needs_lock)
    188         try:
    189             yield file

/usr/share/miniconda/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock)
    203                     kwargs = kwargs.copy()
    204                     kwargs["mode"] = self._mode
--> 205                 file = self._opener(*self._args, **kwargs)
    206                 if self._mode == "w":
    207                     # ensure file doesn't get overriden when opened again

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.__init__()

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()

OSError: [Errno -70] NetCDF: DAP server error: b'https://www.ncei.noaa.gov/thredds-ocean/dodsC/psl/atomic/p3/meteorology/EUREC4A_ATOMIC_P3_Flight-Level_20200117_v1.1.nc'

A map showing each of the eleven flight tracks:

fig = plt.figure(figsize = (12,13.6))
ax  = set_up_map(plt)

for d in flight_dates:
    flight = nav_data.sel(time=slice(d, d + one_day))
    ax.plot(flight.lon, flight.lat,
            lw=2, alpha=0.5, c=color_of_day(d),
            transform=ccrs.PlateCarree(), zorder=7,

plt.legend(ncol=3, loc=(0.0,0.0), fontsize=14, framealpha=0.8, markerscale=5,
           title="Flight date (MM-DD-2020)")

Most dropsondes were deployed from regular dodecagons during the first part of the experiment with short turns after each dropsonde providing an off-nadir look at the ocean surface useful for calibrating the W-band radar. A change in pilots midway through the experiment led to dropsondes being deployed from circular flight tracks starting on 31 Jan. AXBTs were deployed in lawnmower patterns (parallel offset legs) with small loops sometimes employed to lengthen the time between AXBT deployment to allow time for data acquisition given the device’s slow fall speeds. Profiling and especially in situ cloud sampling legs sometimes deviated from straight paths to avoid hazardous weather.

Side view using the same color scale:

fig = plt.figure(figsize = (8.3,5))
ax = plt.axes()

for d in flight_dates:
    flight = nav_data.sel(time=slice(d, d + one_day))
    flight = flight.where(flight.alt > 80, drop=True) # Proxy for take-off time
    plt.plot(flight.time - flight.time.min(), flight.alt/1000.,
             lw=2, alpha=0.5, c=color_of_day(d),

plt.xticks(np.arange(10) * 3600 * 1e9, labels = np.arange(10))
ax.set_xlabel("Time after flight start (h)")
ax.set_ylabel("Aircraft altitude (km)")

Sondes were dropped from the P-3 at about 7.5 km, with each circle taking roughly an hour; transits were frequently performed at this level to conserve fuel. Long intervals near 3 km altitude were used to deploy AXBTs and/or characterize the ocean surface with remote sensing. Stepped legs indicate times devoted to in situ cloud sampling. On most flights the aircraft climbed quickly to roughly 7.5 km, partly to deconflict with other aircraft participating in the experiment. On the three night flights, however, no other aircraft were operating at take-off times and cloud sampling was performed first, nearer Barbados than on other flights.