Orientation to Earthdata Cloud Access

Orientation to Earthdata Cloud Access#

Tutorial Lead: Anna Windle (NASA, SSAI)
Last updated: August 1, 2025

An Earthdata Login account is required to access data from the NASA Earthdata system, including NASA ocean color data.

Summary#

In this example we will use the earthaccess package to search for PACE OCI products on NASA Earthdata. The earthaccess package, published on the Python Package Index and conda-forge, facilitates discovery and use of all NASA Earth Science data products by providing an abstraction layer for NASA’s Common Metadata Repository (CMR) API and by simplifying requests to NASA’s Earthdata Cloud. Searching for data is more approachable using earthaccess than low-level HTTP requests, and the same goes for S3 requests.

In short, earthaccess helps authenticate with an Earthdata Login, makes search easier, and provides a stream-lined way to load data into xarray containers. For more on earthaccess, visit the documentation site. Be aware that earthaccess is under active development.

To understand the discussions below on downloading and opening data, we need to clearly understand where our notebook is running. There are three cases to distinguish:

The notebook is running on the local host. For instance, you started a Jupyter server on your laptop.
The notebook is running on a remote host, but it does not have direct access to the AWS us-west-2 region. For instance, you are running in Google Colab, which is run on the Google cloud.
The notebook is running on a remote host that does have direct access to the NASA Earthdata Cloud (AWS us-west-2 region). This is the case for the PACE Hackweek with CryoCloud.

Learning Objectives#

At the end of this notebook you will know:

How to store your NASA Earthdata Login credentials with earthaccess
How to use earthaccess to search for OCI data using search filters
How to download OCI data, but only when you need to

1. Setup#

We begin by importing the packages used in this notebook.

import cartopy.crs as ccrs
import earthaccess
import matplotlib.pyplot as plt
import numpy as np
import xarray as xr

back to top

2. NASA Earthdata Authentication#

Next, we authenticate using our Earthdata Login credentials. Authentication is not needed to search publicly available collections in Earthdata, but is always needed to access data. We can use the login method from the earthaccess package. This will create an authenticated session when we provide a valid Earthdata Login username and password. The earthaccess package will search for credentials defined by environmental variables or within a .netrc file saved in the home directory. If credentials are not found, an interactive prompt will allow you to input credentials.

The persist=True argument ensures any discovered credentials are stored in a .netrc file, so the argument is not necessary (but it’s also harmless) for subsequent calls to earthaccess.login.

auth = earthaccess.login(persist=True)

back to top

3. Search for Data#

Collections on NASA Earthdata are discovered with the search_datasets function, which accepts an instrument filter as an easy way to get started. Each of the items in the list of collections returned has a “short-name”.

results = earthaccess.search_datasets(instrument="oci")

for item in results:
    summary = item.summary()
    print(summary["short-name"])

PACE_OCI_L1B_SCI
PACE_OCI_L2_AOP
PACE_OCI_L2_BGC_NRT
PACE_OCI_L0_SCI
PACE_OCI_L1A_SCI
PACE_OCI_L1C_SCI
PACE_OCI_L2_AOP_NRT
PACE_OCI_L2_BGC
PACE_OCI_L2_CLOUD
PACE_OCI_L2_CLOUD_MASK
PACE_OCI_L2_CLOUD_MASK_NRT
PACE_OCI_L2_CLOUD_NRT
PACE_OCI_L2_IOP
PACE_OCI_L2_IOP_NRT
PACE_OCI_L2_LANDVI
PACE_OCI_L2_LANDVI_NRT
PACE_OCI_L2_PAR
PACE_OCI_L2_PAR_NRT
PACE_OCI_L2_SFREFL
PACE_OCI_L2_SFREFL_NRT
PACE_OCI_L3B_AVW
PACE_OCI_L3B_AVW_NRT
PACE_OCI_L3B_CARBON
PACE_OCI_L3B_CARBON_NRT
PACE_OCI_L3B_CHL
PACE_OCI_L3B_CHL_NRT
PACE_OCI_L3B_FLH
PACE_OCI_L3B_FLH_NRT
PACE_OCI_L3B_IOP
PACE_OCI_L3B_IOP_NRT
PACE_OCI_L3B_KD
PACE_OCI_L3B_KD_NRT
PACE_OCI_L3B_LANDVI
PACE_OCI_L3B_LANDVI_NRT
PACE_OCI_L3B_PAR
PACE_OCI_L3B_PAR_NRT
PACE_OCI_L3B_POC
PACE_OCI_L3B_POC_NRT
PACE_OCI_L3B_RRS
PACE_OCI_L3B_RRS_NRT
PACE_OCI_L3B_SFREFL
PACE_OCI_L3B_SFREFL_NRT
PACE_OCI_L3M_AVW
PACE_OCI_L3M_AVW_NRT
PACE_OCI_L3M_CARBON
PACE_OCI_L3M_CARBON_NRT
PACE_OCI_L3M_CHL
PACE_OCI_L3M_CHL_NRT
PACE_OCI_L3M_CLOUD
PACE_OCI_L3M_CLOUD_NRT
PACE_OCI_L3M_FLH
PACE_OCI_L3M_FLH_NRT
PACE_OCI_L3M_IOP
PACE_OCI_L3M_IOP_NRT
PACE_OCI_L3M_KD
PACE_OCI_L3M_KD_NRT
PACE_OCI_L3M_LANDVI
PACE_OCI_L3M_LANDVI_NRT
PACE_OCI_L3M_MOANA
PACE_OCI_L3M_MOANA_NRT
PACE_OCI_L3M_PAR
PACE_OCI_L3M_PAR_NRT
PACE_OCI_L3M_POC
PACE_OCI_L3M_POC_NRT
PACE_OCI_L3M_RRS
PACE_OCI_L3M_RRS_NRT
PACE_OCI_L3M_SFREFL
PACE_OCI_L3M_SFREFL_NRT

The short name can also be found on Eartdata Search, directly under the collection name, after clicking on the "i" button for a collection in any search result.

Next, we use the search_data function to find granules within a collection. Let’s use the short_name for the PACE/OCI Level-2 near real time (NRT) product for biogeochemical properties (although you can search for granules across collections too).

The count argument limits the number of granules whose metadata is returned and stored in the results list.

results = earthaccess.search_data(
    short_name="PACE_OCI_L2_BGC",
    count=1,
)
len(results)

We can refine our search by passing more parameters that describe the spatiotemporal domain of our use case. Here, we use the temporal parameter to request a date range and the bounding_box parameter to request granules that intersect with a bounding box. We can even provide a cloud_cover threshold to limit files that have a lower percetnage of cloud cover. We do not provide a count, so we’ll get all granules that satisfy the constraints.

tspan = ("2024-07-01", "2024-07-31")
bbox = (-76.75, 36.97, -75.74, 39.01)
clouds = (0, 50)

results = earthaccess.search_data(
    short_name="PACE_OCI_L2_BGC",
    temporal=tspan,
    bounding_box=bbox,
    cloud_cover=clouds,
)
len(results)

Displaying results shows the direct download link: try it! The link will download one granule to your local machine, which may or may not be what you want to do. Even if you are running the notebook on a remote host, this download link will open a new browser tab or window and offer to save a file to your local machine. If you are running the notebook locally, this may be of use. However, in the next section we’ll see how to download all the results with one command.

results[0]

Data: PACE_OCI.20240701T175112.L2.OC_BGC.V3_0.nc

Size: 13.25 MB

Cloud Hosted: True

back to top

5. Open L3M Data#

Let’s use earthaccess to open some L3 mapped chlorophyll a granules. We will use a new search filter available in earthaccess.search_data: the granule_name argument accepts strings with the “*” wildcard. We need this to distinguish daily (“DAY”) from eight-day (“8D”) composites, as well as to get the 0.1 degree resolution projections.

tspan = ("2024-04-12", "2024-04-24")

results = earthaccess.search_data(
    short_name="PACE_OCI_L3M_CHL",
    temporal=tspan,
    granule_name="*.DAY.*.0p1deg.*",
)

paths = earthaccess.open(results)

Since L3M do not have groups, we can open the first file as a xarray.Dataset:

Because the L3M variables have lat and lon coordinates, it’s possible to stack multiple granules along a new dimension that corresponds to time. Instead of xr.open_dataset, we can use xr.open_mfdataset to create a single xarray.Dataset (the “mf” in open_mfdataset stands for multiple files) from an array of paths.

The paths list is sorted temporally by default, which means the shape of the paths array specifies the way we need to tile the files together into larger arrays. We specify combine=”nested” to combine the files according to the shape of the array of files (or file-like objects), even though paths is not a “nested” list in this case. The concat_dim=”date” argument generates a new dimension in the combined dataset, because “date” is not an existing dimension in the individual files.

dataset = xr.open_mfdataset(
    paths,
    combine="nested",
    concat_dim="date",
)
dataset

<xarray.Dataset> Size: 337MB
Dimensions:  (date: 13, lat: 1800, lon: 3600, rgb: 3, eightbitcolor: 256)
Coordinates:
  * lat      (lat) float32 7kB 89.95 89.85 89.75 89.65 ... -89.75 -89.85 -89.95
  * lon      (lon) float32 14kB -179.9 -179.9 -179.8 ... 179.8 179.9 180.0
Dimensions without coordinates: date, rgb, eightbitcolor
Data variables:
    chlor_a  (date, lat, lon) float32 337MB dask.array<chunksize=(1, 512, 1024), meta=np.ndarray>
    palette  (date, rgb, eightbitcolor) uint8 10kB dask.array<chunksize=(1, 3, 256), meta=np.ndarray>
Attributes: (12/64)
    product_name:                      PACE_OCI.20240412.L3m.DAY.CHL.V3_0.chl...
    instrument:                        OCI
    title:                             OCI Level-3 Standard Mapped Image
    project:                           Ocean Biology Processing Group (NASA/G...
    platform:                          PACE
    source:                            satellite observations from OCI-PACE
    ...                                ...
    identifier_product_doi:            10.5067/PACE/OCI/L3M/CHL/3.0
    keywords:                          Earth Science > Oceans > Ocean Chemist...
    keywords_vocabulary:               NASA Global Change Master Directory (G...
    data_bins:                         533615
    data_minimum:                      0.0058789942
    data_maximum:                      90.5119

A common reason to generate a single dataset from multiple, daily images is to create a composite. Compare the map from a single day …

chla = np.log10(dataset["chlor_a"])
chla.attrs.update(
    {
        "units": f'log({dataset["chlor_a"].attrs["units"]})',
    }
)
plot = chla.sel({"date": 0}).plot(aspect=2, size=4, cmap="GnBu_r")

../../_images/c0a6d1b01746a822a440a99aa8e3ef6ebb05e1372c172ec66ad04a0c14a08fd9.png

… to a map of average values, skipping “NaN” values that result from clouds.

chla_avg = chla.mean("date", keep_attrs=True)
plot = chla_avg.plot(aspect=2, size=4, cmap="GnBu_r")

../../_images/795b90a72338307b80ee551d4f01f458bd0ab84769fd759438233a93ab98d433.png

Orientation to Earthdata Cloud Access

Contents

Orientation to Earthdata Cloud Access#

Summary#

Learning Objectives#

Contents#

1. Setup#

2. NASA Earthdata Authentication#

3. Search for Data#

4. Open L2 Data#

5. Open L3M Data#

6. Download Data#