PACE data simple manipulation

Author: Rui Jin (University of Washington (CICOES))

This tutorial will show you how to do basic data manipulations on Level-3 (gridded) PACE data that you have accessed with earthaccess.

First load the libraries.

import earthaccess
import xarray as xr
from xarray.backends.api import open_datatree
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import numpy as np

Find PACE data

Authenticate to NASA Earth Data using earthaccess and then load a month of data on particulate organic carbon (POC).

auth = earthaccess.login(persist=True)

tspan = ("2024-07-01", "2024-07-31")

results = earthaccess.search_data(
    short_name="PACE_OCI_L3M_POC_NRT",
    temporal=tspan,
    granule_name="*.DAY.*.0p1deg.*",
)

paths = earthaccess.open(results)

Load data

We can load the data using xarray and then select a single array.

dataset = xr.open_dataset(paths[0])
dataset

<xarray.Dataset> Size: 26MB
Dimensions:  (lat: 1800, lon: 3600, rgb: 3, eightbitcolor: 256)
Coordinates:
  * lat      (lat) float32 7kB 89.95 89.85 89.75 89.65 ... -89.75 -89.85 -89.95
  * lon      (lon) float32 14kB -179.9 -179.9 -179.8 ... 179.8 179.9 180.0
Dimensions without coordinates: rgb, eightbitcolor
Data variables:
    poc      (lat, lon) float32 26MB ...
    palette  (rgb, eightbitcolor) uint8 768B ...
Attributes: (12/64)
    product_name:                      PACE_OCI.20240701.L3m.DAY.POC.V2_0.poc...
    instrument:                        OCI
    title:                             OCI Level-3 Standard Mapped Image
    project:                           Ocean Biology Processing Group (NASA/G...
    platform:                          PACE
    source:                            satellite observations from OCI-PACE
    ...                                ...
    identifier_product_doi:            10.5067/PACE/OCI/L3M/POC/2.0
    keywords:                          Earth Science > Oceans > Ocean Chemist...
    keywords_vocabulary:               NASA Global Change Master Directory (G...
    data_bins:                         745741
    data_minimum:                      18.199707
    data_maximum:                      1989.1997

xarray.Dataset

• Dimensions:
  • lat: 1800
  • lon: 3600
  • rgb: 3
  • eightbitcolor: 256
• Coordinates: (2)
  • lat
    (lat)
    float32
    89.95 89.85 89.75 ... -89.85 -89.95

    long_name :

    Latitude

    units :

    degrees_north

    standard_name :

    latitude

    valid_min :

    -90.0

    valid_max :

    90.0

    array([ 89.95    ,  89.85    ,  89.75    , ..., -89.75    , -89.850006,
           -89.950005], dtype=float32)

  • lon
    (lon)
    float32
    -179.9 -179.9 ... 179.9 180.0

    long_name :

    Longitude

    units :

    degrees_east

    standard_name :

    longitude

    valid_min :

    -180.0

    valid_max :

    180.0

    array([-179.95   , -179.85   , -179.75   , ...,  179.75   ,  179.85   ,
            179.95001], dtype=float32)

• Data variables: (2)
  • poc
    (lat, lon)
    float32
    ...

    long_name :

    Particulate Organic Carbon, D. Stramski, 2022 (hybrid version)

    units :

    mg m^-3

    valid_min :

    -32000

    valid_max :

    -22000

    reference :

    Stramski, D., et al. "Ocean color algorithms to estimate the concentration of particulate organic carbon in surface waters of the global ocean in support of a long-term data record from multiple satellite missions." Remote Sensing of Environment 269 (2022)

    display_scale :

    log

    display_min :

    10.0

    display_max :

    1000.0

    [6480000 values with dtype=float32]

  • palette
    (rgb, eightbitcolor)
    uint8
    ...

    [768 values with dtype=uint8]

• Indexes: (2)
  • lat
    PandasIndex

    PandasIndex(Index([ 89.94999694824219,   89.8499984741211,              89.75,
             89.6500015258789,  89.55000305175781,  89.44999694824219,
             89.3499984741211,              89.25,   89.1500015258789,
            89.05000305175781,
           ...
           -89.05000305175781,  -89.1500015258789,             -89.25,
           -89.35000610351562, -89.45000457763672, -89.55000305175781,
            -89.6500015258789,             -89.75, -89.85000610351562,
           -89.95000457763672],
          dtype='float32', name='lat', length=1800))

  • lon
    PandasIndex

    PandasIndex(Index([ -179.9499969482422, -179.85000610351562,             -179.75,
           -179.64999389648438,  -179.5500030517578,  -179.4499969482422,
           -179.35000610351562,             -179.25, -179.14999389648438,
            -179.0500030517578,
           ...
             179.0500030517578,  179.15000915527344,              179.25,
            179.35000610351562,  179.45001220703125,   179.5500030517578,
            179.65000915527344,              179.75,  179.85000610351562,
            179.95001220703125],
          dtype='float32', name='lon', length=3600))

• Attributes: (64)

  product_name :

  PACE_OCI.20240701.L3m.DAY.POC.V2_0.poc.0p1deg.NRT.nc

  instrument :

  OCI

  title :

  OCI Level-3 Standard Mapped Image

  project :

  Ocean Biology Processing Group (NASA/GSFC/OBPG)

  platform :

  PACE

  source :

  satellite observations from OCI-PACE

  temporal_range :

  day

  processing_version :

  2.0

  date_created :

  2024-07-04T20:03:09.000Z

  history :

  l3mapgen par=PACE_OCI.20240701.L3m.DAY.POC.V2_0.poc.0p1deg.NRT.nc.param

  l2_flag_names :

  ATMFAIL,LAND,HILT,HISATZEN,STRAYLIGHT,CLDICE,COCCOLITH,LOWLW,CHLWARN,CHLFAIL,NAVWARN,MAXAERITER,ATMWARN,HISOLZEN,NAVFAIL,FILTER,HIGLINT

  time_coverage_start :

  2024-06-30T23:49:46.000Z

  time_coverage_end :

  2024-07-02T01:52:44.000Z

  start_orbit_number :

  0

  end_orbit_number :

  0

  map_projection :

  Equidistant Cylindrical

  latitude_units :

  degrees_north

  longitude_units :

  degrees_east

  northernmost_latitude :

  90.0

  southernmost_latitude :

  -90.0

  westernmost_longitude :

  -180.0

  easternmost_longitude :

  180.0

  geospatial_lat_max :

  90.0

  geospatial_lat_min :

  -90.0

  geospatial_lon_max :

  180.0

  geospatial_lon_min :

  -180.0

  latitude_step :

  0.1

  longitude_step :

  0.1

  sw_point_latitude :

  -89.95

  sw_point_longitude :

  -179.95

  spatialResolution :

  11.131949 km

  geospatial_lon_resolution :

  11.131949 km

  geospatial_lat_resolution :

  11.131949 km

  geospatial_lat_units :

  degrees_north

  geospatial_lon_units :

  degrees_east

  number_of_lines :

  1800

  number_of_columns :

  3600

  measure :

  Mean

  suggested_image_scaling_minimum :

  10.0

  suggested_image_scaling_maximum :

  1000.0

  suggested_image_scaling_type :

  LOG

  suggested_image_scaling_applied :

  No

  _lastModified :

  2024-07-04T20:03:09.000Z

  Conventions :

  CF-1.6 ACDD-1.3

  institution :

  NASA Goddard Space Flight Center, Ocean Ecology Laboratory, Ocean Biology Processing Group

  standard_name_vocabulary :

  CF Standard Name Table v36

  naming_authority :

  gov.nasa.gsfc.sci.oceandata

  id :

  2.0/L3/PACE_OCI.20240701.L3b.DAY.POC.V2_0.NRT.nc

  license :

  https://science.nasa.gov/earth-science/earth-science-data/data-information-policy/

  creator_name :

  NASA/GSFC/OBPG

  publisher_name :

  NASA/GSFC/OBPG

  creator_email :

  data@oceancolor.gsfc.nasa.gov

  publisher_email :

  data@oceancolor.gsfc.nasa.gov

  creator_url :

  https://oceandata.sci.gsfc.nasa.gov

  publisher_url :

  https://oceandata.sci.gsfc.nasa.gov

  processing_level :

  L3 Mapped

  cdm_data_type :

  grid

  identifier_product_doi_authority :

  http://dx.doi.org

  identifier_product_doi :

  10.5067/PACE/OCI/L3M/POC/2.0

  keywords :

  Earth Science > Oceans > Ocean Chemistry > Organic Carbon

  keywords_vocabulary :

  NASA Global Change Master Directory (GCMD) Science Keywords

  data_bins :

  745741

  data_minimum :

  18.199707

  data_maximum :

  1989.1997

poc = np.array(dataset["poc"])

print(poc.shape)

(1800, 3600)

1. Statistical Operations

# Mean
mean_value = np.nanmean(poc)
print("Mean value:", mean_value)

Mean value: 79.23537

# Median
median_value = np.nanmedian(poc)
print("Median value:", median_value)

Median value: 49.600098

# Standard deviation
std_value = np.nanstd(poc)
print("Standard deviation:", std_value)

Standard deviation: 105.90423

2. Boolean Indexing and Masking

# Create a boolean mask
mask = poc > 100

filtered_poc = poc[mask]
print("Filtered poc (concentration > 100):", filtered_poc)

Filtered poc (concentration > 100): [184.7998 184.7998 184.7998 ... 325.6001 289.1997 262.1997]

3. Regriding

poc_10lat = np.reshape(poc, (180, 10, 3600))

poc_latregrid = np.nanmean(poc_10lat, axis=1)

/tmp/ipykernel_147/4091360230.py:1: RuntimeWarning: Mean of empty slice
  poc_latregrid = np.nanmean(poc_10lat, axis=1)

poc_latregrid_10lon = np.reshape(poc_latregrid, (180, 360, 10))

poc_latregrid_lonregrid = np.nanmean(poc_latregrid_10lon, axis=2)

/tmp/ipykernel_147/1374962890.py:1: RuntimeWarning: Mean of empty slice
  poc_latregrid_lonregrid = np.nanmean(poc_latregrid_10lon, axis=2)

print(poc_latregrid_lonregrid.shape)

(180, 360)

4. Plotting Array Data

plt.imshow(poc_latregrid_lonregrid, cmap='viridis')
plt.colorbar()
plt.title("POC Data Visualization")
plt.show()

dataset = xr.open_dataset(paths[1])
dataset

<xarray.Dataset> Size: 26MB
Dimensions:  (lat: 1800, lon: 3600, rgb: 3, eightbitcolor: 256)
Coordinates:
  * lat      (lat) float32 7kB 89.95 89.85 89.75 89.65 ... -89.75 -89.85 -89.95
  * lon      (lon) float32 14kB -179.9 -179.9 -179.8 ... 179.8 179.9 180.0
Dimensions without coordinates: rgb, eightbitcolor
Data variables:
    poc      (lat, lon) float32 26MB ...
    palette  (rgb, eightbitcolor) uint8 768B ...
Attributes: (12/64)
    product_name:                      PACE_OCI.20240702.L3m.DAY.POC.V2_0.poc...
    instrument:                        OCI
    title:                             OCI Level-3 Standard Mapped Image
    project:                           Ocean Biology Processing Group (NASA/G...
    platform:                          PACE
    source:                            satellite observations from OCI-PACE
    ...                                ...
    identifier_product_doi:            10.5067/PACE/OCI/L3M/POC/2.0
    keywords:                          Earth Science > Oceans > Ocean Chemist...
    keywords_vocabulary:               NASA Global Change Master Directory (G...
    data_bins:                         684721
    data_minimum:                      17.799805
    data_maximum:                      1961.0

xarray.Dataset

• Dimensions:
  • lat: 1800
  • lon: 3600
  • rgb: 3
  • eightbitcolor: 256
• Coordinates: (2)
  • lat
    (lat)
    float32
    89.95 89.85 89.75 ... -89.85 -89.95

    long_name :

    Latitude

    units :

    degrees_north

    standard_name :

    latitude

    valid_min :

    -90.0

    valid_max :

    90.0

    array([ 89.95    ,  89.85    ,  89.75    , ..., -89.75    , -89.850006,
           -89.950005], dtype=float32)

  • lon
    (lon)
    float32
    -179.9 -179.9 ... 179.9 180.0

    long_name :

    Longitude

    units :

    degrees_east

    standard_name :

    longitude

    valid_min :

    -180.0

    valid_max :

    180.0

    array([-179.95   , -179.85   , -179.75   , ...,  179.75   ,  179.85   ,
            179.95001], dtype=float32)

• Data variables: (2)
  • poc
    (lat, lon)
    float32
    ...

    long_name :

    Particulate Organic Carbon, D. Stramski, 2022 (hybrid version)

    units :

    mg m^-3

    valid_min :

    -32000

    valid_max :

    -22000

    reference :

    Stramski, D., et al. "Ocean color algorithms to estimate the concentration of particulate organic carbon in surface waters of the global ocean in support of a long-term data record from multiple satellite missions." Remote Sensing of Environment 269 (2022)

    display_scale :

    log

    display_min :

    10.0

    display_max :

    1000.0

    [6480000 values with dtype=float32]

  • palette
    (rgb, eightbitcolor)
    uint8
    ...

    [768 values with dtype=uint8]

• Indexes: (2)
  • lat
    PandasIndex

    PandasIndex(Index([ 89.94999694824219,   89.8499984741211,              89.75,
             89.6500015258789,  89.55000305175781,  89.44999694824219,
             89.3499984741211,              89.25,   89.1500015258789,
            89.05000305175781,
           ...
           -89.05000305175781,  -89.1500015258789,             -89.25,
           -89.35000610351562, -89.45000457763672, -89.55000305175781,
            -89.6500015258789,             -89.75, -89.85000610351562,
           -89.95000457763672],
          dtype='float32', name='lat', length=1800))

  • lon
    PandasIndex

    PandasIndex(Index([ -179.9499969482422, -179.85000610351562,             -179.75,
           -179.64999389648438,  -179.5500030517578,  -179.4499969482422,
           -179.35000610351562,             -179.25, -179.14999389648438,
            -179.0500030517578,
           ...
             179.0500030517578,  179.15000915527344,              179.25,
            179.35000610351562,  179.45001220703125,   179.5500030517578,
            179.65000915527344,              179.75,  179.85000610351562,
            179.95001220703125],
          dtype='float32', name='lon', length=3600))

• Attributes: (64)

  product_name :

  PACE_OCI.20240702.L3m.DAY.POC.V2_0.poc.0p1deg.NRT.nc

  instrument :

  OCI

  title :

  OCI Level-3 Standard Mapped Image

  project :

  Ocean Biology Processing Group (NASA/GSFC/OBPG)

  platform :

  PACE

  source :

  satellite observations from OCI-PACE

  temporal_range :

  day

  processing_version :

  2.0

  date_created :

  2024-07-04T20:03:29.000Z

  history :

  l3mapgen par=PACE_OCI.20240702.L3m.DAY.POC.V2_0.poc.0p1deg.NRT.nc.param

  l2_flag_names :

  ATMFAIL,LAND,HILT,HISATZEN,STRAYLIGHT,CLDICE,COCCOLITH,LOWLW,CHLWARN,CHLFAIL,NAVWARN,MAXAERITER,ATMWARN,HISOLZEN,NAVFAIL,FILTER,HIGLINT

  time_coverage_start :

  2024-07-02T00:19:27.000Z

  time_coverage_end :

  2024-07-03T01:09:06.000Z

  start_orbit_number :

  0

  end_orbit_number :

  0

  map_projection :

  Equidistant Cylindrical

  latitude_units :

  degrees_north

  longitude_units :

  degrees_east

  northernmost_latitude :

  90.0

  southernmost_latitude :

  -90.0

  westernmost_longitude :

  -180.0

  easternmost_longitude :

  180.0

  geospatial_lat_max :

  90.0

  geospatial_lat_min :

  -90.0

  geospatial_lon_max :

  180.0

  geospatial_lon_min :

  -180.0

  latitude_step :

  0.1

  longitude_step :

  0.1

  sw_point_latitude :

  -89.95

  sw_point_longitude :

  -179.95

  spatialResolution :

  11.131949 km

  geospatial_lon_resolution :

  11.131949 km

  geospatial_lat_resolution :

  11.131949 km

  geospatial_lat_units :

  degrees_north

  geospatial_lon_units :

  degrees_east

  number_of_lines :

  1800

  number_of_columns :

  3600

  measure :

  Mean

  suggested_image_scaling_minimum :

  10.0

  suggested_image_scaling_maximum :

  1000.0

  suggested_image_scaling_type :

  LOG

  suggested_image_scaling_applied :

  No

  _lastModified :

  2024-07-04T20:03:29.000Z

  Conventions :

  CF-1.6 ACDD-1.3

  institution :

  NASA Goddard Space Flight Center, Ocean Ecology Laboratory, Ocean Biology Processing Group

  standard_name_vocabulary :

  CF Standard Name Table v36

  naming_authority :

  gov.nasa.gsfc.sci.oceandata

  id :

  2.0/L3/PACE_OCI.20240702.L3b.DAY.POC.V2_0.NRT.nc

  license :

  https://science.nasa.gov/earth-science/earth-science-data/data-information-policy/

  creator_name :

  NASA/GSFC/OBPG

  publisher_name :

  NASA/GSFC/OBPG

  creator_email :

  data@oceancolor.gsfc.nasa.gov

  publisher_email :

  data@oceancolor.gsfc.nasa.gov

  creator_url :

  https://oceandata.sci.gsfc.nasa.gov

  publisher_url :

  https://oceandata.sci.gsfc.nasa.gov

  processing_level :

  L3 Mapped

  cdm_data_type :

  grid

  identifier_product_doi_authority :

  http://dx.doi.org

  identifier_product_doi :

  10.5067/PACE/OCI/L3M/POC/2.0

  keywords :

  Earth Science > Oceans > Ocean Chemistry > Organic Carbon

  keywords_vocabulary :

  NASA Global Change Master Directory (GCMD) Science Keywords

  data_bins :

  684721

  data_minimum :

  17.799805

  data_maximum :

  1961.0

pocs=[]

for i in range(31):
    dataset = xr.open_dataset(paths[i])
    poc_reshaped = np.array(dataset["poc"]).reshape((180,10,360,10))
    poc_10lat = np.nanmean(poc_reshaped, axis=1)
    poc_10lat_10lon = np.nanmean(poc_10lat, axis=-1)
    pocs.append(poc_10lat_10lon)

poc_7_daily = np.stack(pocs) 

print(poc_7_daily.shape)

(31, 180, 360)

/tmp/ipykernel_147/2594940156.py:6: RuntimeWarning: Mean of empty slice
  poc_10lat = np.nanmean(poc_reshaped, axis=1)
/tmp/ipykernel_147/2594940156.py:7: RuntimeWarning: Mean of empty slice
  poc_10lat_10lon = np.nanmean(poc_10lat, axis=-1)

poc_7 = np.nanmean(poc_7_daily, axis=0)

/tmp/ipykernel_147/3791947967.py:1: RuntimeWarning: Mean of empty slice
  poc_7 = np.nanmean(poc_7_daily, axis=0)

poc_7_log10 = np.log10(poc_7)

plt.imshow(poc_7_log10, cmap='viridis')
plt.colorbar()
plt.title("POC daily average in July Data Visualization")
plt.show()