Skip to content

Instantly share code, notes, and snippets.

View rabernat's full-sized avatar

Ryan Abernathey rabernat

View GitHub Profile
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rabernat
rabernat / analyze_s3_listing.ipynb
Created December 8, 2022 00:08
S3 Listing Benchmarks
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rabernat
rabernat / beam_xarray_zarr.ipynb
Created May 24, 2022 03:30
WIP Xarray to Zarr pipeline with Beam
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAACXBIWXMAAHsIAAB7CAF4JB2hAAABWWlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNS40LjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczp0aWZmPSJodHRwOi8vbnMuYWRvYmUuY29tL3RpZmYvMS4wLyI+CiAgICAgICAgIDx0aWZmOk9yaWVudGF0aW9uPjE8L3RpZmY6T3JpZW50YXRpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgpMwidZAAAI4ElEQVRYCZ1XbYxU1Rl+zrnn3juzMzsLu6uUBSskFCpogdJQMWhWi1Foa1PoSqyJrT9U0i9aE+uP+mNTU5L+aBH6kQhWbEraCLUE+wN+WLsltcbgRqUClWJTZaECC8vHsjNz7z3n9HnvzCiwWDc92bvnnns+nvfjed/3jMKHtn4N7FfAditLbr5j5dTR81jmEizNUj8X2k/VsbraK3ib+JPO45iK9UEd+z3F6erFwa1/+E/j6L4A/fM8+vvdlaAIML719vaagYGBTGY+ddOXr/c19W2X+ZXKh92AgoOH8x42kzd+iQJ4TUnyx8MH2QkVqx1tXcGGN3c8ezBH4JlonpmPm//GCdAC7+39emH43PnHfYrvaSJYb+G
@rabernat
rabernat / NASA_earthdata_OPeNDAP_xarray.py
Created December 2, 2021 17:48
Load NASA EarthData OPeNDAP via Xarray
# first make sure lxml is installed
# pip install lxml
# browse to a Hyrax opendap link, e.g.
# https://acdisc.gesdisc.eosdis.nasa.gov/opendap/hyrax/CMS/GEOS_CASAGFED_D_FIRE.3/GEOSCarb_CASAGFED3v3_Fire.Daily.x720_y360.2003.nc.html
import xarray as xr
from pydap.client import open_url
from pydap.cas.urs import setup_session
@rabernat
rabernat / mds_zarr_adaptor.py
Created October 29, 2021 18:50
Expose mitgcm mds files as zarr arrays
from collections.abc import Mapping
import json
from typing import Tuple, List, Optional, Any
from pydantic import validator
from pydantic.dataclasses import dataclass
from fsspec.implementations.reference import ReferenceFileSystem
import numpy as np
import zarr
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.