Geodata¶

File formats¶

PMTiles¶

PMTiles is a general format for tile data addressed by Z/X/Y coordinates. This can be cartographic vector tiles, remote sensing data, JPEG images or similar.

HTTP Range Requests are used for reading in order to retrieve only the relevant tiles or metadata within a PMTiles archive. The arrangement of tiles and directories is designed to minimise the number of requests when moving and zooming.

However, PMTiles is a read-only format: it is not possible to update part of the archive without rewriting the entire file. If you need transactional updates, you should use a database such as SQLite or PostGIS and ST_asMVT.

See also

Mapbox Vector Tiles (MVT)¶

The Mapbox Vector Tiles file format stores each tile in a directory tree like /Z/X/Y.mvt. This works well for small tile sets, but updating an entire global pyramid of ~300 million tiles is very inefficient. PMTiles, on the other hand, is a single file with tiles de-duplicated, reducing the size of global vector basemaps by ~70%.

For writing, the Geospatial Data Abstraction Library (GDAL) library with SQLite and GEOS support must be installed. The MBTiles are stored in SQLite like mbtiles and can be processed with the MBTiles driver.

See also

MBTiles¶

MBTiles is a container format for tile data based on SQLite. It is optimised for local access, not for access via HTTP like PMTiles.

See also

MBTiles specification

Cloud Optimized GeoTIFF (COG)¶

Cloud Optimized GeoTIFF is a raster TIFF file that, like PMTiles, is optimised for reading from a cloud storage. PMTiles can also deliver other tile data, for example vector tiles. However, COG is backwards compatible with most GIS programmes that work with GeoTIFF.

See also

OGC Cloud Optimized GeoTIFF Standard

GeoParquet¶

Parquet is an open-source, column-orientated data file format that was developed for the efficient storage and retrieval of data. It offers efficient data compression and encoding methods with optimised processing of large, complex data. GeoParquet extends Parquet with interoperable geodata types (point, line, polygon).

GeoPandas supports the reading and writing of GeoParquet.
GeoParquet Downloader Plugin for QGIS enables streaming downloads of large GeoParquet datasets.
DuckDB allows the reading and writing of GeoParquet files with the Spatial Extension.

See also

Data repositories¶

See also

Geosciences and Environmental Sciences

Software¶

Reading and writing¶

Geospatial Data Abstraction Library (GDAL): provides a low-level but more powerful API for reading and writing hundreds of data formats.
pyogrio: provides a GeoPandas-oriented API to OGR vector data sources, such as ESRI Shapefile, GeoPackage, and GeoJSON.
Rasterio: reads and writes GeoTIFF and other forms of raster datasets.
Zarr-Python: Zarr is an open-source file storage format for chunked, compressed, N-dimensional arrays.
Fiona: reads and writes *.shp- and *.json data and many other formats.
netCDF4: is a Python interface to the netCDF C library.
pyModis: Collection of Python scripts for downloading and mosaicking MODIS data.
xmitgcm: reads MITgcm binary MDS files into xarray data structures.

See also

Geo wrappers

Remote sensing¶

Satpy: Easy to use API for sensors of satellite images like MODIS, Sentinel-2 etc.
sentinelsat: Find and download Copernicus Sentinel satellite imagery using command line or Python.
Open Data Cube: Open Source Geospatial Data Management and Analysis Software.
RSGISLib: or The Remote Sensing and GIS Software Library is a set of remote sensing tools for raster processing and analysis.

General purposes¶

pyproj: Python interface to PROJ, a library for cartographic projections and coordinate transformations.

geopython

is a GitHub organization that covers Python projects related to geodata.

pygeoapi: is a Python server implementation of the OGC-API standard suite.
OWSLib: is a Python package for client programming using the Open Geospatial Consortium (OGC) Web Service interface standards and associated content models.
pycsw: is an OGC CSW server implementation written in Python that fully implements the OpenGIS Catalogue Service.
pygeometa: can be used to generate metadata for spatial data sets.

pgeocode

Querying of GPS coordinates and municipality names from postal codes, distances between postal codes as well as general distances.

https://raster.shields.io/github/contributors/symerio/pgeocode

https://raster.shields.io/github/commit-activity/y/symerio/pgeocode

https://raster.shields.io/github/license/symerio/pgeocode

Arcpy

is used by Esri ArcGIS to perform geographic data analysis, data conversion, data management, and map automation.

GIS¶

QGIS: supports viewing, editing, printing, and analysis of geospatial data in a range of data formats.
GeoPandas: extends the datatypes used by pandas to allow spatial operations on geometric types.
regionmask: determines which geographic region each grid point belongs to.
Salem: extends xarray to add geolocalised subsetting, masking, and plotting operations.

Spatiotemporal statistics¶

rasterstats: Summarizing geospatial raster datasets based on vector geometries.
eofs: EOF analysis of spatial-temporal data.

Re-gridding¶

Pyresample: Resampling geospatial image data, primary for resampling in the Satpy library.
xESMF: Universal Regridder for Geospatial Data.

Simulation¶

xarray-simlab: provides both a generic framework for building computational models and a xarray extension for setting and running simulations.
Fastscape: provides a lot a small model components to use with the xarray-simlab modeling framework.
EarthSim: Tools for environmental simulation.

Visualisation¶

PyViz Tutorial

German-language tutorial that provides an overview of the Python visualisation libraries.

Cartopy: creates maps based on Matplotlib and converts points, lines and vectors between the different projections.
GeoPandas: GeoPandas examples.
Iris: implements a data model based on CF conventions, with visualisation based on Matplotlib and Cartopy.
GeoViews: Explore and visualise geographical, meteorological and oceanographic data sets.
ipyleaflet: is a Jupyter widget for Leaflet.js.
xarray-leaflet: is an xarray extension for plotting tiled maps.

Meteorology¶

MetPy: A collection of tools in Python for reading, visualizing, and performing calculations with weather data.
wrf-python: A collection of diagnostic and interpolation routines for use with output from the WRF-ARW Model.
windspharm: Computations on global wind fields in spherical geometry.

Oceanography¶

GSW-Python: Python implementation of the TEOS-10.
PyCO2SYS: Toolbox for solving the marine carbonate system and calculating related seawater properties.
pyoos: High level data collection library for met/ocean data publicly available.
UMWM: UMWM is a spectral ocean wave model.

Climate¶

PyOWM: A Python wrapper around OpenWeatherMap web APIs.
climpred: Verification of weather and climate forecasts.
xgcm: General Circulation Model Postprocessing with xarray.
climlab: Process-oriented climate modeling.
aospy: Computations that use gridded climate and weather data (namely netCDF files) and the management of the results.
OpenClimateGIS: Geoprocessing and computation on CF-compliant climate datasets.
oocgcm: Tools for processing and analysing output of general circulation models and gridded satellite data.
pangaea: Xarray extension for gridded land surface and weather model output.