Skip to content

Introduction

A short video explaining the logic of PyPSA-Eur which is similar to PyPSA-Earth:

For more details on PyPSA-Earth read the below milestone paper. For citations, please use the following BibTeX:

@misc{PyPSAEarth,
  author = {Parzen, Maximilian and Abdel-Khalek, Hazem and Fedorova, Ekaterina and Mahmood, Matin and Frysztacki, Martha Maria and Hampp, Johannes and Franken, Lukas and Schumm, Leon and Neumann, Fabian and Poli, Davide and Kiprakis, Aristides and Fioriti, Davide},
  title = {PyPSA-Earth. A new global open energy system optimization model demonstrated in Africa},
  publisher = {Applied Energy},
  year = {2023},
  url = {https://www.sciencedirect.com/science/article/pii/S030626192300460},
  doi = {https://doi.org/10.1016/j.apenergy.2023.121096},
}

Workflow

The generation of the model is controlled by the workflow management system Snakemake. In a nutshell, the Snakefile declares for each python script in the scripts directory a rule which describes which files the scripts consume and produce (their corresponding input and output files). The snakemake tool then runs the scripts in the correct order according to the rules' input/output dependencies. Moreover, it is able to track what parts of the workflow have to be regenerated when a data file or a script is modified/updated. For example, by executing the following snakemake routine:

.../pypsa-earth % snakemake -j 1 networks/elec_s_128.nc

the following workflow is automatically executed.

Workflow diagram

The blocks represent the individual rules which are required to create the file networks/elec_s_128.nc. Each rule requires scripts (e.g. Python) to convert inputs to outputs. The arrows indicate the outputs from preceding rules which a particular rule takes as input data.

Reproducibility

For reproducibility purposes, the image can be obtained through: 

snakemake --dag networks/elec_s_128.nc | dot -Tpng -o workflow.png
using Graphviz

Data Management System

PyPSA-Earth has a comprehensive set of pre-compiled datasets which provides all the inputs needed to build and run an energy system model for any country of the world. A specification of modelling-ready databundles is available in configs/bundle_config.yaml.

The datasets are available for various spatial scales, from global to national ones and can be extracted at an initial stage of running the model as described in Installation section. A data management system retrieve_databundle_light selects the databundles which matches best settings defined in the configuration file and extracts them automatically.

Folder structure

The content in this package is organized in folders as described below; for more details, please see the documentation.

  • data: Includes input data that is not produced by any snakemake rule.
  • scripts: Includes all the Python scripts executed by the snakemake rules.
  • resources: Stores intermediate results of the workflow which can be picked up again by subsequent rules.
  • networks: Stores intermediate, unsolved stages of the PyPSA network that describes the energy system model.
  • results: Stores the solved PyPSA network data, summary files and plots.
  • benchmarks: Stores snakemake benchmarks.
  • logs: Stores log files about solving, including the solver output, console output and the output of a memory logger.
  • envs: Stores the conda environment files to successfully run the workflow.

License

PyPSA-Earth work is released under multiple licenses:

  • All original source code is licensed as free software under AGPL-3.0 License.
  • The documentation is licensed under CC-BY-4.0.
  • Configuration files are mostly licensed under CC0-1.0.
  • Data files are licensed under different licenses as noted below.

Individual files contain license information in the header or in the dep5. Additional licenses and urls of the data used in PyPSA-Earth:

Files Licence BY NC SA Mark Changes Detail
data/raw/copernicus/* Custom x nan nan x https://cds.climate.copernicus.eu/cdsapp/#!/terms/licence-to-use-copernicus-products
data/raw/eez/* CC4.0 x x x nan http://www.marineregions.org/disclaimer.php
data/raw/landcover/* Custom x x nan nan https://www.protectedplanet.net/en/legal
data/raw/gadm/* CC4.0 x x nan x https://gadm.org/data.html
data/raw/gebco/* nan x nan nan nan https://www.gebco.net/data_and_products/gridded_bathymetry_data/arctic_ocean/#terms
data/hydro_capacities.csv nan x nan nan nan nan
data/raw/hydrobasins/* Custom x x nan nan https://www.hydrosheds.org/page/license
data/raw/WorldPop/* CC x nan nan nan https://www.worldpop.org/
data/raw/GDP/* CC1.0 nan nan nan nan https://datadryad.org/stash/dataset/doi:10.5061/dryad.dk1j0
data/osm/* ODbL nan nan nan nan https://www.openstreetmap.org/copyright
data/demand/unsd/* Custom x nan nan nan https://unstats.un.org/unsd/energystats/data/
data/industrial_database.csv CC x nan nan nan https://globalenergymonitor.org/projects/global-steel-plant-tracker/download-data/
data/industrial_database.csv CC4.0 x nan nan nan https://www.cgfi.ac.uk/spatial-finance-initiative/geoasset-project/cement/
data/industrial_database.csv CC4.0 x nan nan nan https://www.cgfi.ac.uk/spatial-finance-initiative/geoasset-project/pulp-and-paper-mill-database-for-latin-america/
data/airports.csv Public Domain nan nan nan nan https://ourairports.com/data/

Legend:

  • BY: Attribute Source
  • NC: Non-Commercial Use Only
  • SA: Share Alike