Ctrl+K

Release Notes

Release Notes#

Upcoming release#

Please add descriptive release notes like in PyPSA-Eur. E.g. if a new rule becomes available describe how to use it snakemake -j1 run_tests and in one sentence what it does.

New Features and Major Changes

  • Improve Monte Carlo feature with more distributions types, independent by PyPSA component. PR #930

  • Introduce flexible regional selection of the demand files of GEGIS. PR #991

  • Generalize line types for AC and DC networks. PR #999

  • Add an option to merge isolated networks into respective backbone networks by countries. PR #903

Minor Changes and bug-fixing

  • Minor bug-fixing to run the cluster wildcard min PR #1019

  • Add option to adjust load scale for each individual countries PR #1006

  • Minor bug-fixing to get the generalised line types work for DC lines and AC lines. PR #1008 , PR #1011 and PR #1013

  • Minor bug-fixing for GADM_ID format naming. PR #980, PR #986 and PR #989

  • Fix download_osm_data compatibility for earth-osm v2.1. PR #954 and PR #988

  • Improve geometry filtering in clean_osm_data. PR #989

  • Revise bus region definition by gadm. PR #1001

  • Documentation improvements. PR ##1007

  • Remove unnecessary imports. PR ##1020

PyPSA-Earth 0.3.0#

New Features and major Changes (24th December 2023)

  • Keep all traceback in logs. PR #898

  • Function added in clean_osm_data script to allow the use of custom network data instead or on-top of OSM data. PR #842

  • Improve retrieve_databundle to prioritize smallest databundles PR #911

  • Add functionality to load shapefiles for hydrobasins directly from the data source directly PR #919

  • Use new CC0 v1 dataset for the natura input and automate download of WDPA protected planet data PR #913

Minor Changes and bug-fixing

  • Revise databundles and improve logging in retrieve_databundle PR #928

  • Improve documentation on installation and short tutorial PR #918

PyPSA-Earth 0.2.3#

New Features and major Changes (19th October 2023)

  • Add params: section in rule definition to keep track of changed settings in config.yaml. PR #823 and PR #880

  • Fix Natural Gas implementation in “add_electricity” to avoid “Natural Gas” to be filtered out PR #797

  • Improve network simplification routine to account for representation HVDC as Line component PR #743

  • Remove deprecated pypsa.networkclustering approach and replace by pypsa.clustering.spatial PR #786

  • Drop code-dependency from vresutil PR #803

  • Add a check to ensure match between a cutout and a modelled area PR #805

  • Support renewables or renewable expansion to meet a desired share of total load. PR #793

  • Add NorthAmerican and Earth cutouts, and improve African cutout PR #813

  • Bug fixing to restore Africa execution and improve performances PR #817

  • Add Asian cutout PR #826

  • Add a cutout for Western Asia PR #837

  • Add osm_config yaml file PR #822

  • Re-enable offshore wind and revise hydro PR #830

  • Add databundle of cutouts for Kazakhstan for CI test PR #856. The bundle (~5MB) is used in pypsa-kz-data repository during CI tests.

  • Option to specify a global upper capacity limit (using existing BAU functionality) PR #857

  • Add cluster options all, min and flex PR #848

  • Add commit id of pypsa earth in the n.meta of the .nc file per default PR #863

PyPSA-Earth 0.2.2#

New Features and major Changes (8th July 2023)

  • Fix Natural Gas assignment bug in build_powerplants rule PR #754.

  • Add GEM datasets to the powerplantmatching config PR #750.

  • Add merge and replace functionalities when adding custom powerplants PR #739. “Merge” combined the powerplantmatching data with new custom data. “Replace” allows to use fully self-collected data.

  • Add functionality of attaching existing renewable caapcities from custom_powerplants.csv. PR #744. If custom_powerplants are enabled and custom_powerplants.csv contains wind or solar powerplants, then p_nom and p_nom_min for renewables are extracted from custom_powerplants.csv, aggregated for each bus, and set.

  • Fix dask parallel computations for e.g. cutouts calculations. Now again more than 1 core will be used when available that can lead to ~8x speed ups with 8 cores PR #734 and PR #761.

  • Add the usage of custom rules. Custom rule files must be specified in the config as a list, e.g. custom rules: [“my_rules.smk”]. Empty by default (i.e. no custom rules). PR #755

  • Add trailing whitespace linter which removes unnecessary tabs when running pre-commit PR #762

  • Add codespell linter which corrects word spellings PR #763

  • Remove RES addition functionality from attach_conventional_generators PR #769. Currently wind and solar powerplants stored in powerplants.csv are added to the network by attach_conventional_generators.

  • Add functionalities to download and extract emission of countries. PR #748 https://github.com/pypsa-meets-earth/pypsa-earth/pull/748

PyPSA-Earth 0.2.1#

New Features and major Changes (20th May 2023)

  • Fix bug. Add graphviz to docs to compile workflows in the documentation and adapt release notes PR #719

  • License change from GPL to AGPL as voted here

  • Fix hard-coded simplification of lines to 380kV PR #732. It is now possible to simplify the network to any other voltage level with config option base_voltage.

  • Fix a KeyError in simplify_links caused by misinterpretation of AC lines as DC ones PR #740.

PyPSA-Earth 0.2.0#

New Features and major Changes (7th May 2023)

  • Finalize package restructuring PR #462

  • Fix made in config.default and config.tutorial changing Monte-Carlo from true to false PR #463

  • Add new config test design. It is now easy and light to test multiple configs PR #466

  • Revision of documentation PR #471

  • Move to new GADM version PR #478

  • Update natura tiff to global scale, revise default databundle description and remove old limitations to environment PR #470 and PR #500

  • Update docs on installation PR #498

  • Update docs on tutorial PR #507

  • Moved from pycountry to country_converter PR #493

  • Fix workflow in order to solve the landlock countries bug PR #481 and PR #517

  • Add meta data of config to pypsa network per default. Allows keeping track of the config used to generate the network PR #526

  • Fix renewable profiles generation for possible data loss in ERA5-derived cutouts PR #511

  • Adapt dependencies of powerplantmatching to the PyPSA main branch PR #527

  • Calculate the outputs of retrieve_databundle dynamically depending on settings PR #529

  • Fix shape bug in the Voronoi cell creation PR #541

  • Adapt dependencies on PyPSA to the PyPSA main branch PR #538

  • Fix None geometries into regions PR #546

  • Swap OpenStreetMap python download interface from esy-osm to earth-osm PR #547

  • Restore saving of logger outputs PR #559

  • Techno-economic parameters of technologies (e.g. costs and efficiencies) can be now retrieved from a separate repository PyPSA/technology-data that collects assumptions from a variety of sources. It is activated by default with enable: retrieve_cost_data: true and controlled with costs: year: and costs: version:. The location of this data changed from data/costs.csv to resources/costs.csv. Adapted from [#184].

  • Added approaches to process contended areas PR #572

  • Improve parallel capabilities of build_shapes to enable parallelization even within a country shape PR #575

  • Add pypsa-eur scenario management PR #577

  • Minor bug fixing and improvements PR #580

  • Streamline default configuration file PR #589

  • Fix rule run_test, remove code duplication, add gitstars to readme PR #593 <https://github.com/pypsa-meets-earth/pypsa-earth/pull/593>

  • Add new build_demand_profiles.py. It builds demand_profiles.csv and allow easier interfacing of new data PR #582

  • Upgrade technology data to v0.5.0 PR #600

  • Update simplify_network and cluster_network according to PyPSA-Eur developments PR #597

  • Revise OSM cleaning to improve the cleaning process and error resilience PR #620

  • Fix isolated buses when simplifying the network and add clustering by networks PR #632

  • Include hydro runoff normalization PR #631

  • Add REUSE compatibility PR #651

  • Fix bug of missing GitHub issue template PR #660

  • Fix GADM bug when using alternative clustering and store gadm shape with two letter instead of three letter ISO code PR #670

  • Fix GADM naming bug related to level-2 clustering PR #684

  • Fix append bug in build_powerplants rule PR #686

  • Add zenodo_handler.py to update and upload files via code PR #688

  • Fix a few typos in docstrings PR #695

  • Update and improve configuration section in documentation PR #694

  • Improve earth coverage and add improve make_statistics coverage PR #654

  • Fix bug for missing renewable profiles and generators PR #714

  • Update instructions on how to write documentation. PR #720

  • Enable workflow to run including countries with empty OSM data, test on all UN countries `PR #701 https://github.com/pypsa-meets-earth/pypsa-earth/pull/701`__

PyPSA-Earth 0.1.0#

Model rebranded from PyPSA-Africa to PyPSA-Earth. Model is part of the now called PyPSA meets Earth initiative which hosts multiple projects.

New features and major changes (10th September 2022)

  • Identify DC lines but temporary transform them back into AC PR #348

  • Get renewable capacities from IRENA statistics PR #343

  • Bug fixing (script retrieve_databundle) and rule run_test to ease testing PR #322

  • Handling non-numerical entries in raw OSM data: PR #287

  • General user experience improvements: PR #326

  • Fix minor validation notebook inaccuracy: PR #332

  • Make clean_osm_data script work with land-locked country: PR #341

  • Add demand validation notebook for 2030 prediction: PR #344

  • Revise build_powerplants with new version of powerplantmatching: PR #342

  • Fix typo causing the wrong coordinate reference systems (CRS) to be used when determining available land types using CLC PR #345

  • Add high resolution population raster via API: PR #325

  • Fix bounds of cutouts aka weather cells: PR #347

  • Add new countries and update iso code: PR #330

  • Fix solar pv slope and add correction factor for wake losses: PR #335

  • Add renewable potential notebook: PR #351

  • Make cutout workflow simpler: PR #352

  • Add option to run workflow without pop and gdp raster: PR #353

  • Add latitude_optimal to get optimal solar orientation by default: Commit 1b2466b

  • Harmonize CRSs by options: PR #356

  • Fix powerplantmatching problem for DRC and countries with multi-word name: PR #359

  • Change default option for build_natura: PR #360

  • Add renewable potential validation notebook and update others: PR #363 and PR #369

  • Constrain rasterio version and add plotting dependencies: PR #365

  • Change solar power density form 1.7 to 4.6 MW/km2: PR #364

  • Bug fixing of unexpected float value in build_powerplants: PR #372 and PR #373

  • Revise hydro capacities, add hydro validation notebook and minor revisions: PR #366

  • Revise dropnan for regions: PR #366

  • Fix bug in GADM clustering. Missing crs input: PR #379

  • Optimise availabilitymatrix speed by factor 4-5: PR #380

  • Fix bug in inline documentation for GADM and Voronoi clustering: PR #384

  • Fix simple clustering enabling the creation of networks such regions_onshore_elec_s54_14.nc: PR #386

  • Add transformer components which connect different voltage level lines: PR #389

  • Enable the use of a float value for the scale in load_options: PR #397

  • Add operational reserve margin according to PyPSA-Eur: PR #399

  • Add optional normalization of hydro inflows by hydro_capacities or eia stats: PR #376

  • Enable DC carrier in the network model and include converters into the model: PR #392

  • Implement PyPSA-Eur improvements. Add gas limit constraints, add marginal cost sweeps wildcard, add and harmonize aggregation strategies, improve config usability by carrier clarifications, ease debugging by removing snakemake inputs from functions: PR #402

  • Fix and add docs. Fix incomplete tutorial, recommend mamba for installation, add YouTube videos PR #412 and PR #423

  • Restructure the package to ease readability and fix google drive downloading method: PR #355

  • Update config links to adhere to the new structure of the package: PR #420

  • Improve and finalize capacity_validation notebook: PR #406 and PR #455

  • Fix hydro technology with the GADM clustering approach: PR #428

  • Adapt for a custom shapefile for MA as a first step towards generalizing the feature: PR #429

  • Improve line augmentation for network expansion explorations. Use k-edge augmenation for AC lines and random sampling for long HVDC lines: PR #427

  • Fix minor bug in clustering about missing prefix assignment PR #434

  • Fix major aggregation bug and adjust config: PR #435

  • Fix nan techtype and wrong tech for nuclear which improves the representation of existing powerplants PR #436

  • Add notebook to compare results by different solvers PR #421

  • Fix overestimation of the network capacity by simplify network PR #443

  • Fix output electricity column in clean_data PR #441

  • Bug fixing to download global OSM and shape data: PR #433

PyPSA-Africa 0.0.2#

New features and major changes (6th April 2022)

  • Plotting and summary features: PR #211 and PR #214

  • Templates for issue, PR, feature request: PR #216

  • Attach hydro enabled with all hydro types: PR #232

  • Parallel download of osm data: PR #232

  • Decoupling iso coding from geofabrik; rule download_osm_data extended to the world: PR #236

  • Rule build_shape extended to the world: PR #236

  • Validation of geofabrik links: PR #249

  • Generalized version of Data retrieval with google and zenodo hosting platforms: PR #242 and PR #260

  • Fix random state for kmean clustering, adopted from PR 313

  • Implement area exclusions based on land type using the Copernicus Land Cover: PR #272.

  • Flexible demand extraction for multiple years across the globe: PR #275

  • Add CI caching and windows CI: Commit CI windows, PR #277.

  • Change config to allow weather year extraction from snapshots as default: PR #301.

  • Replace Restyler by .pre-commit `PR #307 https://github.com/pypsa-meets-earth/pypsa-earth/pull/307`__.

  • Solved the issue of “overpassing nodes” and restyling osm_build_network: PR #294

  • Revise deprecations in build_shape: PR #315

PyPSA-Africa 0.0.1#

This is the first release of PyPSA-Africa which heavily builds on PyPSA-Eur.

New features and major changes (24th December 2021)

  • Include new data streams for Africa model

  • Demand data implementation from GEGIS. Demand can be chosen for weather years and socioeconomic ssp scenarios

  • Network is built, cleaned and processed solely on OpenStreetMap data

  • Voronoi regions, where data is aggregated towards, can be replaced by administrative GADM zones

  • Augmented line expansion feature can make network meshed, connect isolated mini-grids to the main-grid.

  • Community moved to Discord.

  • Most meeting and agenda’s are open.

Release Process#

  • Checkout a new release branch git checkout -b release-v0.x.x.

  • Finalise release notes at doc/release_notes.rst.

  • Update envs/environment.fixed.yaml via conda env export -n pypsa-earth -f envs/environment.fixed.yaml --no-builds from an up-to-date pypsa-earth environment. Add license note at the top of the new yaml.

  • Update version number in doc/conf.py and *config.*.yaml.

  • Open, review and merge pull request for branch release-v0.x.x. Make sure to close issues and PRs or the release milestone with it (e.g. closes #X). Run pre-commit run --all locally and fix any issues.

  • Tag a release on Github via git tag v0.x.x, git push, git push --tags. Include release notes in the tag message.

  • Upload code to zenodo code repository with GPLv3 license.

  • Create pre-built networks for config.default.yaml by running snakemake -j 1 extra_components_all_networks.

  • Upload pre-built networks to zenodo data repository with CC BY 4.0 license.

  • Send announcement on the PyPSA-Earth Discord channel.

Contents