Skip to content

Network Building

Scripts for building, simplifying, and clustering the electricity network.

base_network

Creates the network topology from a OpenStreetMap.

Relevant Settings

.. code:: yaml

snapshots:

countries:

electricity:
    voltages:

lines:
    types:
    s_max_pu:
    under_construction:

links:
    p_max_pu:
    p_nom_max:
    under_construction:

transformers:
    x:
    s_nom:
    type:

.. seealso:: Documentation of the configuration file config.yaml at :ref:snapshots_cf, :ref:toplevel_cf, :ref:electricity_cf, :ref:load_options_cf, :ref:lines_cf, :ref:links_cf, :ref:transformers_cf

Inputs

Outputs

  • networks/base.nc

    .. image:: /img/base.png :width: 33 %

Description

add_electricity

Adds electrical generators, load and existing hydro storage units to a base network.

Relevant Settings

.. code:: yaml

costs:
    output_currency:

electricity:
    max_hours:
    marginal_cost:
    capital_cost:
    conventional_carriers:
    co2limit:
    extendable_carriers:
    include_renewable_capacities_from_OPSD:
    estimate_renewable_capacities_from_capacity_stats:

renewable:
    hydro:
        carriers:
        hydro_max_hours:
        hydro_max_hours_default:
        hydro_capital_cost:

lines:
    length_factor:

.. seealso:: Documentation of the configuration file config.yaml at :ref:costs_cf, :ref:electricity_cf, :ref:load_options_cf, :ref:renewable_cf, :ref:lines_cf

Inputs

  • resources/costs.csv: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.

  • data/bundle/hydro_capacities.csv: Hydropower plant store/discharge power capacities, energy storage capacity, and average hourly inflow by country. Not currently used!

    .. image:: /img/hydrocapacities.png

  • data/geth2015_hydro_capacities.csv: alternative to capacities above; not currently used!

  • resources/demand_profiles.csv: a csv file containing the demand profile associated with buses
  • resources/shapes/gadm_shapes.geojson: confer :ref:shapes
  • resources/powerplants.csv: confer :ref:powerplants
  • resources/profile_{}.nc: all technologies in config["renewables"].keys(), confer :ref:renewableprofiles
  • networks/base.nc: confer :ref:base

Outputs

  • networks/elec.nc:

    .. image:: /img/elec.png :width: 75 % :align: center

Description

The rule :mod:add_electricity ties all the different data inputs from the preceding rules together into a detailed PyPSA network that is stored in networks/elec.nc. It includes:

  • today's transmission topology and transfer capacities (in future, optionally including lines which are under construction according to the config settings lines: under_construction and links: under_construction),
  • today's thermal and hydro power generation capacities (for the technologies listed in the config setting electricity: conventional_carriers), and
  • today's load time-series (upsampled in a top-down approach according to population and gross domestic product)

It further adds extendable generators with zero capacity for

  • photovoltaic, onshore and AC- as well as DC-connected offshore wind installations with today's locational, hourly wind and solar capacity factors (but no current capacities),
  • additional open- and combined-cycle gas turbines (if OCGT and/or CCGT is listed in the config setting electricity: extendable_carriers)

load_powerplants(ppl_fn, costs=None, fill_values=None, grouping_years=None)

Load and preprocess powerplant matching data, fill missing datein/dateout, and assign grouping years. Parameters

ppl_fn : str Path to powerplant matching csv file. costs : pd.DataFrame DataFrame containing technology costs. fill_values : dict Dictionary containing default values for lifetime. grouping_years : list List of years to group build years into.

Returns

ppl : pd.DataFrame Power plant list DataFrame.

fill_datein_dateout(ppl, costs, fill_values)

Fill missing datein and dateout values in ppl DataFrame.

Parameters

ppl : pd.DataFrame Dataframe containing power plants. costs : pd.DataFrame DataFrame containing cost assumptions. fill_values : dict Dictionary containing default values for lifetime.

Returns

ppl : pd.DataFrame Power plant list DataFrame with filled missing datein and dateout columns.

attach_load(n, demand_profiles)

Add load profiles to network buses.

Parameters

n: pypsa network

demand_profiles: str Path to csv file of elecric demand time series, e.g. "resources/demand_profiles.csv" Demand profile has snapshots as rows and bus names as columns.

Returns

n : pypsa network Now attached with load time series

get_grouping_year(build_year, grouping_years)

Map build_year to the nearest grouping year bin.

Each build year is assigned to the first grouping year that is greater than or equal to it (i.e. the right edge of its bin).

Example: grouping_years = [1980, 2000, 2010, 2015, 2020] build_year = 2012 → returns 2015 build_year = 2018 → returns 2020

aggregate_ppl_by_bus_carrier_year(ppl)

Aggregate power plants by (bus, carrier, grouping_year).

Creates a new carrier name with grouping year suffix (e.g., "CCGT-2020") and aggregates capacity and other attributes.

Parameters

ppl : pd.DataFrame Power plant DataFrame with columns: bus, carrier, grouping_year, p_nom, efficiency, marginal_cost, datein, dateout and so on.

Returns

pd.DataFrame Aggregated power plants with columns: bus, carrier, carrier_gy, p_nom, efficiency, marginal_cost, build_year, lifetime.

Example

Input: bus carrier grouping_year p_nom bus1 CCGT 2015 100 bus1 CCGT 2015 200 bus1 CCGT 2020 150

Output: bus carrier carrier_gy p_nom bus1 CCGT CCGT-2015 300 bus1 CCGT CCGT-2020 150

aggregate_inflow_by_group(ppl, ppl_grouped, inflow_t)

Aggregate inflow time series by (bus, carrier, grouping_year) groups.

Parameters

ppl : pd.DataFrame Original (ungrouped) power plant DataFrame with columns: bus, carrier, grouping_year. ppl_grouped : pd.DataFrame Aggregated power plant DataFrame with columns: bus, carrier, carrier_gy. inflow_t : pd.DataFrame Inflow time series DataFrame with plant indices as columns.

Returns

pd.DataFrame Aggregated inflow time series with ppl_grouped indices as columns.

get_irena_targets_for_carrier(carrier, estimate_renewable_capacities_config, countries)

Return IRENA installed capacity targets for a given carrier as a Series indexed by country (MW).

The function reads IRENASTAT installed capacity data using the existing estimate_renewable_capacities configuration. Offshore wind is mapped entirely to offwind-ac.

attach_wind_and_solar(n, costs, ppl, input_files, carriers, extendable_carriers, line_length_factor)

Attach wind and solar generators.

Existing capacities are taken from powerplants.csv and spatialized to buses. National capacity gaps with respect to IRENA targets are filled and redistributed uniformly across buses within each country.

Offshore wind is treated entirely as offwind-ac.

Parameters

n : pypsa.Network The PyPSA network to modify. costs : pd.DataFrame DataFrame containing technology costs. ppl : pd.DataFrame Power plant DataFrame. input_files : dict Snakemake input object containing renewable profile files. carriers : set Set of renewable carriers to be added. extendable_carriers : dict Dictionary of extendable carriers for different component types. line_length_factor : float Factor to adjust line lengths for connection cost calculations.

Returns

None

attach_conventional_generators(n, costs, ppl, conventional_carriers, extendable_carriers, renewable_carriers, conventional_config, conventional_inputs)

Add existing conventional generators to the network and extendable conventional generators at all buses.

Parameters

n : pypsa.Network The PyPSA network to modify. costs : pd.DataFrame DataFrame containing technology costs. ppl : pd.DataFrame Power plant DataFrame. conventional_carriers : list List of conventional carriers to be added. extendable_carriers : dict Dictionary of extendable carriers for different component types. renewable_carriers : set Set of renewable carriers. conventional_config : list List of conventional configuration settings. conventional_inputs : dict Dictionary of conventional input parameters.

Returns

None

apply_nuclear_p_max_pu(n, nuclear_p_max_pu)

Apply country-level static nuclear p_max_pu limits based on historical Energy Availability Factor (IAEA, 2022–2024).

  • If country is in CSV: apply p_max_pu
  • If country is NOT in CSV: keep default p_max_pu = 1.0 and warn

attach_hydro(n, costs, ppl, hydro_min_inflow_pu=1.0)

Add existing hydro powerplants to the network as Hydro Storage units, Run-Of-River generators, and Pumped Hydro storage units.

Parameters

n : pypsa.Network The PyPSA network to modify. costs : pd.DataFrame DataFrame containing technology costs. ppl : pd.DataFrame Power plant DataFrame.

Returns

None

attach_existing_batteries(n, costs, ppl)

Add existing battery storage units from powerplants.csv to the network.

Parameters

n : pypsa.Network The PyPSA network to modify. costs : pd.DataFrame DataFrame containing technology costs. ppl : pd.DataFrame Power plant DataFrame.

Returns

None

attach_extendable_generators(n, costs, ppl)

Add extendable conventional generators (OCGT, CCGT, nuclear) with zero capacity.

Parameters

n : pypsa.Network The PyPSA network to modify. costs : pd.DataFrame DataFrame containing technology costs. ppl : pd.DataFrame Power plant DataFrame.

Returns

None

add_nice_carrier_names(n, config)

Add nice names and colors to carriers.

Parameters

n : pypsa.Network The PyPSA network to modify. config : dict The configuration dictionary containing plotting settings.

Returns

None

simplify_network

Lifts electrical transmission network to a single 380 kV voltage layer, removes dead-ends of the network, and reduces multi-hop HVDC connections to a single link.

Relevant Settings

.. code:: yaml

clustering:
    simplify:
    aggregation_strategies:

costs:
    output_currency:

lines:
    length_factor:

links:
    p_max_pu:

solving:
    solver:
        name:

.. seealso:: Documentation of the configuration file config.yaml at :ref:costs_cf, :ref:electricity_cf, :ref:renewable_cf, :ref:lines_cf, :ref:links_cf, :ref:solving_cf

Inputs

  • resources/costs.csv: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.
  • resources/regions_onshore.geojson: confer :ref:busregions
  • resources/regions_offshore.geojson: confer :ref:busregions
  • networks/elec.nc: confer :ref:electricity

Outputs

  • resources/regions_onshore_elec_s{simpl}.geojson:

    .. image:: /img/regions_onshore_elec_s.png :width: 33 %

  • resources/regions_offshore_elec_s{simpl}.geojson:

    .. image:: /img/regions_offshore_elec_s .png :width: 33 %

  • resources/busmap_elec_s{simpl}.csv: Mapping of buses from networks/elec.nc to networks/elec_s{simpl}.nc;

  • networks/elec_s{simpl}.nc:

    .. image:: /img/elec_s.png :width: 33 %

Description

The rule :mod:simplify_network does up to four things:

  1. Create an equivalent transmission network in which all voltage levels are mapped to the 380 kV level by the function simplify_network(...).

  2. DC only sub-networks that are connected at only two buses to the AC network are reduced to a single representative link in the function simplify_links(...). The components attached to buses in between are moved to the nearest endpoint. The grid connection cost of offshore wind generators are added to the capital costs of the generator.

  3. Stub lines and links, i.e. dead-ends of the network, are sequentially removed from the network in the function remove_stubs(...). Components are moved along.

  4. Optionally, if an integer were provided for the wildcard {simpl} (e.g. networks/elec_s500.nc), the network is clustered to this number of clusters with the routines from the cluster_network rule with the function cluster_network.cluster(...). This step is usually skipped!

simplify_network_to_base_voltage(n, linetype, base_voltage)

Fix all lines to a voltage level of base voltage level and remove all transformers.

The function preserves the transmission capacity for each line while updating its voltage level, line type and number of parallel bundles (num_parallel). Transformers are removed and connected components are moved from their starting bus to their ending bus. The corresponding starting buses are removed as well.

Simplifies multi-node DC link components into single links between end-points.

Parameters

n : pypsa.Network The PyPSA network to be simplified. costs : pd.DataFrame DataFrame containing technology costs. renewable_config : dict Configuration dictionary for renewable technologies. hvdc_as_lines : bool Flag indicating whether HVDC lines are treated as lines. config_lines : dict Configuration dictionary for lines. config_links : dict Configuration dictionary for links. output : object Output object containing file paths for saving results. exclude_carriers : list, optional List of carriers to exclude from simplification, by default []. aggregation_strategies : dict, optional Strategies for aggregating components, by default dict().

Returns

n : pypsa.Network The simplified PyPSA network.

drop_isolated_networks(n, threshold)

Find isolated subnetworks in the network and drop those of them which don't have load or have a load value lower than a threshold.

Parameters

iso_code : PyPSA.Network Original network threshold: float Load power used as a threshold to drop isolated nodes

Returns

modified network

merge_into_network(n, threshold, aggregation_strategies=dict())

Find isolated AC nodes and sub-networks in the network and merge those of them which have load value and a number of buses below than the specified thresholds into a backbone network.

Parameters

n : PyPSA.Network Original network threshold : float Load power used as a threshold to merge isolated nodes aggregation_strategies: dictionary Functions to be applied to calculate parameters of the aggregated grid

Returns

modified network

merge_isolated_networks(n, threshold, aggregation_strategies=dict())

Find isolated subnetworks in the network and merge those of them which have load value below than a specified threshold into a single isolated node which represents all the remote generation.

Parameters

n : PyPSA.Network Original network threshold : float Load power used as a threshold to merge isolated nodes aggregation_strategies: dictionary Functions to be applied to calculate parameters of the aggregated grid

Returns

modified network

cluster_network

Creates networks clustered to {cluster} number of zones with aggregated buses, generators and transmission corridors.

Relevant Settings

.. code:: yaml

clustering:
    aggregation_strategies:

focus_weights:

solving:
    solver:
        name:

lines:
    length_factor:

.. seealso:: Documentation of the configuration file config.yaml at :ref:toplevel_cf, :ref:renewable_cf, :ref:solving_cf, :ref:lines_cf

Inputs

  • resources/regions_onshore_elec_s{simpl}.geojson: confer :ref:simplify
  • resources/regions_offshore_elec_s{simpl}.geojson: confer :ref:simplify
  • resources/busmap_elec_s{simpl}.csv: confer :ref:simplify
  • networks/elec_s{simpl}.nc: confer :ref:simplify
  • data/custom_busmap_elec_s{simpl}_{clusters}.csv: optional input

Outputs

  • resources/regions_onshore_elec_s{simpl}_{clusters}.geojson:

    .. image:: /img/regions_onshore_elec_s_X.png :width: 33 %

  • resources/regions_offshore_elec_s{simpl}_{clusters}.geojson:

    .. image:: /img/regions_offshore_elec_s_X.png :width: 33 %

  • resources/busmap_elec_s{simpl}_{clusters}.csv: Mapping of buses from networks/elec_s{simpl}.nc to networks/elec_s{simpl}_{clusters}.nc;

  • resources/linemap_elec_s{simpl}_{clusters}.csv: Mapping of lines from networks/elec_s{simpl}.nc to networks/elec_s{simpl}_{clusters}.nc;

  • networks/elec_s{simpl}_{clusters}.nc:

    .. image:: /img/elec_s_X.png :width: 40 %

Description

.. note::

**Why is clustering used both in** ``simplify_network`` **and** ``cluster_network`` **?**

    Consider for example a network ``networks/elec_s100_50.nc`` in which
    ``simplify_network`` clusters the network to 100 buses and in a second
    step ``cluster_network``` reduces it down to 50 buses.

    In preliminary tests, it turns out, that the principal effect of
    changing spatial resolution is actually only partially due to the
    transmission network. It is more important to differentiate between
    wind generators with higher capacity factors from those with lower
    capacity factors, i.e. to have a higher spatial resolution in the
    renewable generation than in the number of buses.

    The two-step clustering allows to study this effect by looking at
    networks like ``networks/elec_s100_50m.nc``. Note the additional
    ``m`` in the ``{cluster}`` wildcard. So in the example network
    there are still up to 100 different wind generators.

    In combination these two features allow you to study the spatial
    resolution of the transmission network separately from the
    spatial resolution of renewable generators.

**Is it possible to run the model without the** ``simplify_network`` **rule?**

    No, the network clustering methods in the PyPSA module
    `pypsa.clustering.spatial <https://github.com/PyPSA/PyPSA/blob/master/pypsa/networkclustering.py>`_
    do not work reliably with multiple voltage levels and transformers.

.. tip:: The rule :mod:cluster_all_networks runs for all scenario s in the configuration file the rule :mod:cluster_network.

Exemplary unsolved network clustered to 512 nodes:

.. image:: /img/elec_s_512.png :width: 40 % :align: center

Exemplary unsolved network clustered to 256 nodes:

.. image:: /img/elec_s_256.png :width: 40 % :align: center

Exemplary unsolved network clustered to 128 nodes:

.. image:: /img/elec_s_128.png :width: 40 % :align: center

Exemplary unsolved network clustered to 37 nodes:

.. image:: /img/elec_s_37.png :width: 40 % :align: center

distribute_clusters(inputs, build_shape_options, country_list, distribution_cluster, n, n_clusters, focus_weights=None, solver_name=None)

Determine the number of clusters per country.

groupby_bus_carrier(network, aggregation_strategies, exclude_carriers=[])

Group generators and storage units by (bus, carrier).

Parameters

network : pypsa.Network The PyPSA network to modify in-place. aggregation_strategies : dict Aggregation strategies for different columns. exclude_carriers : list, optional List of carriers to exclude from grouping, by default [].

Returns

None

augmented_line_connections

Guarantees that every bus has at least X-number of connections.

Relevant Settings

.. code:: yaml

.. seealso::

Inputs

Outputs

Description

add_extra_components

Adds extra extendable components to the clustered and simplified network.

Relevant Settings

.. code:: yaml

sector:
    transmission_efficiency:

.. seealso:: Documentation of the configuration file config.yaml at :ref:sector_cf

Inputs

  • resources/costs.csv: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.

Outputs

  • networks/elec_s{simpl}_{clusters}_ec.nc:

Description

The rule :mod:add_extra_components attaches additional extendable components to the clustered and simplified network. These can be configured in the config.yaml at electricity: extendable_carriers:. It processes networks/elec_s{simpl}_{clusters}.nc to build networks/elec_s{simpl}_{clusters}_ec.nc, which in contrast to the former (depending on the configuration) contain with zero initial capacity

  • StorageUnits of carrier 'H2' and/or 'battery'. If this option is chosen, every bus is given an extendable StorageUnit of the corresponding carrier. The energy and power capacities are linked through a parameter that specifies the energy capacity as maximum hours at full dispatch power and is configured in electricity: max_hours:. This linkage leads to one investment variable per storage unit. The default max_hours lead to long-term hydrogen and short-term battery storage units.

  • Stores of carrier 'H2' and/or 'battery' in combination with Links. If this option is chosen, the script adds extra buses with corresponding carrier where energy Stores are attached and which are connected to the corresponding power buses via two links, one each for charging and discharging. This leads to three investment variables for the energy capacity, charging and discharging capacity of the storage unit.

prepare_network

Prepare PyPSA network for solving according to :ref:opts and :ref:ll, such as.

  • adding an annual limit of carbon-dioxide emissions,
  • adding an exogenous price per tonne emissions of carbon-dioxide (or other kinds),
  • setting an N-1 security margin factor for transmission line capacities,
  • specifying an expansion limit on the cost of transmission expansion,
  • specifying an expansion limit on the volume of transmission expansion, and
  • reducing the temporal resolution by averaging over multiple hours or segmenting time series into chunks of varying lengths using tsam.

Relevant Settings

.. code:: yaml

links:

lines:

costs:
    emission_prices:

electricity:
    co2limit:
    max_hours:

.. seealso:: Documentation of the configuration file config.yaml at :ref:costs_cf, :ref:electricity_cf

Inputs

  • resources/costs.csv.: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.
  • networks/elec_s{simpl}_{clusters}.nc: confer :ref:cluster

Outputs

  • networks/elec_s{simpl}_{clusters}_ec_l{ll}_{opts}.nc: Complete PyPSA network that will be handed to the solve_network rule.

Description

.. tip:: The rule :mod:prepare_all_networks runs for all scenario s in the configuration file the rule :mod:prepare_network.

download_emission_data()

Download emission file from EDGAR.

Returns

global emission file for all countries in the world.

emission_extractor(filename, emission_year, country_names)

Extracts CO2 emission values for given country codes from the global emission file.

Parameters

filename : str Global emission filename emission_year : int Year of CO2 emissions country_names : numpy.ndarray Two letter country codes of analysed countries.

Returns

CO2 emission values of studied countries.