osemosys2iamc

Use this Python package to convert OSeMOSYS results to the IAMC format

Acknowledgements

This work was financially supported by:

• The European Climate and Energy Modelling Forum (ECEMF) has received funding from the European Union’s Horizon 2020 Research and Innovation programme under the grant agreement No 101022622
• The IAM COMPACT project has received funding from the European Union’s HORIZON EUROPE Research and Innovation Programme under grant agreement No 101056306

Install from Python Packaging Index (PyPI)

pip install osemosys2iamc

Run the package

$ osemosys2iamc --help
osemosys2iamc <inputs_path> <results_path> <config_path> <output_path>

inputs_path: Path to a folder of csv files (OSeMOSYS inputs). File names should correspond to OSeMOSYS parameter names. results_path: Path to a folder of csv files (OSeMOSYS results). File names should correspond to OSeMOSYS variable names. config_path: Path to the configuration file (see below) output_path: Path to the .xlsx file you wish to write out

The IAMC format

The IAMC format was developed by the Integrated Assessment Modeling Consortium (IAMC) and is used in many model comparison projects at the global and national level. It can be used for integrated-assessment models, energy-systems scenarios and analysis of specific sectors like transport, industry or buildings.

The format is a tabular structure with the columns model, scenario, region, variable, unit, and a time domain. Each project defines "codelists" to be used across modelling teams for comparison and analysis of results.

The most recent high-profile application of the IAMC format is the AR6 Scenario Explorer hosting the scenario ensemble supporting the quantitative assessment in the contribution by Working Group III to the IPCC's Sixth Assessment Report (AR6).

Please refer to the Horizon 2020 project openENTRANCE for more information about the format and its usage in that project.

Writing a configuration file

Write a configuration file in YAML format. A simple configuration file with two result variables looks like this:

model: OSeMBE v1.0.0
scenario: DIAG-C400-lin-ResidualFossil
region: iso2_x, iso3_x, from_csv, or a name of a country/region [substitute x with start, end, or a positive number]
results:
- iamc_variable: 'Carbon Capture|Biomass'
  tech_emi: ['(?=^.{2}(BM))^.{4}(CS)']
  emissions: [CO2]
  unit: kt CO2/yr
  transform: abs
  osemosys_param: AnnualTechnologyEmission
- iamc_variable: 'Capital Investment'
  capacity: ['^.*$']
  unit: MUSD
  osemosys_param: CapitalInvestment

The first section of the configuration file with the keys model, scenario, and region are used to define the metadata for the IAMC template.

model and scenario are user-defined, taking any word or phrase and having no set format. region may be defined using 4 methods:

• iso2_x - using the ISO 3166-1 alpha-2 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*
• iso3_x - using the ISO 3166-1 alpha-3 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*
• from_csv - using the region as defined in the REGION column of the CSV file
• Any other word or phrase the user defines (other than an invalid iso option) will be used as the region for all variables. eg. Austria, Nepal, Ethiopia, Guyana, etc.

The x in the above ISO options is to be replaced by:

• start, if the codes are at the beginning of the names
• end, if the codes are at the end of the names, or
• a positive number indicating the position of the first letter of the code in the name. eg. iso2_5 will target the 'GH' in 'POWRGHSOL'

The second section, results, is where you describe each of the IAMC variables and provide instructions to osemosys2iamc on how to compute the values.

iamc_variable - this should match one of the IAMC variable names unit - provide the units of the OSeMOSYS results transform - only abs is currently available. This returns the absolute value of the results osemosys_param - provide the name of the result file from which the script should extract the result data

One or more of the following filter keys. These filter the results by one or more columns found in OSeMOSYS results. Following the fitering, the remaining columns except region and year are discarded and rows are summed.

tech_emi - filter the results by TECHNOLOGY and EMISSION columns using the provide regular expression and an emissions entry emissions - a list of emissions to filter the results by the EMISSION column fuel - filter by the FUEL column capacity - filter the TECHNOLOGY column primary_technology - filter the TECHNOLOGY column (can be replaced by capacity key) excluded_prod_tech - filter the TECHNOLOGY column (can be replaced by capacity key) el_prod_technology - filter the TECHNOLOGY column (can be replaced by capacity key) demand - filters by the FUEL column (final energy)

The value for each of these keys is a list of regular expressions. These regular expressions are used to filter the rows of data in the chosen column to those that match the regular expression.

Writing regular expressions can be tricky, but there are useful tools to help. Below we provide some examples:

^.{2}(WI) match rows with any two characters followed by WI

(?=^.{2}(HF))^((?!00).)*$ match rows with HF in 3rd and 4th position which do not include 00

(?=^.{2}(NG))^((?!(00)|(CS)).)*$ match rows with NG in 3rd and 4th position that do not include 00 or CS

^.{6}(I0) match rows which contain any 6 characters followed by IO in the 7th and 8th position

Putting this together, the following entry extracts results from the result file ProductionByTechnologyAnnual.csv, filters out the rows by matching values in the TECHNOLOGY column with a list of 6 regular expressions (this is an OR operation) and assigns the unit PJ/yr and adds the aggregated (summing over region and year) total under Primary Energy in the IAMC template format.

- iamc_variable: 'Primary Energy'
  primary_technology: ['^.{6}(I0)','^.{6}(X0)','^.{2}(HY)','^.{2}(OC)','^.{2}(SO)','^.{2}(WI)']
  unit: PJ/yr
  osemosys_param: ProductionByTechnologyAnnual

List of relevant IAMC variables for OSeMOSYS

Primary Energy

Level	Commodity
Primary Energy
Primary Energy	Biomass
Primary Energy	Coal
Primary Energy	Fossil
Primary Energy	Gas
Primary Energy	Geothermal
Primary Energy	Hydro
Primary Energy	Non-Biomass Renewables
Primary Energy	Nuclear
Primary Energy	Ocean
Primary Energy	Oil
Primary Energy	Other
Primary Energy	Secondary Energy Trade
Primary Energy	Solar
Primary Energy	Wind

Priority Variables

Level	Commodity	Fuel
Secondary Energy	Electricity	Coal, w/ and w/o CCS
Secondary Energy	Liquids	Coal
Secondary Energy	Gases	Coal
Secondary Energy	Solids	Coal
Secondary Energy	hydrogen	Coal, w/ and w/o CCS

Level	Commodity	Fuel
Final Energy	Solids	Coal
Final Energy	Industry	Solids
Final Energy	Residential and Commercial	Solids
Final Energy	Other Sector	Solids

Level	Commodity	Sector	Sector	Sector
Emissions	CO2	Energy and Industrial Processes
Emissions	CO2	Energy	Supply
Emissions	CO2	Energy	Supply	Electricity
Emissions	CO2	Energy	Supply	Other Sector
Emissions	CO2	Energy	Demand
Emissions	CO2	Energy	Demand	Industry
Emissions	CO2	Energy	Demand	Residential and Commercial
Emissions	CO2	Energy	Demand	Transportation
Emissions	CO2	Energy	Demand	Agriculture
Emissions	CO2	Energy	Demand	Other Sector
Emissions	CO2	Industrial Processes
Emissions	CO2	AFOLU

Level	Commodity	Technology	Type
Capacity	Electricity	Solar	PV
Capacity	Electricity	Solar	CSP
Capacity	Electricity	Wind
Capacity	Electricity	Wind	Offshore
Capacity	Electricity	Wind	Onshore
Carbon Sequestration	CCS
Carbon Sequestration	CCS	Biomass
Carbon Sequestration	CCS	Fossil
Carbon Sequestration	Direct Air Capture
Emissions	N2O
Emissions	CH4