Pyam (compat.pyam)

Package documentation

The “IAMC data structure” is a particular data structure with either 6 or 7 dimensions: model, scenario, region, variable, unit, either year or time, and optionally subannual. Data with this structure are usually stored in a tablular “IAMC format,” wherein each dimension is stored as one column, and the remaining column, labeled value, contains observation values.

Using add_as_pyam() (Computer.add(..., "as_pyam", ...)):

• model and scenario are populated from the attributes of the object returned by the Computer key scenario;

• variable contains the name(s) of each of the quantities, or others constructed by collapse (see below);

• unit contains the units associated with each of the quantities; and

• year, time, and optionally subannual can be created using rename or collapse operations.

A callback function (collapse) can be supplied that modifies the data before it is converted to an IamDataFrame; for instance, to concatenate extra dimensions into variable labels. Other dimensions can simply be dropped (with drop). Dimensions that are not collapsed or dropped will appear as additional columns in the resulting IamDataFrame; this is valid, but non-standard data per the IAMC format.

For example, here the labels for the MESSAGEix t (technology) and m (mode) dimensions are appended to a fixed string to construct variable labels:

c = Computer

def m_t(df):
   """Collapse `t` and `m` dimensions to an IAMC 'Variable' string."""
   df["variable"] = "Activity|" + df["t"] + "|" + df["m"]
   return df

ACT = c.full_key('ACT')
keys = c.add(ACT, "as_pyam", "ya", collapse=m_t, drop=["t", "m"])

as_pyam(scenario, quantity, *[, rename, ...])	Return a pyam.IamDataFrame containing the data from quantity.
add_as_pyam(func, c, quantities[, tag])	Computer.add() helper for as_pyam().
concat(*args, **kwargs)	Concatenate args, which must all be pyam.IamDataFrame.
write_report(obj, path)	Write obj to the file at path.

genno.compat.pyam.computations.add_as_pyam(func, c: Computer, quantities: Key | str | Iterable[Key | str], tag='iamc', /, **kwargs)

Computer.add() helper for as_pyam().

Add conversion of one or more quantities to the IAMC data structure.

Parameters:

• quantities (str or Key or list of (str, Key)) – Keys for quantities to transform.

• tag (str, optional) – Tag to append to new Keys.

• kwargs – Any keyword arguments accepted by as_pyam().

Returns:

Each task converts a Quantity into a pyam.IamDataFrame.

Return type:

list of Key

genno.compat.pyam.computations.as_pyam(scenario, quantity: Quantity, *, rename={}, collapse: Callable | None = None, replace={}, drop: Collection[str] | str = 'auto', unit=None)

Return a pyam.IamDataFrame containing the data from quantity.

Warnings are logged if the arguments result in additional, unhandled columns in the resulting data frame that are not part of the IAMC spec.

The conversion has the following steps:

1. quantity is converted to a temporary pandas.DataFrame.

2. Labels for the following IAMC dimensions are filled:

   • model, scenario: from attributes of the scenario argument.

   • variable: from the name of quantity, if any.

   • unit: from the units of quantity, if any.

3. The actions specified by the optional arguments rename, collapse, replace, drop, and unit, if any, are applied in that order.

4. The resulting data frame is converted to pyam.IamDataFrame.

Parameters:

• scenario – Any object with model and scenario attributes of type str, for instance an ixmp.Scenario.

• rename (dict (str -> str), optional) – Mapping from dimension names in quantity to column names; either IAMC dimension names, or others that are consumed by collapse.

• collapse (callable, optional) – Function that takes a pandas.DataFrame and returns the same type. This function may collapse 2 or more dimensions, for example to construct labels for the IAMC variable dimension, or any other.

• replace (optional) – Values to be replaced and their replaced. Passed directly to pandas.DataFrame.replace().

• drop (str or collection of str, optional) – Columns to drop. Passed to util.drop(), so if not given, all non-IAMC columns are dropped.

• unit (str, optional) – Label for the IAMC unit dimension. Passed to clean_units().

Raises:

ValueError – If the resulting data frame has duplicate keys in the IAMC dimensions. pyam.IamDataFrame cannot handle such data.

genno.compat.pyam.computations.concat(*args, **kwargs)

Concatenate args, which must all be pyam.IamDataFrame.

Otherwise, equivalent to genno.computations.concat().

genno.compat.pyam.computations.write_report(obj, path: str | PathLike) → None

Write obj to the file at path.

If obj is a pyam.IamDataFrame and path ends with “.csv” or “.xlsx”, use pyam methods to write the file to CSV or Excel format, respectively. Otherwise, equivalent to genno.computations.write_report().

Configuration

compat.pyam adds a handler for a iamc: configuration file section.

pyam.iamc(info)

Handle one entry from the iamc: config section.

Computer-specific configuration.

Invokes add_as_pyam() and adds additional computations to convert data from Quantity to a pyam.IamDataFrame. Each entry contains:

variable: (str)

Variable name. This is used two ways: it is placed in the variable label of the resulting IamDataFrame; and the Computer key for executing the conversion is <variable>:iamc.

base: (str)

Key for the quantity to convert.

select: (dict, optional)

Keyword arguments to computations.select(). This selection is performed while data is in Quantity format, before it is passed to as_pyam().

rename: (dict, optional)

Passed to as_pyam().

replace: (dict, optional)

Passed to as_pyam().

drop: (list of str, optional)

Passed to as_pyam().

unit: (str, optional)

Passed to as_pyam().

Any further additional entries are passed as keyword arguments to collapse(), which is then given as the collapse callback for as_pyam().

collapse() formats the variable labels of the IamDataFrame. The variable name replacements from the iamc variable names: section of the config file are applied to all variables.

Utilities

genno.compat.pyam.util.clean_units(df: DataFrame, unit=None) → DataFrame

Convert magnitudes and units of df to unit in str.

Raises:

ValueError – if there is more than one unit.

genno.compat.pyam.util.collapse(df: DataFrame, columns: Mapping[str, Sequence[str]] = {}, sep='|') → DataFrame

Collapse columns into the IAMC columns of df.

genno.compat.pyam.util.drop(df: DataFrame, columns: Collection[str] | str) → DataFrame

Drop columns if given, or all non-IAMC columns.