Pyam (compat.pyam)

Package documentation

The “IAMC data structure” is a particular data structure with either 6 or 7 dimensions: model, scenario, region, variable, unit, either year or time, and optionally subannual. Data with this structure are usually stored in a tablular “IAMC format,” wherein each dimension is stored as one column, and the remaining column, labeled value, contains observation values.

Using add_as_pyam() (Computer.add(..., "as_pyam", ...)):

• model and scenario are populated from the attributes of the object returned by the Computer key scenario;

• variable contains the name(s) of each of the quantities, or others constructed by collapse (see below);

• unit contains the units associated with each of the quantities; and

• year, time, and optionally subannual can be created using rename or collapse operations.

A callback function (collapse) can be supplied that modifies the data before it is converted to an IamDataFrame; for instance, to concatenate extra dimensions into variable labels. Other dimensions can simply be dropped (with drop). Dimensions that are not collapsed or dropped will appear as additional columns in the resulting IamDataFrame; this is valid, but non-standard data per the IAMC format.

For example, here the labels for the MESSAGEix t (technology) and m (mode) dimensions are appended to a fixed string to construct variable labels:

c = Computer

def m_t(df):
   """Collapse `t` and `m` dimensions to an IAMC 'Variable' string."""
   df["variable"] = "Activity|" + df["t"] + "|" + df["m"]
   return df

ACT = c.full_key('ACT')
keys = c.add(ACT, "as_pyam", "ya", collapse=m_t, drop=["t", "m"])

genno.compat.pyam.HAS_PYAM = True

bool indicating whether pyam: analysis and visualization  of integrated-assessment & macro-energy scenarios is available.

as_pyam	Return a pyam.IamDataFrame containing the data from quantity.
add_as_pyam(func, c, quantities[, tag])	Computer.add() helper for as_pyam().

This module also registers implementations of concat() and write_report() that handle pyam.IamDataFrame objects.

genno.compat.pyam.operator.add_as_pyam(func, c: Computer, quantities: Key | str | Iterable[Key | str], tag='iamc', /, **kwargs)

Computer.add() helper for as_pyam().

Add conversion of one or more quantities to the IAMC data structure.

Parameters:

• quantities (str or Key or list of str or Key) – Keys for quantities to transform.

• tag (str, optional) – Tag to append to new Keys.

• kwargs – Any keyword arguments accepted by as_pyam().

Returns:

Each task converts a Quantity into a pyam.IamDataFrame.

Return type:

list of Key

genno.compat.pyam.operator.quantity_from_iamc(qty: TQuantity | pyam.IamDataFrame | pandas.DataFrame, variable: str, *, fail: int | str = 'warning') → TQuantity

Extract data for a single measure from qty with IAMC-like structure.

Parameters:

• qty – Must have at least 2 dimensions named ‘v’ (or ‘variable’, any case) and ‘u’ (or ‘unit’, any case).

• variable (str) – Regular expression to match full labels on the v dimension of qty. If the expression contains match groups, they are used to rewrite v labels: only the contents of the first match group are kept. This may be used to discard a portion of the label.

Returns:

The ‘variable’ dimension contains reduced labels. The Quantity.units attribute contains the unique units for the subset of data.

Return type:

Quantity

See also

unique_units_from_dim

Configuration

compat.pyam adds a handler for a iamc: configuration file section.

pyam.iamc(info)

Handle one entry from the iamc: config section.

Computer-specific configuration.

Invokes add_as_pyam() and adds additional computations to convert data from Quantity to a pyam.IamDataFrame. Each entry contains:

variable: (str)

Variable name. This is used two ways: it is placed in the variable label of the resulting IamDataFrame; and the Computer key for executing the conversion is <variable>:iamc.

base: (str)

Key for the quantity to convert.

select: (dict, optional)

Keyword arguments to operator.select(). This selection is performed while data is in Quantity format, before it is passed to as_pyam().

rename: (dict, optional)

Passed to as_pyam().

replace: (dict, optional)

Passed to as_pyam().

drop: (list of str, optional)

Passed to as_pyam().

unit: (str, optional)

Passed to as_pyam().

Any further additional entries are passed as keyword arguments to collapse(), which is then given as the collapse callback for as_pyam().

collapse() formats the variable labels of the IamDataFrame. The variable name replacements from the iamc variable names: section of the config file are applied to all variables.

Utilities

genno.compat.pyam.util.IAMC_DIMS = frozenset({'model', 'region', 'scenario', 'time', 'unit', 'variable', 'year'})

Dimensions of the IAMC data structure used by pyam: analysis and visualization  of integrated-assessment & macro-energy scenarios.

genno.compat.pyam.util.clean_units(df: DataFrame, unit=None) → DataFrame

Convert magnitudes and units of df to unit in str.

Raises:

ValueError – if there is more than one unit.

genno.compat.pyam.util.collapse(df: DataFrame, columns: Mapping[str, Sequence[str]] = {}, sep='|') → DataFrame

Collapse columns into the IAMC columns of df.

genno.compat.pyam.util.drop(df: DataFrame, columns: IndexLabel) → DataFrame

Drop columns if given, or all non-IAMC columns.