Operators

Operators#

Elementary operators for genno.

Unless otherwise specified, these functions accept and return Quantity objects for data arguments/return values. The names and signatures of many operators match the corresponding methods on the Quantity class, and thus also the xarray.DataArray methods of the same names.

Genno’s compatibility modules each provide additional operators.

Numerical operators:

`add`(*quantities[, fill_value])	Sum across multiple quantities.
`aggregate`(quantity, groups, keep)	Aggregate quantity by groups.
`broadcast_map`(quantity, map[, rename, strict])	Broadcast quantity using a map.
`clip`(qty[, min, max, keep_attrs])	Call `Quantity.clip()`.
`combine`(*quantities[, select, weights])	Sum distinct quantities by weights.
`disaggregate_shares`(quantity, shares)	Deprecated: Disaggregate quantity by shares.
`div`(numerator, denominator)	Compute the ratio numerator / denominator.
`group_sum`(qty, group, sum)	Group by dimension group, then sum across dimension sum.
`index_to`(qty, dim_or_selector[, label])	Compute an index of qty against certain of its values.
`interpolate`(qty[, coords, method, ...])	Interpolate qty.
`mul`(*quantities)	Compute the product of any number of quantities.
`pow`(a, b)	Compute a raised to the power of b.
`product`(*quantities)	Alias of `mul()`, for backwards compatibility.
`ratio`(numerator, denominator)	Alias of `div()`, for backwards compatibility.
`round`(qty, args, *kwargs)	Like `xarray.DataArray.round()`.
`sub`(a, b)	Subtract b from a.
`sum`(quantity[, weights, dimensions])	Sum quantity over dimensions, with optional weights.
`add_sum`(func, c, key, qty[, weights, dimensions])	`Computer.add()` helper for `sum()`.
`where`(qty, cond[, other, drop])	Call `Quantity.where()`.

Data manipulation and transformation:

`apply_units`(qty, units)	Apply units to qty.
`as_quantity`(info)	Convert various values to Quantity.
`assign_units`(qty, units)	Set the units of qty without changing magnitudes.
`concat`()	Concatenate Quantity objs.
`convert_units`(qty, units)	Convert magnitude of qty from its current units to units.
`drop_vars`(qty, names, *[, errors])	Return a Quantity with dropped variables (coordinates).
`relabel`(qty[, labels])	Replace specific labels along dimensions of qty.
`rename`(qty[, new_name_or_name_dict])	Returns a new Quantity with renamed dimensions or a new name.
`rename_dims`(qty[, name_dict])	Returns a new Quantity with renamed dimensions or a new name.
`select`(qty, indexers, *[, inverse, drop])	Select from qty based on indexers.
`unique_units_from_dim`(qty, dim, *[, fail])	Assign `Quantity.units` using coords from the dimension dim.

Input and output:

`load_file`(path[, dims, units, name])	Read the file at path and return its contents as a `Quantity`.
`add_load_file`(func, c, path[, key])	`Computer.add()` helper for `load_file()`.
`write_report`(-> None -> None -> None -> None)	Write a quantity to a file.

genno.operator.add(*quantities: AttrSeries | SparseDataArray, fill_value: float = 0.0) → AttrSeries | SparseDataArray[source]#

Sum across multiple quantities.

Raises:: ValueError – if any of the quantities have incompatible units.
Returns:: Units are the same as the first of quantities.
Return type:: Quantity

See also

add_binop

genno.operator.aggregate(quantity: AttrSeries | SparseDataArray, groups: Mapping[str, Mapping], keep: bool) → AttrSeries | SparseDataArray[source]#

Aggregate quantity by groups.

Parameters:

groups (dict of dict) – Top-level keys are the names of dimensions in quantity. Second-level keys are group names; second-level values are lists of labels along the dimension to sum into a group. Labels may be literal values, or compiled re.Pattern objects; in the latter case, all matching labels (according to re.Pattern.fullmatch()) are included in the group to be aggregated.
keep (bool) – If True, the members that are aggregated into a group are returned with the group sums. If False, they are discarded.

Returns:

Same dimensionality as quantity.

Return type:

Quantity

genno.operator.apply_units(qty: AttrSeries | SparseDataArray, units: str | Unit | Quantity) → AttrSeries | SparseDataArray[source]#

Apply units to qty.

If qty has existing units…

…with compatible dimensionality to units, the magnitudes are adjusted, i.e. behaves like convert_units().
…with incompatible dimensionality to units, the units attribute is overwritten and magnitudes are not changed, i.e. like assign_units(), with a log message on level WARNING.

To avoid ambiguities between the two cases, use convert_units() or assign_units() instead.

Parameters:: units (str or pint.Unit) – Units to apply to qty.

genno.operator.as_quantity(info: dict | float | str) → AttrSeries | SparseDataArray[source]#

Convert various values to Quantity.

This operator can be useful when handling values from user input or various file formats.

Examples

str, via pint:

>>> as_quantity("3.0 kg")

dict:

A ‘_dim’ key is removed and treated as Quantity.dims.
A ‘_unit’ key is removed and treated as Quantity.units.

>>> value = {
...     ("x0", "y0"): 1.0,
...     ("x1", "y1"): 2.0,
...     "_dim": ("x", "y"),
...     "_unit": "km",
... }
>>> as_quantity(value)

For other values, the Quantity constructor should be used directly:

>>> Quantity(1.2)

genno.operator.assign_units(qty: AttrSeries | SparseDataArray, units: str | Unit | Quantity) → AttrSeries | SparseDataArray[source]#

Set the units of qty without changing magnitudes.

Logs on level INFO if qty has existing units.

Parameters:: units (str or pint.Unit) – Units to assign to qty.

genno.operator.broadcast_map(quantity: AttrSeries | SparseDataArray, map: AttrSeries | SparseDataArray, rename: Mapping = {}, strict: bool = False) → AttrSeries | SparseDataArray[source]#

Broadcast quantity using a map.

The map must be a 2-dimensional Quantity with dimensions (d1, d2), such as returned by ixmp.report.operator.map_as_qty(). quantity must also have a dimension d1. Typically len(d2) > len(d1).

quantity is ‘broadcast’ by multiplying it with map, and then summing on the common dimension d1. The result has the dimensions of quantity, but with d2 in place of d1.

Parameters:

rename (dict, optional) – Dimensions to rename on the result; mapping from original dimension (str) to target name (str).
strict (bool, optional) – Require that each element of d2 is mapped from exactly 1 element of d1.

genno.operator.clip(qty: AnyQuantity, min: types.ScalarOrArray | None = None, max: types.ScalarOrArray | None = None, *, keep_attrs: bool | None = None) → AnyQuantity[source]#: Call Quantity.clip().

genno.operator.combine(*quantities: AttrSeries | SparseDataArray, select: List[Mapping] | None = None, weights: List[float] | None = None) → AttrSeries | SparseDataArray[source]#

Sum distinct quantities by weights.

Parameters:

*quantities (Quantity) – The quantities to be added.
select (list of dict) – Elements to be selected from each quantity. Must have the same number of elements as quantities.
weights (list of float) – Weight applied to each quantity. Must have the same number of elements as quantities.

Raises:

ValueError – If the quantities have mismatched units.

genno.operator.concat(*objs: AttrSeries | SparseDataArray, **kwargs) → AttrSeries | SparseDataArray[source]#

genno.operator.concat(*args: IamDataFrame, **kwargs) → IamDataFrame

Concatenate Quantity objs.

Any strings included amongst objs are discarded, with a logged warning; these usually indicate that a quantity is referenced which is not in the Computer.

genno.operator.convert_units(qty: AttrSeries | SparseDataArray, units: str | Unit | Quantity) → AttrSeries | SparseDataArray[source]#

Convert magnitude of qty from its current units to units.

Parameters:: units (str or pint.Unit) – Units to assign to qty.
Raises:: ValueError – if units does not match the dimensionality of the current units of qty.

genno.operator.disaggregate_shares(quantity: AttrSeries | SparseDataArray, shares: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray[source]#

Deprecated: Disaggregate quantity by shares.

This operator is identical to mul(); use mul() and its helper instead.

genno.operator.div(numerator: AttrSeries | SparseDataArray | float, denominator: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray[source]#

Compute the ratio numerator / denominator.

Parameters:

numerator (Quantity) –
denominator (Quantity) –

See also

add_load_file

genno.operator.mul(*quantities: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray[source]#: Compute the product of any number of quantities.

See also

add_binop

genno.operator.pow(a: AttrSeries | SparseDataArray, b: AttrSeries | SparseDataArray | int) → AttrSeries | SparseDataArray[source]#

Compute a raised to the power of b.

Returns:: If b is int or a Quantity with all int values that are equal to one another, then the quantity has the units of a raised to this power; for example, “kg²” → “kg⁴” if b is 2. In other cases, there are no meaningful units, so the returned quantity is dimensionless.
Return type:: Quantity

genno.operator.product(*quantities: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray#: Alias of mul(), for backwards compatibility.

Note

This may be deprecated and possibly removed in a future version.

genno.operator.ratio(numerator: AttrSeries | SparseDataArray | float, denominator: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray#: Alias of div(), for backwards compatibility.

Note

This may be deprecated and possibly removed in a future version.

genno.operator.relabel(qty: AttrSeries | SparseDataArray, labels: Mapping[Hashable, Mapping] | None = None, **dim_labels: Mapping) → AttrSeries | SparseDataArray[source]#

Replace specific labels along dimensions of qty.

Parameters:

labels – Keys are strings identifying dimensions of qty; values are further mappings from original labels to new labels. Dimensions and labels not appearing in qty have no effect.
dim_labels – Mappings given as keyword arguments, where argument name is the dimension.

Raises:

ValueError – if both labels and dim_labels are given.

genno.operator.rename(qty: AttrSeries | SparseDataArray, new_name_or_name_dict: Hashable | Mapping[Any, Hashable] | None = None, **names: Hashable) → AttrSeries | SparseDataArray[source]#

Returns a new Quantity with renamed dimensions or a new name.

Like xarray.DataArray.rename(), and identical in behaviour to rename_dims().

genno.operator.rename_dims(qty: AttrSeries | SparseDataArray, name_dict: Hashable | Mapping[Any, Hashable] | None = None, **names: Hashable) → AttrSeries | SparseDataArray[source]#

Returns a new Quantity with renamed dimensions or a new name.

Like xarray.DataArray.rename(), and identical in behaviour to rename(). The two names are provided for more expressive user code.

genno.operator.round(qty: AttrSeries | SparseDataArray, *args, **kwargs) → AttrSeries | SparseDataArray[source]#: Like xarray.DataArray.round().

genno.operator.select(qty: AttrSeries | SparseDataArray, indexers: Mapping[Hashable, Iterable[Hashable]], *, inverse: bool = False, drop: bool = False) → AttrSeries | SparseDataArray[source]#

Select from qty based on indexers.

Parameters:

indexers (dict) –
Elements to be selected from qty. Mapping from dimension names (str) to either:
- list of str: coords along the respective dimension of qty, or
- xarray.DataArray: xarray-style indexers.
Values not appearing in the dimension coords are silently ignored.
inverse (bool, optional) – If True, remove the items in indexers instead of keeping them.
drop (bool, optional) – If True, drop dimensions that are indexed by a scalar value (for instance, "foo" or 999) in indexers. Note that dimensions indexed by a length-1 list of labels (for instance ["foo"]) are not dropped; this behaviour is consistent with xarray.DataArray.

genno.operator.sub(a: AttrSeries | SparseDataArray, b: AttrSeries | SparseDataArray) → AttrSeries | SparseDataArray[source]#: Subtract b from a.

See also

add_binop

Sum quantity over dimensions, with optional weights.

Parameters:

weights (Quantity, optional) – If dimensions is given, weights must have at least these dimensions. Otherwise, any dimensions are valid.
dimensions (list of str, optional) – If not provided, sum over all dimensions. If provided, sum over these dimensions.

genno.operator.unique_units_from_dim(qty: AttrSeries | SparseDataArray, dim: str, *, fail: str | int = 'raise') → AttrSeries | SparseDataArray[source]#

Assign Quantity.units using coords from the dimension dim.

The dimension dim is dropped from the result.

Raises:: ValueError – if (a) fail is “raise” (the default) and (b) the dimension dim contains more than one unique value. If fail is anything else, a message is logged with level fail, and the returned Quantity is dimensionless.

genno.operator.where(qty: ~genno.core.attrseries.AttrSeries | ~genno.core.sparsedataarray.SparseDataArray, cond: ~typing.Any, other: ~typing.Any = <NA>, drop: bool = False) → AttrSeries | SparseDataArray[source]#: Call Quantity.where().

genno.operator.write_report(quantity: object, path: str | PathLike, kwargs: dict | None = None) → None[source]#

genno.operator.write_report(quantity: str, path: str | PathLike, kwargs: dict | None = None)

genno.operator.write_report(quantity: DataFrame, path: str | PathLike, kwargs: dict | None = None) → None

genno.operator.write_report(quantity: AttrSeries | SparseDataArray, path: str | PathLike, kwargs: dict | None = None) → None

genno.operator.write_report(obj: DataMessage, path, kwargs=None) → None

genno.operator.write_report(quantity: IamDataFrame, path, kwargs=None) → None

Write a quantity to a file.

write_report() is a singledispatch() function. This means that user code can extend this operator to support different types for the quantity argument:

import genno.operator

@genno.operator.write_report.register
def my_writer(qty: MyClass, path, kwargs):
    ... # Code to write MyClass to file

Parameters:

quantity – Object to be written. The base implementation supports Quantity and pandas.DataFrame.
path (str or pathlib.Path) – Path to the file to be written.
kwargs –
Keyword arguments. For the base implementation, these are passed to pandas.DataFrame.to_csv() or pandas.DataFrame.to_excel() (according to path), except for:
- ”header_comment”: valid only for path ending in .csv. Multi-line text that is prepended to the file, with comment characters (”# “) before each line.

Raises:

NotImplementedError – If quantity is of a type not supported by the base implementation or any overloads.

Helper functions for adding tasks to Computers#

genno.operator.add_binop(func, c: Computer, key, *quantities, **kwargs) → Key[source]#

Computer.add() helper for binary operations.

Add a computation that applies add(), div(), mul(), or sub() to quantities.

Parameters:

key (str or Key) – Key or name of the new quantity. If a Key, any dimensions are ignored; the dimensions of the result are the union of the dimensions of quantities.
sums (bool, optional) – If True, all partial sums of the new quantity are also added.

Returns:

The full key of the new quantity.

Return type:

Key

Example

>>> c = Computer()
>>> x = c.add("x:a-b-c", ...)
>>> y = c.add("y:c-d-e", ...)
>>> z = c.add("z", "mul", x, y)
>>> z
<z:a-b-c-d-e>

genno.operator.add_load_file(func, c: Computer, path, key=None, **kwargs)[source]#

Computer.add() helper for load_file().

Add a task to load an exogenous quantity from path. Computing the key or using it in other computations causes path to be loaded and converted to Quantity.

Parameters:

path (os.PathLike) – Path to the file, e.g. ‘/path/to/foo.ext’.
key (str or Key, optional) – Key for the quantity read from the file.
dims (dict or list or set) – Either a collection of names for dimensions of the quantity, or a mapping from names appearing in the input to dimensions.
units (str or pint.Unit) – Units to apply to the loaded Quantity.

Returns:

Either key (if given) or e.g. file foo.ext based on the path name, without directory components.

Return type:

Key

genno.operator.add_sum(func, c: Computer, key, qty, weights=None, dimensions=None, **kwargs) → Key | str | Tuple[Key | str, ...][source]#

Computer.add() helper for sum().

If key has the name “*”, the returned key has name and dimensions inferred from qty and dimensions, and only the tag (if any) of key is preserved.

Operators

Contents

Operators#

Helper functions for adding tasks to Computers#