Top-level classes and functions¶

`Computer`(**kwargs)	Class for describing and executing computations.
`Key`(name_or_value[, dims, tag, _fast])	A hashable key for a quantity that includes its dimensionality.
`Keys`(**kwargs)	A collection of `Key`.
`Quantity`	alias of `AttrSeries`
`assert_quantity`(*args)	Assert that each of args is a Quantity object.
`configure`([path])	Configure `genno` globally.
`get_class`()	Get the current `Quantity` implementation in use.
`set_class`([name])	Set the `Quantity` implementation to be used.

Also: ComputationError, KeyExistsError, MissingKeyError, Operator, literal(), quote().

class genno.Computer(**kwargs)[source]¶

Class for describing and executing computations.

Parameters:: kwargs – Passed to configure().

A Computer is used to prepare (add() and related methods) and then execute (get() and related methods) computations stored in a graph. Advanced users may manipulate the graph directly; but most computations can be prepared using the methods of Computer.

Instance attributes:

`default_key`	The default key to `get()` with no argument.
`graph`	A dask-format graph (see 1, 2).
`keys`()	Return the keys of `graph`.
`modules`	List of modules containing operators.
`unit_registry`	The `pint.UnitRegistry` used by the Computer.

General-purpose methods for preparing computations and tasks:

`add`(data, args, *kwargs)	General-purpose method to add computations.
`add_queue`(queue[, max_tries, fail])	Add tasks from a list or queue.
`add_single`(key, *computation[, strict, index])	Add a single computation at key.
`aggregate`(qty, tag, dims_or_groups[, ...])	Deprecated.
`apply`(generator, keys, *kwargs)	Add computations by applying generator to keys.
`cache`(func)	Decorate func so that its return value is cached.
`describe`([key, quiet])	Return a string describing the computations that produce key.
`duplicate`(key, tag)	Duplicate the task at key and all of its inputs.
`eval`(expr)	Evaluate expr to add tasks and keys.
`insert`(key, *args[, tag])	Insert a task before key, using args, kwargs.
`visualize`(filename[, key, optimize_graph])	Generate an image describing the Computer structure.

Executing computations:

`get`([key])	Execute and return the result of the computation key.
`write`(key, path, **kwargs)	Compute key and write the result directly to path.

Utility and configuration methods:

`check_keys`(*keys[, predicate, action])	Check that keys are in the Computer.
`configure`([path, fail, config])	Configure the Computer.
`full_key`(name_or_key)	Return the full-dimensionality key for name_or_key.
`get_operator`(name)	Return a function, `Operator`, or callable for use in a task.
`infer_keys`(key_or_keys[, dims])	Infer complete key_or_keys.
`require_compat`(pkg)	Register a module for `get_operator()`.

Deprecated:

`add_file`(args, *kwargs)	Deprecated.
`add_product`(args, *kwargs)	Deprecated.
`convert_pyam`(args, *kwargs)	Deprecated.
`disaggregate`(qty, new_dim[, method, args])	Deprecated.

graph: genno.core.graph.Graph = {'config': {}}¶

A dask-format graph (see 1, 2).

Dictionary keys are either Key, str, or any other hashable value.

Dictionary values are computations, one of:

Any other, existing key in the Computer. This functions as an alias.
Any other literal value or constant, to be returned directly.
A task tuple: a callable (such as a function or any object with a __call__() method), followed by zero or more keys (referring to the output of other computations), or computations directly.
A list containing zero or more of (1), (2), and/or (3).

genno reserves some keys for special usage:

"config": A dict storing configuration settings. See Configuration. Because this information is stored in the graph, it can be used as one input to other operators.

Some inputs to tasks may be confused for (1) or (4), above. The recommended way to protect these is:

Literal str inputs to tasks: use functools.partial() on the function that is the first element of the task tuple.
list of str: use dask.core.quote() to wrap the list.

add(data, *args, **kwargs) → KeyLike | tuple[KeyLike, ...][source]¶

General-purpose method to add computations.

add() can be called in several ways; its behaviour depends on data; see below. It chains to methods such as add_single(), add_queue(), and/or apply(); each can also be called directly.

Returns:: Some or all of the keys added to the Computer.
Return type:: KeyLike or tuple of KeyLike

The data argument may be:

list: A list of computations, like [(list(args1), dict(kwargs1)), (list(args2), dict(kwargs2)), ...] → passed to add_queue().
str naming an operator: e.g. “select”, retrievable with get_operator(). add_single() is called with (key=args[0], data, *args[1], **kwargs), that is, applying the named operator to the other parameters.
Key or other str:: Passed to add_single().

add() may be used to:

Provide an alias from one key to another:

>>> from genno import Computer
>>> c = Computer()  # Create a new Computer object
>>> c.add('aliased name', 'original name')

Define an arbitrarily complex operator in a Python function that operates directly on the ixmp.Scenario:

>>> def my_func(scenario):
>>>     # many lines of code
>>>     return 'foo'
>>> c.add('my report', my_func, 'scenario')
>>> c.finalize(scenario)
>>> c.get('my report')
foo

add_queue(queue: Iterable[tuple], max_tries: int = 1, fail: str | int | None = None) → tuple[KeyLike, ...][source]¶

Add tasks from a list or queue.

Parameters:

queue (collections.abc.Iterable of tuple) – Each item is either a N-tuple of positional arguments to add(), or a 2-tuple of (tuple of positional arguments, dict of keyword arguments).
max_tries (int, optional) – Retry adding elements up to this many times.
fail ("raise" or str or logging level, optional) – Action to take when a computation from queue cannot be added after max_tries: “raise” an exception, or log messages on the indicated level and continue.

This method allows to add many computations at once by, in effect, calling add() repeatedly with sets of positional and (optionally) keyword arguments taken from the queue. The argument may be:

A prepared/static data structure, like a list, where each item is either a 2-tuple of (args, kwargs) or only a tuple of args that can be passed to add().
A generator that yields items of the same type(s).

Given this initial sequence of items, add_queue() will…

Pass each item in turn to add();
If an item fails to be added—for instance, with MissingKeyError on one of its inputs—and max_tries > 1: re-append that item to the queue so that it can be attempted again;
If an item fails to be added at least max_tries times: take an action according to fail.

This behaviour makes add_queue() tolerant of entries in queue that are out-of-order: individual items may fail in calls to add() on initial passes through the queue, but eventually succeed once their inputs are available.

apply(generator: Callable, *keys, **kwargs) → KeyLike | tuple[KeyLike, ...][source]¶

Add computations by applying generator to keys.

Parameters:

generator (typing.Callable) –
Function to apply to keys. This function may take a first positional argument annotated with Computer or a subtype; if so, then it is provided with a reference to self.

The function may:
- yield or return an iterable of (key, computation). These are used to directly update the graph, and then apply() returns the added keys.
- If it is provided with a reference to the Computer, call add() or any other method to update the graph. In this case, it should return a Key or sequence of keys, indicating what was added; these are in turn returned by apply().
keys (Hashable) – The starting key(s). These are provided as positional arguments to generator.
kwargs – Keyword arguments to generator.

The generator may have a type annotation for Computer on its first positional argument. In this case, a reference to the Computer is supplied, and generator can use the Computer methods to add many keys and computations:

def my_gen0(c: genno.Computer, **kwargs):
    c.load_file("file0.txt", **kwargs)
    c.load_file("file1.txt", **kwargs)

# Use the generator to add several computations
c.apply(my_gen0, units="kg")

Or, generator may yield a sequence (0 or more) of (key, computation), which are added to the graph:

def my_gen1(**kwargs):
    op = partial(operator.load_file, **kwargs)
    yield from (f"file:{i}", (op, "file{i}.txt")) for i in range(2)

c.apply(my_gen1, units="kg")

eval(expr: str) → tuple[Key, ...][source]¶

Evaluate expr to add tasks and keys.

Parse a statement or block of statements using ast from the Python standard library. expr may include:

Constants.
References to existing keys in the Computer by their name; these are expanded using full_key().
Multiple statements on separate lines or separated by “;”.
Python arithmetic operators including +, -, *, /, **; these are mapped to the corresponding operator.
Function calls, also mapped to the corresponding operator via get_operator(). These may include simple positional (constants or key references) or keyword (constants only) arguments.

Parameters:

expr (str) – Expression to be evaluated.

Returns:

One key for the left-hand side of each expression.

Return type:

tuple of Key

Raises:

NotImplementedError – For complex expressions not supported; if any of the statements is anything other than a simple assignment.
NameError – If a function call references a non-existent computation.

Examples

Parse a multi-line string and add tasks to compute z, a, b, d, and e. The dimensions of each are automatically inferred given the dimension of the existing operand, x.

>>> c = Computer()
>>> # (Here, add tasks to compute a quantity like "x:t-y")
>>> added = c.eval(
...     """
...     z = - (0.5 / (x ** 3))
...     a = x ** 3 + z
...     b = a + a
...     d = assign_units(b, "km")
...     e = index_to(d, dim="t", label="foo1")
...     """
... )
>>> added[-1]
<e:t-y>

Deprecated.

Add a computation that aggregates qty.

Deprecated since version 1.18.0: Instead, for a mapping/dict dims_or_groups, use:

c.add(qty, "aggregate", groups=dims_or_groups, keep=keep, ...)

Or, for str or sequence of str dims_or_groups, use:

c.add(None, "sum", qty, dimensions=dims_or_groups, ...)

Parameters:

qty (Key or str) – Key of the quantity to be aggregated.
tag (str) – Additional string to add to the end the key for the aggregated quantity.
dims_or_groups (str or collections.abc.Iterable of str or dict) – Name(s) of the dimension(s) to sum over, or nested dict.
weights (xarray.DataArray, optional) – Weights for weighted aggregation.
keep (bool, optional) – Passed to operator.aggregate.
sums (bool, optional) – Passed to add().
fail (str or int, optional) – Passed to add_queue() via add().

Returns:

The key of the newly-added node.

Return type:

Key

add_file(*args, **kwargs)[source]¶

Deprecated.

Deprecated since version 1.18.0: Instead use add_load_file() via:

c.add(..., "load_file", ...)

add_product(*args, **kwargs)[source]¶

Deprecated.

Deprecated since version 1.18.0: Instead use add_binop() via:

c.add(..., "mul", ...)

add_single(key: KeyLike, *computation, strict=False, index=False) → KeyLike[source]¶

Add a single computation at key.

Parameters:

key (str or Key or collections.abc.Hashable) – A string, Key, or other value identifying the output of computation.
computation (object) – Any computation. See graph.
strict (bool, optional) – If True, key must not already exist in the Computer, and any keys referred to by computation must exist.
index (bool, optional) – If True, key is added to the index as a full-resolution key, so it can be later retrieved with full_key().

Raises:

KeyExistsError – If strict is True and either (a) key already exists; or (b) sums is True and the key for one of the partial sums of key already exists.
MissingKeyError – If strict is True and any key referred to by computation does not exist.

Deprecated.

Add a computation that aggregates qty.

Deprecated since version 1.18.0: Instead, for a mapping/dict dims_or_groups, use:

c.add(qty, "aggregate", groups=dims_or_groups, keep=keep, ...)

Or, for str or sequence of str dims_or_groups, use:

c.add(None, "sum", qty, dimensions=dims_or_groups, ...)

Parameters:

qty (Key or str) – Key of the quantity to be aggregated.
tag (str) – Additional string to add to the end the key for the aggregated quantity.
dims_or_groups (str or collections.abc.Iterable of str or dict) – Name(s) of the dimension(s) to sum over, or nested dict.
weights (xarray.DataArray, optional) – Weights for weighted aggregation.
keep (bool, optional) – Passed to operator.aggregate.
sums (bool, optional) – Passed to add().
fail (str or int, optional) – Passed to add_queue() via add().

Returns:

The key of the newly-added node.

Return type:

Key

cache(func)[source]¶: Decorate func so that its return value is cached.

See also

Caching

check_keys(*keys: str | Key, predicate=None, action='raise') → list[KeyLike][source]¶

Check that keys are in the Computer.

Parameters:

keys (KeyLike) – Some Keys or strings.
predicate (typing.Callable, optional) – Function to run on each of keys; see below.
action ("raise" or str) – Action to take on missing keys.

Returns:

One item for each item k in keys:

k itself, unchanged, if predicate is given and predicate(k) returns True.
Graph.unsorted_key(), that is, k but with its dimensions in a specific order that already appears in graph.
Graph.full_key(), that is, an existing key with the name k with its full dimensionality.
None otherwise.

Return type:

list of KeyLike

Raises:

MissingKeyError – If action is “raise” and 1 or more of keys do not appear (either in different dimension order, or full dimensionality) in the graph.

configure(path: Path | str | None = None, fail: str | int = 'raise', config: Mapping[str, Any] | None = None, **config_kw)[source]¶

Configure the Computer.

Accepts a path to a configuration file and/or keyword arguments. Configuration keys loaded from file are superseded by keyword arguments. Messages are logged at level logging.INFO if config contains unhandled sections.

See Configuration for a list of all configuration sections and keys, and details of the configuration file format.

Parameters:

path (pathlib.Path, optional) – Path to a configuration file in JSON or YAML format.
fail ("raise" or str or logging level, optional) – Passed to add_queue(). If not “raise”, then log messages are generated for config handlers that fail. The Computer may be only partially configured.
config – Configuration keys/sections and values, as a mapping. Use this if any of the keys/sections are not valid Python names, for instance if they contain “-” or “ “.
**config_kw – Configuration keys/sections and values, as keyword arguments.

convert_pyam(*args, **kwargs)[source]¶

Deprecated.

Deprecated since version 1.18.0: Instead use add_as_pyam() via:

c.require_compat("pyam")
c.add(..., "as_pyam", ...)

default_key: KeyLike | None = None¶: The default key to get() with no argument.

describe(key=None, quiet=True)[source]¶

Return a string describing the computations that produce key.

If key is not provided, all keys in the Computer are described.

Unless quiet, the string is also printed to the console.

Returns:: Description of computations. If a malformed graph is detected (one key is its own direct ancestor), the text “← CYCLE DETECTED” is shown, and recursion stops.
Return type:: str

disaggregate(qty, new_dim, method='shares', args=[])[source]¶

Deprecated.

Deprecated since version 1.18.0: Instead, for method = “disaggregate_shares”, use:

c = Computer()
c.add(qty.append(new_dim), "mul", qty, ..., strict=True)

Or for a callable() method, use:

c.add(qty.append(new_dim), method, qty, ..., strict=True)

duplicate(key: TKeyLike, tag: str) → TKeyLike[source]¶

Duplicate the task at key and all of its inputs.

Parameters:

key – Starting key to duplicate.
tag – tag to add to duplicated keys.

full_key(name_or_key: KeyLike) → KeyLike[source]¶

Return the full-dimensionality key for name_or_key.

An quantity ‘foo’ with dimensions (a, c, n, q, x) is available in the Computer as 'foo:a-c-n-q-x'. This Key can be retrieved with:

c.full_key("foo")
c.full_key("foo:c")
# etc.

Raises:: KeyError – if name_or_key is not in the graph.

get(key=None)[source]¶

Execute and return the result of the computation key.

Only key and its dependencies are computed.

Parameters:: key (str, optional) – If not provided, default_key is used.
Raises:: ValueError – If key and default_key are both None.

get_comp(name) → Callable | None¶: Alias of get_operator().

get_operator(name) → Callable | None[source]¶

Return a function, Operator, or callable for use in a task.

get_operator() checks each of the modules for a callable with the given name. Modules at the end of the list take precedence over those earlier in the list.

Returns:

typing.Callable
None – If there is no callable with the given name in any of modules.

infer_keys(key_or_keys: KeyLike | Iterable[KeyLike], dims: Iterable[str] = [])[source]¶

Infer complete key_or_keys.

Each return value is one of:

a Key with either
- dimensions dims, if any are given, otherwise
- its full dimensionality (cf. full_key())
str, the same as input, if the key is not defined in the Computer.

Parameters:

key_or_keys (KeyLike or list of KeyLike)
dims (list of str, optional) – Drop all but these dimensions from the returned key(s).

Returns:

KeyLike – If key_or_keys is a single KeyLike.
list of KeyLike – If key_or_keys is an iterable of KeyLike.

insert(key: KeyLike, *args, tag: str = 'pre', **kwargs) → None[source]¶

Insert a task before key, using args, kwargs.

The existing task at key is moved to key + tag. The args and kwargs are passed to add() to insert a new task at key. The args must include at least 2 items:

the new callable or Operator, and
the .Ellipsis (...), which is replaced by the shifted key + tag.

If there are more than 2 items, each instance of the Ellipsis is replaced per (2); all other items (and kwargs) are passed on as-is.

The effect is that all existing tasks to which key are input will receive, instead, the output of the added task.

One way to use insert() is with a ‘pass-through’ operation that, for instance, performs logging, assertions, or other steps, then returns its input unchanged. It is also possible to insert a new task that mutates its input in certain ways.

keys()[source]¶: Return the keys of graph.

modules: MutableSequence[ModuleType] = []¶

List of modules containing operators.

By default, this includes the genno built-in operators in genno.operator. require_compat() appends additional modules, for instance genno.compat.plotnine, to this list. User code may also add modules to this list directly.

require_compat(pkg: str | ModuleType)[source]¶

The specified module is appended to modules.

Parameters:

pkg (str or module) –

One of:

the name of a package (for instance “plotnine”), corresponding to a submodule of genno.compat (genno.compat.plotnine). genno.compat.{pkg}.operator is added.
the name of any importable module, for instance “foo.bar”.
a module object that has already been imported.

Raises:

ModuleNotFoundError – If the required packages are missing.

Examples

Operators packaged with genno for compatibility:

>>> c = Computer()
>>> c.require_compat("pyam")

Operators in another module, using the module name:

>>> c.require_compat("ixmp.reporting.computations")

or using imported module object directly:

>>> import ixmp.reporting.computations as mod
>>> c.require_compat(mod)

property unit_registry¶: The pint.UnitRegistry used by the Computer.

visualize(filename, key=None, optimize_graph=False, **kwargs)[source]¶

Generate an image describing the Computer structure.

This is similar to dask.visualize(); see compat.graphviz.visualize(). Requires graphviz.

write(key, path, **kwargs)[source]¶: Compute key and write the result directly to path.

class genno.Key(name_or_value: str | Key | AnyQuantity, dims: Iterable[str] = [], tag: str | None = None, _fast: bool = False)[source]¶

A hashable key for a quantity that includes its dimensionality.

Quantities are indexed by 0 or more dimensions. A Key refers to a quantity using three components:

a string name,
zero or more ordered dims, and
an optional tag.

For example, for a quantity \(\text{foo}\) with with three dimensions \(a, b, c\):

\[\text{foo}^{abc}\]

…Key allows a specific, explicit reference to various forms of “foo”:

in its full resolution; that is, indexed by a, b, and c:

>>> k1 = Key("foo", ["a", "b", "c"])
>>> k1
<foo:a-b-c>

in a partial sum over one dimension, for instance summed across dimension c, with remaining dimensions a and b:
```
>>> k2 = k1 / "c"
>>> k2 == "foo:a-b"
True
```

in a partial sum over multiple dimensions, etc.:

>>> k1.drop("a", "c") == k1 / ("a", "c") == k2 / "a" == "foo:b"
True

after it has been manipulated by other computations, e.g.

>>> k3 = k1.add_tag(""normalized")
>>> k3
<foo:a-b-c:normalized>
>>> k4 = k3 + "rescaled"
>>> k4
<foo:a-b-c:normalized+rescaled>

Key comparison.

Keys with the same name, dimensions, and tag compare and hash() equal—even if the dimensions are in a different order.
A key compares (but does not hash()) equal to a str with the same name, dimensions (in any order) and tag.

repr(key) prints the Key in angle brackets (‘<>’) to signify that it is a Key object.

>>> str(k1)
'foo:a-b-c'
>>> repr(k1)
'<foo:a-b-c>'
>>> hash(k1) == hash("foo:a-b-c")
True
>>> k1 == "foo:c-b-a"
True

Keys are immutable: the properties name, dims, and tag are read-only, and the methods append(), drop(), and add_tag() return new Key objects.

Keys may be generated concisely by defining a convenience method:

>>> def foo(dims):
>>>     return Key('foo', dims.split())
>>> foo('a b c')
<foo:a-b-c>

Key arithmetic. Keys can be manipulated using some of the Python arithmetic operators:

+: and -: manipulate tag, same as add_tag() and remove_tag() respectively:

>>> k1 = Key("foo", "abc", "bar+baz+qux")
>>> k1
<foo:a-b-c:bar+baz+qux>
>>> k2 + "newtag"
<foo:a-b-c:bar+baz+qux+newtag>
>>> k1 - "baz"
<foo:a-b-c:bar+qux>
>>> k1 - ("bar", "baz")
<foo:a-b-c:qux>

* and /: manipulate dims, similar to append()/product and drop, respectively:

>>> k1 * "d"
<foo:a-b-c-d>
>>> k1 * ("e", "f")
<foo:a-b-c-e-f>
>>> k1 * Key("bar", "ghi")
<foo:a-b-c-g-h-i>

>>> k1 / "a"
<foo:b-c>
>>> k1 / ("a", "c")
<foo:b>
>>> k1 / Key("baz", "cde")
<foo:a-b>

Key generation and derivation. When preparing chains or complicated graphs of computations, it can be useful to use a sequence or set of similar keys to refer to the intermediate steps. Python item-access syntax ([...]) and the built-in function next() can be used to generate or derive keys from an original one, in any order:

>>> k1 = Key("foo:a-b-c")
>>> k[0]
<foo:a-b-c:0>
>>> k[1]
<foo:a-b-c:1>
>>> k["bar"]
<foo:a-b-c:bar>
>>> k[99]
<foo:a-b-c:99>

next() always returns the next key in a sequence of integers, starting with 0 and continuing from the highest previously created tag/Key:

>>> next(k)
<foo:a-b-c:100>
# Same

A Key is callable, with any value that has a str representation:

>>> k()
<foo:a-b-c:101>
# Same as item-access syntax
>>> k("baz")
<foo:a-b-c:baz>

The attributes last and generated allow to inspect one or all of the keys that have been derived from an original:

>>> k.last
<foo:a-b-c:baz>
>>> k.generated
(<foo:a-b-c:0>,
 <foo:a-b-c:1>,
 <foo:a-b-c:bar>,
 <foo:a-b-c:99>,
 <foo:a-b-c:100>,
 <foo:a-b-c:101>,
 <foo:a-b-c:baz>)

add_tag(tag: str | None) → Key[source]¶: Return a new Key with tag appended.

append(*dims: str) → Key[source]¶: Return a new Key with additional dimensions dims.

classmethod bare_name(value) → str | None[source]¶: If value is a bare name (no dims or tags), return it; else None.

property dims: tuple[str, ...]¶: Dimensions of the quantity, tuple of str.

drop(*dims: str | bool) → Key[source]¶: Return a new Key with dims dropped.

drop_all() → Key[source]¶: Return a new Key with all dimensions dropped / zero dimensions.

classmethod from_str_or_key(value: str | Key | AnyQuantity, drop: Iterable[str] | bool = [], append: Iterable[str] = [], tag: str | None = None) → Key[source]¶

Return a new Key from value.

Changed in version 1.18.0: Calling from_str_or_key() with a single argument is no longer necessary; simply give the same value as an argument to Key.

The class method is retained for convenience when calling with multiple arguments. However, the following are equivalent and may be more readable:

k1 = Key("foo:a-b-c:t1", drop="b", append="d", tag="t2")
k2 = Key("foo:a-b-c:t1").drop("b").append("d)"

Parameters:

value (str or Key) – Value to use to generate a new Key.
drop (list of str or True, optional) – Existing dimensions of value to drop. See drop().
append (list of str, optional) – New dimensions to append to the returned Key. See append().
tag (str, optional) – Tag for returned Key. If value has a tag, the two are joined using a ‘+’ character. See add_tag().

Return type:

Key

iter_sums() → Generator[tuple[Key, Callable, Key], None, None][source]¶: Generate (key, task) for all possible partial sums of the Key.

property name: str¶: Name of the quantity, str.

classmethod product(new_name: str, *keys, tag: str | None = None) → Key[source]¶

Return a new Key that has the union of dimensions on keys.

Dimensions are ordered by their first appearance:

First, the dimensions of the first of the keys.
Next, any additional dimensions in the second of the keys that were not already added in step 1.
etc.

Parameters:

new_name (str) – Name for the new Key. The names of keys are discarded.
keys – May include instances of Key, str (converted to Key), or Quantity (the dimensions of the quantity are used directly).

remove_tag(*tags: str) → Key[source]¶

Return a key with any of tags dropped.

Raises:: ValueError – If none of tags are in tags.

rename(name: str) → Key[source]¶: Return a Key with a replaced name.

property sorted: Key¶: A version of the Key with its dims sorted().

property tag: str | None¶: Quantity tag, str or None.

class genno.Keys(**kwargs: Key | str)[source]¶

A collection of Key.

This is essentially the same as types.SimpleNamespace, except every attribute is a Key.

>>> k = Keys(foo="X:a-b-c-d-e-f", bar="Y:a-b-c:long+sequence+of+tags")
>>> k.baz = "Z:a-b-c-e-f"

class genno.KeySeq(*args, **kwargs)[source]¶

Utility class for generating similar Keys.

Note

As of genno 1.28.0, Key provides most of the conveniences and shorthand that were previously provided by KeySeq. User could should prefer use of Key and Keys. KeySeq may eventually be deprecated and removed.

KeySeq supports several ways to create related keys starting from a base key:

>>> ks = KeySeq("foo:x-y-z:bar")

Access the most recently generated item:
```
>>> ks.prev
<foo:x-y-z:bar+7>
```

Access the base Key or its properties:

>>> ks.base
<foo:x-y-z:bar>
>>> ks.name
"foo"

Access a dict of all previously-created keys. Because dict is order-preserving, the order of keys and values reflects the order in which they were created:
```
>>> tuple(ks.keys)
("a", "b", 0, 5, 6, "a", 7)
```

The same Python arithmetic operators usable with Key are usable with KeySeq; they return a new KeySeq with a different base:

>>> ks * "w"
<KeySeq from 'foo:x-y-z-w:bar'>
>>> ks / ("x", "z")
<KeySeq from 'foo:z:bar'>

property base: Key¶: The base Key.

property dims: tuple[str, ...]¶: Dimensions of the base Key.

property keys: MappingProxyType¶

Read-only view of previously-created Keys.

In the form of a dict mapping tags (int or str) to Key values.

property name: str¶: Name of the base Key.

property prev: Key¶: Alias of KeyGeneratorMixin.last.

property tag: str | None¶: Tag of the base Key.

genno.Quantity¶: alias of AttrSeries

The Quantity constructor converts its arguments to an internal, xarray.DataArray-like data format:

# Existing data
data = pd.Series(...)

# Convert to a Quantity for use in genno calculations
qty = Quantity(data, name="Quantity name", units="kg")
c.add("new_qty", qty)

Common genno usage, e.g. in message_ix, creates large, sparse data frames (billions of possible elements, but <1% populated); DataArray’s default, ‘dense’ storage format would be too large for available memory.

Currently, Quantity implemented as is AttrSeries, a wrapped pandas.Series that behaves like a DataArray.
In the future, genno will use SparseDataArray, and eventually directly DataArray backed by sparse data.

The goal is that all genno-based code, including built-in and user functions, can treat quantity arguments as if they were DataArray.

Quantity has a units attribute, which can be set using either str or pint.Unit.

Quantity supports the standard binary operations with unit-aware behaviour: __add__(), __radd__(), __mul__(), __rmul__(), __pow__(), __rpow__(), __sub__(), __radd__(), __truediv__(), and __rtruediv__(). This means that correct units are derived from the units of operands and attached to the resulting Quantity.

Quantity has the following methods and attributes that exactly mirror the signatures and types of the corresponding xarray.DataArray items.

`assign_coords`([coords])	Like `xarray.DataArray.assign_coords()`.
`bfill`(dim[, limit])	Like `xarray.DataArray.bfill()`.
`clip`([min, max, keep_attrs])	Like `xarray.DataArray.clip()`.
`coords`	Like `xarray.DataArray.coords`.
`cumprod`([dim, axis, skipna])	Like `xarray.DataArray.cumprod()`.
`data`
`dims`	Like `xarray.DataArray.dims`.
`drop`(label)	Like `xarray.DataArray.drop()`.
`drop_vars`(names, *[, errors])	Like `xarray.DataArray.drop_vars()`.
`expand_dims`([dim, axis, ...])	Like `xarray.DataArray.expand_dims()`.
`ffill`(dim[, limit])	Like `xarray.DataArray.ffill()`.
`interp`([coords, method, assume_sorted, kwargs])	Like `xarray.DataArray.interp()`.
`item`(*args)	Like `xarray.DataArray.item()`.
`rename`([new_name_or_name_dict])	Like `xarray.DataArray.rename()`.
`sel`([indexers, method, tolerance, drop])	Like `xarray.DataArray.sel()`.
`shape`	Like `xarray.DataArray.shape`.
`shift`([shifts, fill_value])	Like `xarray.DataArray.shift()`.
`squeeze`([dim, drop, axis])	Like `xarray.DataArray.squeeze()`.
`sum`([dim, skipna, min_count, keep_attrs])	Like `xarray.DataArray.sum()`.
`to_dataframe`([name, dim_order])	Like `xarray.DataArray.to_dataframe()`.
`to_series`()	Like `xarray.DataArray.to_series()`.
`transpose`(*dims)	Like `xarray.DataArray.transpose()`.
`where`(cond[, other, drop, axis, inplace])	Like `xarray.DataArray.where()`.

genno.configure(path: Path | str | None = None, **config)[source]

Configure genno globally.

Modifies global variables that affect the behaviour of all Computers and operators. Configuration keys loaded from file are superseded by keyword arguments. Messages are logged at level logging.INFO if config contains unhandled sections.

Parameters:

path (pathlib.Path, optional) – Path to a configuration file in JSON or YAML format.
**config – Configuration keys/sections and values.

exception genno.ComputationError(exc)[source]¶

Wrapper to print intelligible exception information for Computer.get().

In order to aid in debugging, this helper:

Omits the parts of the stack trace that are internal to Dask, and
Gives the key in the Computer.graph and the computation/task that caused the exception.

exception genno.KeyExistsError[source]¶: Raised by Computer.add() when the target key exists.

exception genno.MissingKeyError[source]¶: Raised by Computer.add() when a required input key is missing.

class genno.Operator[source]¶

Base class for a callable with convenience methods.

Example

>>> from genno import Operator
>>>
>>> @Operator.define()
... def myfunc(q1: Quantity, q2: Quantity) -> Quantity:
...     # Operator code
>>>
>>> @myfunc.helper
... def add_myfunc(f, computer, *args, **kwargs):
...     # Custom code to add tasks to `computer`
...     # Perform checks or handle `args` and `kwargs`.

Or:

>>> from genno import Operator
>>>
>>> def add_myfunc(f, computer, *args, **kwargs):
...     # ... as above
>>>
>>> @Operator.define(helper=add_myfunc)
... def myfunc(q1: Quantity, q2: Quantity) -> Quantity:
...     # ... as above

add_tasks(c: Computer, *args, **kwargs) → tuple[Key | str, ...][source]¶: Invoke _add_task to add tasks to c.

static define(deprecated_func_arg: Callable | None = None, *, helper: Callable | None = None) → Callable[[Callable], Operator][source]¶

Return a decorator that wraps func in a Operator instance.

Parameters:: helper (Callable, optional) – Equivalent to calling helper() on the Operator instance.

func: ClassVar[Callable]¶: Function or callable for the Operator.

helper(func: Callable[[...], Key | str | tuple[Key | str, ...]]) → Callable[source]¶: Register func as the convenience method for adding task(s).

genno.assert_quantity(*args)[source]¶

Assert that each of args is a Quantity object.

Raises:: TypeError – with a indicative message.

genno.get_class() → Type[AttrSeries | SparseDataArray][source]¶

Get the current Quantity implementation in use.

Returns one of the classes AttrSeries or SparseDataArray.

class genno.literal(data)[source]¶: A small serializable object to wrap literal values without copying

genno.quote(x)[source]¶

Ensure that this value remains this value in a dask graph

Some values in dask graph take on special meaning. Sometimes we want to ensure that our data is not interpreted but remains literal.

>>> add = lambda x, y: x + y
>>> quote((add, 1, 2))
(literal<type=tuple>,)

genno.set_class(name: Literal['AttrSeries', 'SparseDataArray'] = 'AttrSeries') → Type[AttrSeries | SparseDataArray][source]¶

Set the Quantity implementation to be used.

This also updates genno.Quantity and genno.quantity.Quantity to refer to the selected class. It does not update previously-imported references to one class or the other; code that uses set_class() should refer to one of those two locations:

import genno
from genno import Quantity  # AttrSeries, by default

Quantity()        # AttrSeries
genno.Quantity()  # AttrSeries

genno.set_class("SparseDataArray")

Quantity()        # AttrSeries
genno.Quantity()  # SparseDataArray

Another approach is to update the local reference with the return value of the function:

from genno import Quantity, set_class

Quantity()  # AttrSeries

Quantity = set_class("SparseDataArray")

Quantity()  # SparseDataArray

In code that does not use set_class(), from genno import Quantity is safe.