Utilities and compatibility

Compatibility with Dask.

genno.compat.dask.cull(dsk, keys)

Like dask.optimization.cull().

This version calls to_keylike() on the culled graph and dependencies to ensure the resulting graph does not contain Key.

genno.compat.dask.to_keylike(value)

Rewrite Key value (or in value) to str.

Collections such as tuple and list are rewritten recursively.

class genno.compat.graphviz.Visualizer(data_attributes: Mapping, function_attributes: Mapping, graph_attr: Mapping, node_attr: Mapping, edge_attr: Mapping, kwargs: Mapping)

Handle arguments for visualize().

add_edge(a, b) → None

Add an edge to the graph.

add_node(kind: Literal['data', 'func'], name: str, k, v=None) → None

Add a data node to the graph.

get_attrs(kind: Literal['data', 'func'], name: str, **defaults) → dict

Prepare attributes for a node of kind.

If name is in self.da or self.fa, use those values, filling with defaults; otherwise, attributes are empty except for defaults.

process(dsk: Mapping, collapse_outputs: bool)

Process the dask graph dsk.

genno.compat.graphviz.unwrap(label: str) → str

Unwrap any number of paired ‘<’ and ‘>’ at the start/end of label.

These characters cause errors in graphviz/dot.

genno.compat.graphviz.visualize(dsk: Mapping, filename: str | PathLike | None = None, format: str | None = None, data_attributes: Mapping | None = None, function_attributes: Mapping | None = None, graph_attr: Mapping | None = None, node_attr: Mapping | None = None, edge_attr: Mapping | None = None, collapse_outputs: bool = False, **kwargs)

Generate a Graphviz visualization of dsk.

This is merged and extended version of dask.base.visualize(), dask.dot.dot_graph(), and dask.dot.to_graphviz() that produces informative output for genno graphs.

Parameters:

• dsk – The graph to display.

• filename (pathlib.Path or str, optional) – The name of the file to write to disk. If the file name does not have a suffix, “.png” is used by default. If filename is None, no file is written, and dask communicates with dot using only pipes.

• format ({'png', 'pdf', 'dot', 'svg', 'jpeg', 'jpg'}, optional) – Format in which to write output file, if not given by the suffix of filename. Default “png”.

• data_attributes – Graphviz attributes to apply to single nodes representing keys, in addition to node_attr.

• function_attributes – Graphviz attributes to apply to single nodes representing operations or functions, in addition to node_attr.

• graph_attr – Mapping of (attribute, value) pairs for the graph. Passed directly to graphviz.Digraph.

• node_attr – Mapping of (attribute, value) pairs set for all nodes. Passed directly to graphviz.Digraph.

• edge_attr – Mapping of (attribute, value) pairs set for all edges. Passed directly to graphviz.Digraph.

• collapse_outputs (bool, optional) – Omit nodes for keys that are the output of intermediate calculations.

• kwargs – All other keyword arguments are added to graph_attr.

Examples

Prepare a computer:

>>> from genno import Computer
>>> from genno.testing import add_test_data
>>> c = Computer()
>>> add_test_data(c)
>>> c.add_product("z", "x:t", "x:y")
>>> c.add("y::0", itemgetter(0), "y")
>>> c.add("y0", "y::0")
>>> c.add("index_to", "z::indexed", "z:y", "y::0")
>>> c.add_single("all", ["z::indexed", "t", "config", "x:t"])

Visualize its contents:

>>> c.visualize("example.svg")

This produces the output:

See also

describe.label

genno.compat.pandas.disable_copy_on_write(name: str) → Iterator[None]

Context manager to disable Pandas Copy-on-Write (CoW).

A message is logged with level logging.DEBUG if the setting is changed. The fixture has no effect in pandas 3.0.0 and later, in which the option is always enabled.

genno.compat.pandas.handles_parquet_attrs() → bool

Return True if pandas can read/write attrs to/from Parquet files.

If not, a message is logged.

genno.compat.pandas.manager_classes() → tuple[type, ...]

Pandas class(es) for which a fast-path pd.Series.__init__() can be used.

genno.compat.pandas.version() → Version

Return the version of pandas as a packaging.version.Version.

genno.core.describe.MAX_ITEM_LENGTH = 160

Default maximum length for outputs from describe_recursive().

genno.core.describe.describe_recursive(graph, comp, depth=0, seen=None)

Recursive helper for describe().

Parameters:

• graph – A dask graph.

• comp – A dask computation.

• depth (int) – Recursion depth. Used for indentation.

• seen (set) – Keys that have already been described. Used to avoid double-printing.

genno.core.describe.is_list_of_keys(arg: Any, graph: Mapping) → bool

Identify a task which is a list of other keys.

genno.core.describe.label(arg, max_length=160) → str

Return a label for arg.

The label depends on the type of arg:

• xarray.DataArray: the first line of the string representation.

• functools.partial() object: a less-verbose version that omits None arguments.

• Item protected with dask.core.quote(): its literal value.

• A callable, e.g. a function: its name.

• Anything else: its str representation.

In all cases, the string is no longer than max_length.

class genno.core.graph.Graph(*args, **kwargs)

A dictionary for a graph indexed by Key.

Graph maintains indexes on set/delete/pop/update operations that allow for fast lookups/member checks in certain special cases:

unsorted_key(key)	Return key with its original or unsorted dimensions.
full_key(name_or_key)	Return name_or_key with its full dimensions.

These basic features are used to provide higher-level helpers for Computer:

infer(key[, dims])

Infer a key.

full_key(name_or_key: KeyLike) → KeyLike | None

Return name_or_key with its full dimensions.

infer(key: str | Key, dims: Iterable[str] = []) → KeyLike | None

Infer a key.

Parameters:

dims (list of str, optional) – Drop all but these dimensions from the returned key(s).

Returns:

• str – If key is not found in the Graph.

• Key – key with either its full dimensions (cf. full_key()) or, if dims are given, with only these dims.

pop(*args)

Overload dict.pop() to also call _deindex().

unsorted_key(key: KeyLike) → KeyLike | None

Return key with its original or unsorted dimensions.

update(arg=None, **kwargs)

Overload dict.update() to also call _index().

genno.core.key.KeyLike = genno.core.key.Key | str

Type shorthand for Key or any other value that can be used as a key.

genno.core.key.iter_keys(value: Key | str | tuple[Key | str, ...]) → Iterator[Key]

Yield Keys from value.

Raises:

TypeError – value is not an iterable of Key.

See also

Computer.add

genno.core.key.single_key(value: Key | str | tuple[Key | str, ...] | Iterator) → Key

Ensure value is a single Key.

Raises:

TypeError – value is not a Key or 1-tuple of Key.

See also

Computer.add

Types for hinting.

This module and its contents should usually be imported within an if TYPE_CHECKING block.

class genno.types.TKeyLike

Similar to KeyLike, but as a variable that can be use to match function/method outputs to inputs.

alias of TypeVar(‘TKeyLike’, ~genno.core.key.Key, str)

class genno.types.TQuantity

Similar to .AnyQuantity, but as a variable that can be used to match function /method outputs to inputs.

alias of TypeVar(‘TQuantity’, ~genno.core.attrseries.AttrSeries, ~genno.core.sparsedataarray.SparseDataArray)

class genno.types.Unit(*args, **kwargs)

genno.util.REPLACE_UNITS = {'%': 'percent'}

Replacements to apply to Quantity units before parsing by pint. Mapping from original unit -> preferred unit.

The default values include:

• The ‘%’ symbol cannot be supported by pint, because it is a Python operator; it is replaced with “percent”.

Additional values can be added with configure(); see units:.

genno.util.clean_units(input_string)

Tolerate messy strings for units.

• Dimensions enclosed in “[]” have these characters stripped.

• Replacements from REPLACE_UNITS are applied.

genno.util.collect_units(*args)

Return the “_unit” attributes of the args.

genno.util.filter_concat_args(args)

Filter out str and Key from args.

A warning is logged for each element removed.

genno.util.free_parameters(func: Callable) → Mapping

Retrieve information on the free parameters of func.

Identical to inspect.signature(func).parameters; that is, to inspect.Signature.parameters. free_parameters also:

• Handles functions that have been functools.partial()’d, returning only the parameters that have not already been assigned a value by the partial() call—the “free” parameters.

• Caches return values for better performance.

genno.util.parse_units(data: Iterable, registry=None) → pint.Unit

Return a pint.Unit for an iterable of strings.

Valid unit expressions not already present in the registry are defined, e.g.:

u = parse_units(["foo/bar", "foo/bar"], reg)

…results in the addition of unit definitions equivalent to:

reg.define("foo = [foo]")
reg.define("bar = [bar]")
u = reg.foo / reg.bar

Raises:

ValueError – if data contains more than 1 unit expression, or the unit expression contains characters not parseable by pint, e.g. -?$.

genno.util.partial_split(func: Callable, kwargs: Mapping) → tuple[Callable, MutableMapping]

Forgiving version of functools.partial().

Returns a partial object and leftover keyword arguments that are not applicable to func.

genno.util.units_with_multiplier(value: UnitLike | None) → tuple[pint.Unit, float]

Separate units and multiplier from .UnitLike.

Returns:

1. pint.Unit.

2. float; any multiplier on the units.

Return type:

tuple

genno.util.unquote(value)

Reverse dask.core.quote().

genno.util.update_recursive(base: MutableMapping, other: Mapping) → None

Recursively update base with contents of other.

Contents of dict-like members are merged.