Utilities and compatibility

Compatibility with Dask.

genno.compat.dask.cull(dsk, keys)[source]

Like dask.optimization.cull().

This version calls to_keylike() on the culled graph and dependencies to ensure the resulting graph does not contain Key.

genno.compat.dask.to_keylike(value)[source]

Rewrite Key value (or in value) to str.

Collections such as tuple and list are rewritten recursively.

class genno.compat.graphviz.Visualizer(data_attributes: Mapping, function_attributes: Mapping, graph_attr: Mapping, node_attr: Mapping, edge_attr: Mapping, kwargs: Mapping)[source]

Handle arguments for visualize().

add_edge(a, b) None[source]

Add an edge to the graph.

add_node(kind: Literal['data', 'func'], name: str, k, v=None) None[source]

Add a data node to the graph.

get_attrs(kind: Literal['data', 'func'], name: str, **defaults) dict[source]

Prepare attributes for a node of kind.

If name is in self.da or self.fa, use those values, filling with defaults; otherwise, attributes are empty except for defaults.

process(dsk: Mapping, collapse_outputs: bool)[source]

Process the dask graph dsk.

genno.compat.graphviz.unwrap(label: str) str[source]

Unwrap any number of paired ‘<’ and ‘>’ at the start/end of label.

These characters cause errors in graphviz/dot.

genno.compat.graphviz.visualize(dsk: Mapping, filename: str | PathLike | None = None, format: str | None = None, data_attributes: Mapping | None = None, function_attributes: Mapping | None = None, graph_attr: Mapping | None = None, node_attr: Mapping | None = None, edge_attr: Mapping | None = None, collapse_outputs: bool = False, **kwargs)[source]

Generate a Graphviz visualization of dsk.

This is merged and extended version of dask.base.visualize(), dask.dot.dot_graph(), and dask.dot.to_graphviz() that produces informative output for genno graphs.

Parameters:
  • dsk – The graph to display.

  • filename (pathlib.Path or str, optional) – The name of the file to write to disk. If the file name does not have a suffix, “.png” is used by default. If filename is None, no file is written, and dask communicates with dot using only pipes.

  • format ({'png', 'pdf', 'dot', 'svg', 'jpeg', 'jpg'}, optional) – Format in which to write output file, if not given by the suffix of filename. Default “png”.

  • data_attributes – Graphviz attributes to apply to single nodes representing keys, in addition to node_attr.

  • function_attributes – Graphviz attributes to apply to single nodes representing operations or functions, in addition to node_attr.

  • graph_attr – Mapping of (attribute, value) pairs for the graph. Passed directly to graphviz.Digraph.

  • node_attr – Mapping of (attribute, value) pairs set for all nodes. Passed directly to graphviz.Digraph.

  • edge_attr – Mapping of (attribute, value) pairs set for all edges. Passed directly to graphviz.Digraph.

  • collapse_outputs (bool, optional) – Omit nodes for keys that are the output of intermediate calculations.

  • kwargs – All other keyword arguments are added to graph_attr.

Examples

Prepare a computer:

>>> from genno import Computer
>>> from genno.testing import add_test_data
>>> c = Computer()
>>> add_test_data(c)
>>> c.add_product("z", "x:t", "x:y")
>>> c.add("y::0", itemgetter(0), "y")
>>> c.add("y0", "y::0")
>>> c.add("index_to", "z::indexed", "z:y", "y::0")
>>> c.add_single("all", ["z::indexed", "t", "config", "x:t"])

Visualize its contents:

>>> c.visualize("example.svg")

This produces the output:

Example output from graphviz.visualize.

See also

describe.label

genno.compat.pandas.disable_copy_on_write(name)[source]

Context manager to disable Pandas Copy-on-Write (CoW).

A message is logged with level logging.DEBUG if the setting is changed.

genno.compat.pandas.handles_parquet_attrs() bool[source]

Return True if pandas can read/write attrs to/from Parquet files.

If not, a message is logged.

genno.core.describe.MAX_ITEM_LENGTH = 160

Default maximum length for outputs from describe_recursive().

genno.core.describe.describe_recursive(graph, comp, depth=0, seen=None)[source]

Recursive helper for describe().

Parameters:
  • graph – A dask graph.

  • comp – A dask computation.

  • depth (int) – Recursion depth. Used for indentation.

  • seen (set) – Keys that have already been described. Used to avoid double-printing.

genno.core.describe.is_list_of_keys(arg: Any, graph: Mapping) bool[source]

Identify a task which is a list of other keys.

genno.core.describe.label(arg, max_length=160) str[source]

Return a label for arg.

The label depends on the type of arg:

  • xarray.DataArray: the first line of the string representation.

  • functools.partial() object: a less-verbose version that omits None arguments.

  • Item protected with dask.core.quote(): its literal value.

  • A callable, e.g. a function: its name.

  • Anything else: its str representation.

In all cases, the string is no longer than max_length.

class genno.core.graph.Graph(*args, **kwargs)[source]

A dictionary for a graph indexed by Key.

Graph maintains indexes on set/delete/pop/update operations that allow for fast lookups/member checks in certain special cases:

unsorted_key(key)

Return key with its original or unsorted dimensions.

full_key(name_or_key)

Return name_or_key with its full dimensions.

These basic features are used to provide higher-level helpers for Computer:

infer(key[, dims])

Infer a key.

full_key(name_or_key: Key | str) Key | str | None[source]

Return name_or_key with its full dimensions.

infer(key: str | Key, dims: Iterable[str] = []) Key | str | None[source]

Infer a key.

Parameters:

dims (list of str, optional) – Drop all but these dimensions from the returned key(s).

Returns:

  • str – If key is not found in the Graph.

  • Keykey with either its full dimensions (cf. full_key()) or, if dims are given, with only these dims.

pop(*args)[source]

Overload dict.pop() to also call _deindex().

unsorted_key(key: Key | str) Key | str | None[source]

Return key with its original or unsorted dimensions.

update(arg=None, **kwargs)[source]

Overload dict.pop() to also call _index().

genno.core.key.KeyLike

Type shorthand for Key or any other value that can be used as a key.

alias of Key | str

genno.core.key.iter_keys(value: Key | str | tuple[Key | str, ...]) Iterator[Key][source]

Yield Keys from value.

Raises:

TypeErrorvalue is not an iterable of Key.

See also

Computer.add

genno.core.key.single_key(value: Key | str | tuple[Key | str, ...] | Iterator) Key[source]

Ensure value is a single Key.

Raises:

TypeErrorvalue is not a Key or 1-tuple of Key.

See also

Computer.add

genno.util.REPLACE_UNITS = {'%': 'percent'}

Replacements to apply to Quantity units before parsing by pint. Mapping from original unit -> preferred unit.

The default values include:

  • The ‘%’ symbol cannot be supported by pint, because it is a Python operator; it is replaced with “percent”.

Additional values can be added with configure(); see units:.

genno.util.clean_units(input_string)[source]

Tolerate messy strings for units.

  • Dimensions enclosed in “[]” have these characters stripped.

  • Replacements from REPLACE_UNITS are applied.

genno.util.collect_units(*args)[source]

Return the “_unit” attributes of the args.

genno.util.filter_concat_args(args)[source]

Filter out str and Key from args.

A warning is logged for each element removed.

genno.util.free_parameters(func: Callable) Mapping[source]

Retrieve information on the free parameters of func.

Identical to inspect.signature(func).parameters; that is, to inspect.Signature.parameters. free_parameters also:

  • Handles functions that have been functools.partial()’d, returning only the parameters that have not already been assigned a value by the partial() call—the “free” parameters.

  • Caches return values for better performance.

genno.util.parse_units(data: Iterable, registry=None) pint.Unit[source]

Return a pint.Unit for an iterable of strings.

Valid unit expressions not already present in the registry are defined, e.g.:

u = parse_units(["foo/bar", "foo/bar"], reg)

…results in the addition of unit definitions equivalent to:

reg.define("foo = [foo]")
reg.define("bar = [bar]")
u = reg.foo / reg.bar
Raises:

ValueError – if data contains more than 1 unit expression, or the unit expression contains characters not parseable by pint, e.g. -?$.

genno.util.partial_split(func: Callable, kwargs: Mapping) tuple[Callable, MutableMapping][source]

Forgiving version of functools.partial().

Returns a partial object and leftover keyword arguments that are not applicable to func.

genno.util.units_with_multiplier(value: str | pint.Unit | pint.Quantity | None) tuple[pint.Unit, float][source]

Separate units and multiplier from .UnitLike.

Returns:

  1. pint.Unit.

  2. float; any multiplier on the units.

Return type:

tuple

genno.util.unquote(value)[source]

Reverse dask.core.quote().