Utilities and compatibility

Utilities and compatibility#

class genno.compat.graphviz.Visualizer(data_attributes: Mapping, function_attributes: Mapping, graph_attr: Mapping, node_attr: Mapping, edge_attr: Mapping, kwargs: Mapping)[source]#

Handle arguments for visualize().

add_edge(a, b) → None[source]#: Add an edge to the graph.

add_node(kind: Literal['data', 'func'], name: str, k, v=None) → None[source]#: Add a data node to the graph.

get_attrs(kind: Literal['data', 'func'], name: str, **defaults) → dict[source]#

Prepare attributes for a node of kind.

If name is in self.da or self.fa, use those values, filling with defaults; otherwise, attributes are empty except for defaults.

process(dsk: Mapping, collapse_outputs: bool)[source]#: Process the dask graph dsk.

genno.compat.graphviz.unwrap(label: str) → str[source]#

Unwrap any number of paired ‘<’ and ‘>’ at the start/end of label.

These characters cause errors in graphviz/dot.

Generate a Graphviz visualization of dsk.

This is merged and extended version of dask.base.visualize(), dask.dot.dot_graph(), and dask.dot.to_graphviz() that produces informative output for genno graphs.

Parameters:

dsk – The graph to display.
filename (pathlib.Path or str, optional) – The name of the file to write to disk. If the file name does not have a suffix, “.png” is used by default. If filename is None, no file is written, and dask communicates with dot using only pipes.
format ({'png', 'pdf', 'dot', 'svg', 'jpeg', 'jpg'}, optional) – Format in which to write output file, if not given by the suffix of filename. Default “png”.
data_attributes – Graphviz attributes to apply to single nodes representing keys, in addition to node_attr.
function_attributes – Graphviz attributes to apply to single nodes representing operations or functions, in addition to node_attr.
graph_attr – Mapping of (attribute, value) pairs for the graph. Passed directly to graphviz.Digraph.
node_attr – Mapping of (attribute, value) pairs set for all nodes. Passed directly to graphviz.Digraph.
edge_attr – Mapping of (attribute, value) pairs set for all edges. Passed directly to graphviz.Digraph.
collapse_outputs (bool, optional) – Omit nodes for keys that are the output of intermediate calculations.
kwargs – All other keyword arguments are added to graph_attr.

Examples

Prepare a computer:

>>> from genno import Computer
>>> from genno.testing import add_test_data
>>> c = Computer()
>>> add_test_data(c)
>>> c.add_product("z", "x:t", "x:y")
>>> c.add("y::0", itemgetter(0), "y")
>>> c.add("y0", "y::0")
>>> c.add("index_to", "z::indexed", "z:y", "y::0")
>>> c.add_single("all", ["z::indexed", "t", "config", "x:t"])

Visualize its contents:

>>> c.visualize("example.svg")

This produces the output:

See also

describe.label

genno.compat.pandas.disable_copy_on_write(name)[source]#

Context manager to disable Pandas Copy-on-Write (CoW).

A message is logged with level logging.DEBUG if the setting is changed.

genno.compat.pandas.handles_parquet_attrs() → bool[source]#

Return True if pandas can read/write attrs to/from Parquet files.

If not, a message is logged.

genno.core.describe.MAX_ITEM_LENGTH = 160#: Default maximum length for outputs from describe_recursive().

genno.core.describe.describe_recursive(graph, comp, depth=0, seen=None)[source]#

Recursive helper for describe().

Parameters:

graph – A dask graph.
comp – A dask computation.
depth (int) – Recursion depth. Used for indentation.
seen (set) – Keys that have already been described. Used to avoid double-printing.

genno.core.describe.is_list_of_keys(arg: Any, graph: Mapping) → bool[source]#: Identify a task which is a list of other keys.

genno.core.describe.label(arg, max_length=160) → str[source]#

Return a label for arg.

The label depends on the type of arg:

xarray.DataArray: the first line of the string representation.
functools.partial() object: a less-verbose version that omits None arguments.
Item protected with dask.core.quote(): its literal value.
A callable, e.g. a function: its name.
Anything else: its str representation.

In all cases, the string is no longer than max_length.

class genno.core.graph.Graph(*args, **kwargs)[source]#

A dictionary for a graph indexed by Key.

Graph maintains indexes on set/delete/pop/update operations that allow for fast lookups/member checks in certain special cases:

`unsorted_key`(key)	Return key with its original or unsorted dimensions.
`full_key`(name_or_key)	Return name_or_key with its full dimensions.

These basic features are used to provide higher-level helpers for Computer:

infer(key[, dims])

Infer a key.

full_key(name_or_key: Key | str) → Key | str | None[source]#: Return name_or_key with its full dimensions.

infer(key: str | Key, dims: Iterable[str] = []) → Key | str | None[source]#

Infer a key.

Parameters:

dims (list of str, optional) – Drop all but these dimensions from the returned key(s).

Returns:

str – If key is not found in the Graph.
Key – key with either its full dimensions (cf. full_key()) or, if dims are given, with only these dims.

pop(*args)[source]#: Overload dict.pop() to also call _deindex().

unsorted_key(key: Key | str) → Key | str | None[source]#: Return key with its original or unsorted dimensions.

update(arg=None, **kwargs)[source]#: Overload dict.pop() to also call _index().

genno.core.key.KeyLike#

Type shorthand for Key or any other value that can be used as a key.

alias of Key | str

genno.core.key.iter_keys(value: Key | str | Tuple[Key | str, ...]) → Iterator[Key][source]#

Yield Keys from value.

Raises:: TypeError – value is not an iterable of Key.

See also

Computer.add

genno.util.REPLACE_UNITS = {'%': 'percent'}#

Replacements to apply to Quantity units before parsing by pint. Mapping from original unit -> preferred unit.

The default values include:

The ‘%’ symbol cannot be supported by pint, because it is a Python operator; it is replaced with “percent”.

Additional values can be added with configure(); see units:.

genno.util.clean_units(input_string)[source]#

Tolerate messy strings for units.

Dimensions enclosed in “[]” have these characters stripped.
Replacements from REPLACE_UNITS are applied.

genno.util.collect_units(*args)[source]#: Return the “_unit” attributes of the args.

genno.util.filter_concat_args(args)[source]#

Filter out str and Key from args.

A warning is logged for each element removed.

genno.util.free_parameters(func: Callable) → Mapping[source]#

Retrieve information on the free parameters of func.

Identical to inspect.signature(func).parameters; that is, to inspect.Signature.parameters. free_parameters also:

Handles functions that have been functools.partial()’d, returning only the parameters that have not already been assigned a value by the partial() call—the “free” parameters.
Caches return values for better performance.

genno.util.parse_units(data: Iterable, registry=None) → pint.Unit[source]#

Return a pint.Unit for an iterable of strings.

Valid unit expressions not already present in the registry are defined, e.g.:

u = parse_units(["foo/bar", "foo/bar"], reg)

…results in the addition of unit definitions equivalent to:

reg.define("foo = [foo]")
reg.define("bar = [bar]")
u = reg.foo / reg.bar

Raises:: ValueError – if data contains more than 1 unit expression, or the unit expression contains characters not parseable by pint, e.g. -?$.

genno.util.partial_split(func: Callable, kwargs: Mapping) → Tuple[Callable, MutableMapping][source]#

Forgiving version of functools.partial().

Returns a partial object and leftover keyword arguments that are not applicable to func.

genno.util.units_with_multiplier(value: str | pint.Unit | pint.Quantity | None) → Tuple[pint.Unit, float][source]#

Separate units and multiplier from .UnitLike.

Returns:

pint.Unit.
float; any multiplier on the units.

Return type:

tuple

genno.util.unquote(value)[source]#: Reverse dask.core.quote().

Utilities and compatibility

Contents

Utilities and compatibility#