Utilities and compatibility¶
Compatibility with Dask
.
- genno.compat.dask.cull(dsk, keys)[source]¶
Like
dask.optimization.cull()
.This version calls
to_keylike()
on the culled graph and dependencies to ensure the resulting graph does not containKey
.
- genno.compat.dask.to_keylike(value)[source]¶
Rewrite
Key
value (or in value) tostr
.Collections such as
tuple
andlist
are rewritten recursively.
- class genno.compat.graphviz.Visualizer(data_attributes: Mapping, function_attributes: Mapping, graph_attr: Mapping, node_attr: Mapping, edge_attr: Mapping, kwargs: Mapping)[source]¶
Handle arguments for
visualize()
.- add_node(kind: Literal['data', 'func'], name: str, k, v=None) None [source]¶
Add a data node to the graph.
- genno.compat.graphviz.unwrap(label: str) str [source]¶
Unwrap any number of paired ‘<’ and ‘>’ at the start/end of label.
These characters cause errors in graphviz/dot.
- genno.compat.graphviz.visualize(dsk: Mapping, filename: str | PathLike | None = None, format: str | None = None, data_attributes: Mapping | None = None, function_attributes: Mapping | None = None, graph_attr: Mapping | None = None, node_attr: Mapping | None = None, edge_attr: Mapping | None = None, collapse_outputs: bool = False, **kwargs)[source]¶
Generate a Graphviz visualization of dsk.
This is merged and extended version of
dask.base.visualize()
,dask.dot.dot_graph()
, anddask.dot.to_graphviz()
that produces informative output for genno graphs.- Parameters:
dsk – The graph to display.
filename (
pathlib.Path
orstr
, optional) – The name of the file to write to disk. If the file name does not have a suffix, “.png” is used by default. If filename isNone
, no file is written, and dask communicates with dot using only pipes.format (
{'png', 'pdf', 'dot', 'svg', 'jpeg', 'jpg'}
, optional) – Format in which to write output file, if not given by the suffix of filename. Default “png”.data_attributes – Graphviz attributes to apply to single nodes representing keys, in addition to node_attr.
function_attributes – Graphviz attributes to apply to single nodes representing operations or functions, in addition to node_attr.
graph_attr – Mapping of (attribute, value) pairs for the graph. Passed directly to
graphviz.Digraph
.node_attr – Mapping of (attribute, value) pairs set for all nodes. Passed directly to
graphviz.Digraph
.edge_attr – Mapping of (attribute, value) pairs set for all edges. Passed directly to
graphviz.Digraph
.collapse_outputs (
bool
, optional) – Omit nodes for keys that are the output of intermediate calculations.kwargs – All other keyword arguments are added to graph_attr.
Examples
Prepare a computer:
>>> from genno import Computer >>> from genno.testing import add_test_data >>> c = Computer() >>> add_test_data(c) >>> c.add_product("z", "x:t", "x:y") >>> c.add("y::0", itemgetter(0), "y") >>> c.add("y0", "y::0") >>> c.add("index_to", "z::indexed", "z:y", "y::0") >>> c.add_single("all", ["z::indexed", "t", "config", "x:t"])
Visualize its contents:
>>> c.visualize("example.svg")
This produces the output:
See also
- genno.compat.pandas.disable_copy_on_write(name)[source]¶
Context manager to disable Pandas Copy-on-Write (CoW).
A message is logged with level
logging.DEBUG
if the setting is changed.
- genno.compat.pandas.handles_parquet_attrs() bool [source]¶
Return
True
ifpandas
can read/write attrs to/from Parquet files.If not, a message is logged.
- genno.core.describe.MAX_ITEM_LENGTH = 160¶
Default maximum length for outputs from
describe_recursive()
.
- genno.core.describe.describe_recursive(graph, comp, depth=0, seen=None)[source]¶
Recursive helper for
describe()
.
- genno.core.describe.is_list_of_keys(arg: Any, graph: Mapping) bool [source]¶
Identify a task which is a list of other keys.
- genno.core.describe.label(arg, max_length=160) str [source]¶
Return a label for arg.
The label depends on the type of arg:
xarray.DataArray
: the first line of the string representation.functools.partial()
object: a less-verbose version that omits None arguments.Item protected with
dask.core.quote()
: its literal value.A callable, e.g. a function: its name.
Anything else: its
str
representation.
In all cases, the string is no longer than max_length.
- class genno.core.graph.Graph(*args, **kwargs)[source]¶
A dictionary for a graph indexed by
Key
.Graph maintains indexes on set/delete/pop/update operations that allow for fast lookups/member checks in certain special cases:
unsorted_key
(key)Return key with its original or unsorted dimensions.
full_key
(name_or_key)Return name_or_key with its full dimensions.
These basic features are used to provide higher-level helpers for
Computer
:infer
(key[, dims])Infer a key.
- full_key(name_or_key: Key | str) Key | str | None [source]¶
Return name_or_key with its full dimensions.
- infer(key: str | Key, dims: Iterable[str] = []) Key | str | None [source]¶
Infer a key.
- Parameters:
dims (
list
ofstr
, optional) – Drop all but these dimensions from the returned key(s).- Returns:
str
– If key is not found in the Graph.Key
– key with either its full dimensions (cf.full_key()
) or, if dims are given, with only these dims.
- pop(*args)[source]¶
Overload
dict.pop()
to also call_deindex()
.
- unsorted_key(key: Key | str) Key | str | None [source]¶
Return key with its original or unsorted dimensions.
- update(arg=None, **kwargs)[source]¶
Overload
dict.pop()
to also call_index()
.
- genno.core.key.iter_keys(value: Key | str | tuple[Key | str, ...]) Iterator[Key] [source]¶
Yield
Keys
from value.See also
- genno.core.key.single_key(value: Key | str | tuple[Key | str, ...] | Iterator) Key [source]¶
Ensure value is a single
Key
.See also
- genno.util.REPLACE_UNITS = {'%': 'percent'}¶
Replacements to apply to Quantity units before parsing by pint. Mapping from original unit -> preferred unit.
The default values include:
The ‘%’ symbol cannot be supported by pint, because it is a Python operator; it is replaced with “percent”.
Additional values can be added with
configure()
; see units:.
- genno.util.clean_units(input_string)[source]¶
Tolerate messy strings for units.
Dimensions enclosed in “[]” have these characters stripped.
Replacements from
REPLACE_UNITS
are applied.
- genno.util.filter_concat_args(args)[source]¶
Filter out str and Key from args.
A warning is logged for each element removed.
- genno.util.free_parameters(func: Callable) Mapping [source]¶
Retrieve information on the free parameters of func.
Identical to
inspect.signature(func).parameters
; that is, toinspect.Signature.parameters
.free_parameters
also:Handles functions that have been
functools.partial()
’d, returning only the parameters that have not already been assigned a value by thepartial()
call—the “free” parameters.Caches return values for better performance.
- genno.util.parse_units(data: Iterable, registry=None) pint.Unit [source]¶
Return a
pint.Unit
for an iterable of strings.Valid unit expressions not already present in the registry are defined, e.g.:
u = parse_units(["foo/bar", "foo/bar"], reg)
…results in the addition of unit definitions equivalent to:
reg.define("foo = [foo]") reg.define("bar = [bar]") u = reg.foo / reg.bar
- Raises:
ValueError – if data contains more than 1 unit expression, or the unit expression contains characters not parseable by
pint
, e.g.-?$
.
- genno.util.partial_split(func: Callable, kwargs: Mapping) tuple[Callable, MutableMapping] [source]¶
Forgiving version of
functools.partial()
.Returns a partial object and leftover keyword arguments that are not applicable to func.