lingvo.core.nested_map module

NestedMap dict structure.

lingvo.core.nested_map._FromNestedDict(x: Mapping[str, Any]) → NestedMap

lingvo.core.nested_map._FromNestedDict(x: T) → T

Converts every dict in nested structure ‘x’ to a NestedMap.

class lingvo.core.nested_map.NestedMap(*args, **kwargs)

Bases: Dict[str, Any]

A simple helper to maintain a dict.

It is a sub-class of dict with the following extensions/restrictions:

• It supports attr access to its members (see examples below).

• Member keys have to be valid identifiers.

E.g.:

>>> foo = NestedMap()
>>> foo['x'] = 10
>>> foo.y = 20
>>> assert foo.x * 2 == foo.y

_HAS_DYNAMIC_ATTRIBUTES = True

_RESERVED_KEYS = frozenset({'__class__', '__class_getitem__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__ior__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__or__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__ror__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'})

_DELETE = <object object>

copy() → a shallow copy of D

DeepCopy() → NestedMapT

Deep-copies the structure but not the leaf objects.

static FromNestedDict(x)

Converts every dict in nested structure ‘x’ to a NestedMap.

ToNestedDict() → Mapping[str, Any]

Converts every NestedMap in nested structure to a ‘dict’ instance.

This is relevant for code that checks a dictionary’s exact type, instead of isinstance (e.g. parts of tf.data). In those cases, we need a ‘dict’ object, not NestedMap.

Returns

‘dict’ instance where nested NestedMaps are also converted to ‘dict’.

static CheckKey(key: str) → None

Asserts that key is valid NestedMap key.

static SquareBracketIndex(key: str) → Tuple[str, Optional[int]]

Extracts the name and the index from the indexed key (e.g., k[0]).

GetItem(key: str) → Any

Gets the value for the nested key.

Names with underscores will be considered as one key.

Parameters

key – str of the form ([A-Za-z_][A-Za-z0-9_]*)(.[A-Za-z_][A-Za-z0-9_]*)*..

Returns

The value for the given nested key.

Raises

• KeyError – if a key is not present.

• IndexError – when an intermediate item is a list and we try to access an element which is out of range.

• TypeError – when an intermediate item is a list and we try to access an element of it with a string.

Get(key: str, default: Optional[Any] = None) → Any

Gets the value for nested key, returns default if key does not exist.

Names with underscores will be considered as one key.

Parameters

• key – str of the form ([A-Za-z_][A-Za-z0-9_]*)(.[A-Za-z_][A-Za-z0-9_]*)*..

• default – Optional default value, defaults to None.

Returns

The value for the given nested key or default if the key does not exist.

Set(key: str, value: Any) → None

Sets the value for a nested key.

There is limited support for indexing lists when square bracket indexing is used, e.g., key[0], key[1], etc. Names with underscores will be considered as one key. When key[idx] is set, all of the values with indices before idx must be already set. E.g., setting key=’a[2]’ to value=42 when key=’a’ wasn’t referenced before will throw a ValueError. Setting key=’a[0]’ will not.

Parameters

• key – str of the form key_part1.key_part2…key_partN where each key_part is of the form [A-Za-z_][A-Za-z0-9_]* or [A-Za-z_][A-Za-z0-9_]*[d+]

• value – The value to insert.

Raises

• ValueError if a sub key is not a NestedMap or dict or idx > list length –

• for key='key[idx]'. –

_RecursiveMap(fn: Callable[[str, Any], Any], flatten: Literal[False] = False) → NestedMapT

_RecursiveMap(fn: Callable[[str, Any], Any], flatten: Literal[True]) → List[Any]

Traverse recursively into lists, dicts, and NestedMaps applying fn.

Parameters

• fn – The function to apply to each item (leaf node).

• flatten – If true, the result should be a single flat list. Otherwise the result will have the same structure as this NestedMap.

Returns

The result of applying fn.

Flatten() → List[Any]

Returns a list containing the flattened values in the NestedMap.

Unlike py_utils.Flatten(), this will only descend into lists, dicts, and NestedMaps and not tuples, or namedtuples.

FlattenItems() → List[Tuple[Any, Any]]

Flatten the NestedMap and returns <key, value> pairs in a list.

Returns

A list of <key, value> pairs, where keys for nested entries will be represented in the form of foo.bar[10].baz.

Pack(lst: Sequence[Any]) → NestedMapT

Returns a copy of this with each value replaced by a value in lst.

Transform(fn: Callable[[Any], Any]) → NestedMapT

Returns a copy of this NestedMap with fn applied on each value.

TransformWithKey(fn: Callable[[str, Any], Any]) → NestedMapT

Returns a copy of this NestedMap with fn applied on each key/value.

IsCompatible(other: NestedMap) → bool

Returns true if self and other are compatible.

If x and y are two compatible NestedMap, x.Pack(y.Flatten()) produces y and vice versa.

Parameters

other – Another NestedMap.

Filter(fn: Callable[[Any], bool]) → NestedMapT

Returns a copy with entries where fn(entry) is True.

FilterKeyVal(fn: Callable[[str, Any], bool]) → NestedMapT

Returns a copy of this NestedMap filtered by fn.

If fn(key, entry) is True, the entry is copied into the returned NestedMap. Otherwise, it is not copied.

Parameters

fn – a callable of (string, entry)->boolean.

Returns

A NestedMap contains copied entries from this '.NestedMap.

_ToStrings() → List[str]

Returns debug strings in a list for this NestedMap.

DebugString() → str

Returns a debug string for this NestedMap.

VLog(level: Optional[int] = None, prefix: Optional[str] = None) → None

Logs the debug string at the level.