lingvo.tasks.car.builder_lib module

A collection of helper functions to build lingvo layer params.

lingvo.tasks.car.builder_lib.TruncatedGaussianInit(filter_shape)

lingvo.tasks.car.builder_lib.KaimingUniformFanInRelu(shape)

class lingvo.tasks.car.builder_lib.ModelBuilderBase

Bases: object

Model builder with commonly used layers.

_Rep(name, repeat, *subs)

Helper to construct a sequential layer repeated several time.

_Seq(name, *subs)

_Branch(name, body, fetches)

_BN(name, dims)

_Linear(name, idims, odims, params_init=None)

_Bias(name, dims, params_init=None)

_Activation(name, activation_fn)

_Relu(name)

_Swish(name)

_Sigmoid(name)

_FC(name, idims, odims, use_bn=True, activation_fn=None)

Fully connected layer, with optional batch norm.

_MLP(name, dims, use_bn=True, activation_fn=None)

_Map(name, fn, **kwargs)

_Max(name)

_Reshape(name, shape)

_Matmul(name, *subs)

_GLU(name, idims, odims)

_Dropout(name, keep_prob)

_Fetch(name)

_FirstN(name, n)

Return the first n args.

_ArgIdx(name, index)

_Join(name, *subs)

Juxtapose outputs from *subs in a single output tuple.

_Par(name, fn, *subs)

Helper to construct a parallel layer which merge branches.

_Concat(name, *subs)

Concatenate outputs from *subs along the last dimensions.

_BroadcastConcat(name, *subs)

Concatenate outputs from *subs, broadcasting to match leading shape.

_ApplyFnMulti(name, fn, *subs)

A common use case where every branch produces a single output tensor.

_ApplyInParallelAndMerge(name, merge_fn, *subs)

Applies the subs in parallel to the given input and merges their outputs.

Parameters

• name – String layer name.

• merge_fn – Function to merge the outputs of subs, which will be a flattened list of subs outputs. For example, if there were 3 subs with outputs (out1,), (out21, out22), (out31, out32), the output will be [out1, out21, out22, out31, out32].

• *subs – Modules to be applied in parallel to the input

Returns

Params for this layer.

_ApplyFn(name, fn)

Apply fn over the input tuple.

_MakeInputFeatureFromPoints(name)

Transforms points to features using the points with constant padding.

Layer input/output shapes: [N x P x 3] -> ([N x P x 3], [N x P x 4])

Parameters

name – String layer name.

Returns

Params for this layer.

_Squeeze(name, axis=None)

_ConvPlain(name, filter_shape, filter_stride=(1, 1), padding='SAME', conv_init_method=None)

_DeconvPlain(name, filter_shape, filter_stride=(1, 1))

_Conv(name, filter_shape, stride=(1, 1), padding='SAME', use_bn=True)

Helper to construct a conv/normalize/actvation layer.

_Identity(name)

Apply identity transformation.

_Shortcut(name, idims, odims, stride)

Apply ResNet shortcut transformation.

_ResidualLayer(name, filter_size, stride)

ResNet basic layer, as in https://arxiv.org/pdf/1512.03385.pdf Fig. 2.

Parameters

• name – string layer name.

• filter_size – tuple of integers (filter_height, filter_width, idims, odims) , which represent filter height, filter width, input channel size, output channel size.

• stride – integer of tuple of integers to apply to all dimensions when using 2D convolution. This will be applied to the shortcut layer and the first convolution layer.

Returns

Params for a residual layer.

_ResidualBlock(name, filter_size, stride, repeats)

ResNet block that downsamples at the beginning.

Parameters

• name – string block name.

• filter_size – tuple of integers (filter_height, filter_width, idims, odims) , which represent filter height, filter width, input channel size, output channel size.

• stride – integer of tuple of integers to apply to all dimensions when using 2D convolution. It is applied only to the first downsampling layer for efficiency.

• repeats – integer number of residual layers to stack.

Returns

Params for a residual block.

_SelfAttenStack(name, depth, dims, hdims, heads, keep_prob)

A self-attentional stack.

Parameters

• name – string layer name.

• depth – int. The number of transformer layers.

• dims – int. The dimension of each transformer layer.

• hdims – int. The hidden dimension of each transformer layer. Typically, hdims >= dims.

• heads – int. The number of attention heads.

• keep_prob – Dropout keep probability.

Returns

Params for this layer.

_Atten(name, dims, hdims, heads, keep_prob=1.0)

Transformer self-attention layer.

_LN(name, dims)

Layer norm.

_Project(name, idims, odims)

Project layer. Simply X*W+b.

_Add(name, lhs, rhs)

Add two branches. out = lhs(in) + rhs(in).

_Multiply(name, lhs, rhs)

Multiply two branches. out = lhs(in) * rhs(in).

_AttenFF(name, dims, hdims, keep_prob=1.0)

Transformer feed-forward layer.

_AttenSelf(name, dims, hdims, heads, keep_prob=1.0)

Dot-attention, multiple heads, self-attention.

class _Decorators

Bases: object

Internal decorators for builder functions.

classmethod ExpectsNestedMapTensor(expected_keys=())

Adds a validation layer before the layer produced by builder_fn.

Parameters

expected_keys – A string, or an iterable of strings that are expected to be in the input NestedMap.

Returns

A decorator function that can be applied on builder functions.

classmethod ExpectsNestedMapPointsTensor(builder_fn)

Adds a validation layer before the layer produced by builder_fn.

classmethod ExpectsNestedMapRangeImage(builder_fn)

Adds a validation layer before the layer produced by builder_fn.

_GetValue(*args, **kwargs)

_ParMap(*args, **kwargs)

_SeqToKey(*args, **kwargs)

_SeqOnKey(name, key, *subs)

Sequential layers operating on a specific key only.

_SeqOnFeatures(*args, **kwargs)

_PaddedMax(*args, **kwargs)

_PaddedMean(*args, **kwargs)

_PaddedSum(*args, **kwargs)

_FeaturesFC(*args, **kwargs)

_FeaturesMLP(*args, **kwargs)

_CondFC(name, idims, adims, odims, use_bn=True, activation_fn=None)

Conditional FC layer to use with GIN as a combiner.

([…, P, idims], […, 1, adims]) -> […, P, odims]

This layer expects an input tuple (features, aggregate), where features contains per-point features, and aggregate is a global feature (e.g., computed using max-pooling over all the points). The aggregate tensor is used to compute a linear transformation that is used in this FC layer. Each example in the batch has a different linear transformation. This layer is similar to the T-Net transformation in PointNet.

Parameters

• name – String name for this layer.

• idims – Dimension for last axis of features tensor.

• adims – Dimension for last axis of aggregate tensor.

• odims – Number of output dimensions.

• use_bn – Whether to enable batch norm.

• activation_fn – Optional override for the activation function. If None, defaults to the activation fn that the builder is initialized with.

Returns

Params for a layer.

_GINCondFC(name, lhs, rhs, idims, adims, odims, use_bn=True, activation_fn=None)

_GIN(*args, **kwargs)

_GINIntermediateLayer(*args, **kwargs)

_ConcatPointsToFeatures(*args, **kwargs)

_SetAbstraction(*args, **kwargs)

_PointConvParametricConv(*args, **kwargs)