lingvo.tasks.car.input_preprocessors module¶
Input preprocessors.
-
lingvo.tasks.car.input_preprocessors._ConsistentShuffle(tensors, seed)[source]¶ Shuffle multiple tensors with the same shuffle order.
-
lingvo.tasks.car.input_preprocessors._GetApplyPointMaskFn(points_mask)[source]¶ Returns a function that applies a mask to one of our points tensors.
-
class
lingvo.tasks.car.input_preprocessors.Preprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.core.base_layer.BaseLayerBase class for input preprocessor.
Input preprocessors expect the combined output of all extractors and performs a transformation on them. Input preprocessors can add/edit/remove fields from the NestedMap of features.
Note: Features correspond to that for one example (no batch dimension).
Sub-classes need to implement the following three functions:
TransformFeatures(features): Given a NestedMap of features representing the output of all the extractors, apply a transformation on the features.
TransformShapes(shapes): Given a corresponding NestedMap of shapes, produce a NestedMap of shapes that corresponds to the transformation of the features after TransformFeatures.
TransformDTypes(dtypes): Given a corresponding NestedMap of dtypes, produce a NestedMap of dtypes that corresponds to the transformation of the features after TransformFeatures.
The preprocessor is expected to explicitly pass through untouched fields. For example, a preprocessor that does data augmentation should modify the features NestedMap on the fields it cares about augmenting, and then return the features NestedMap.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
TransformBatchedFeatures(features)[source]¶ Transforms the features for a batch of examples.
- Parameters
features – A
NestedMapof batched tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.EntryPreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorA Preprocessor that transforms a NestedMap sub-structure.
Some preprocessors want to apply a function to any NestedMap whose key matches a specific prefix. An EntryPreprocessor provides an interface for specifying the function transformation for a NestedMap of inputs, adding, modifying, or deleting the entries in that NestedMap.
- For example, if an input contains a nested structure such as:
- lasers.front.xyz
.features
- lasers.side.xyz
.features
and one wants to apply a transform that modifies the .xyz features on both structures, one can define an EntryPreprocessor that implements:
UpdateEntry(entry): UpdateEntryShape(shapes): UpdateEntryDType(dtypes):
and set self.params.prefixes = [‘lasers.front’, ‘lasers.side’] where the prefixes refer to a fully-qualified NestedMap sub-structure.
- The arguments to these functions will contain just the NestedMap structure
whose key prefix can be found in self.params.prefixes. One can then modify these structures as desired.
Example
- def UpdateEntry(self, entry):
# entry is a NestedMap. assert ‘xyz’ in entry entry.xyz = self._ApplyFn(entry.xyz)
-
_ApplyToMatchingStructure(nested_map, fn)[source]¶ Apply fn to any NestedMap sub-structure whose prefix is in p.prefixes.
-
UpdateEntry(entry)[source]¶ Update the Tensors in a NestedMap entry.
- Parameters
entry – A NestedMap of Tensors.
-
UpdateEntryShape(shapes)[source]¶ Update the shapes in a NestedMap entry.
- Parameters
shapes – A NestedMap of TensorShapes.
-
UpdateEntryDType(dtypes)[source]¶ Transform the dtypes in a NestedMap entry.
- Parameters
dtypes – A NestedMap of dtypes.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.CreateDecoderCopy(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorCreates references to current lasers, images, and labels.
This is useful if the data is further transformed.
If desired, the keys that are copied can be customized by overriding the default keys param.
This preprocessor expects features to optionally contain the following keys: - lasers - a NestedMap of tensors - images - a NestedMap of tensors - labels - a NestedMap of tensors
- Adds the following features (if the features existed):
decoder_copy.lasers - a copy of the lasers NestedMap
decoder_copy.images - a copy of the images NestedMap
decoder_copy.labels - a copy of the labels NestedMap
The processor also by default pads the laser features; this can be disabled by setting the pad_lasers param to None.
-
_DeepCopyIfExists(keys, nested_map, parent_key)[source]¶ Deep copy a specific key to a parent key if it exists.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.FilterByKey(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorFilters features to keep only specified keys.
This keeps only feature entries that are specified. This allows us to reduce the number of fields returned. For example, during training, one may not need the actual laser points if training with a pillars based model that has a preprocessor that already maps the points to grid.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.FilterGroundTruthByNumPoints(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRemoves ground truth boxes with less than params.min_num_points points.
- This preprocessor expects features to contain the following keys::
labels.labels of shape […, L] labels.bboxes_3d of shape […, L, 7] labels.bboxes_3d_mask of shape […, L] labels.unfiltered_bboxes_3d_mask of shape […, L] labels.bboxes_3d_num_points of shape […, L].
Modifies the bounding box data to turn off ground truth objects that don’t meet the params.min_num_points point filter:
labels.labels: Boxes with less than params.min_num_points have their label set to params.background_id (defaults to 0).
labels.bboxes_3d_mask: Boxes with less than params.min_num_points are set to 0.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.FilterGroundTruthByDifficulty(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRemoves groundtruth boxes based on detection difficulty.
- This preprocessor expects features to contain the following keys::
labels.single_frame_detection_difficulties of shape […, L] labels.labels of shape […, L] labels.bboxes_3d_mask of shape […, L] labels.unfiltered_bboxes_3d_mask of shape […, L]
The preprocessor masks out the bboxes_3d_mask / labels based on whether single_frame_detection_difficulties is greater than p.difficulty_threshold.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.CountNumberOfPointsInBoxes3D(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorComputes bboxes_3d_num_points.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - labels.bboxes_3d of shape [L, 7] - labels.bboxes_3d_mask of shape [L]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Adds the following features:
- labels.bboxes_3d_num_points: [L] - integer tensor containing the number of
laser points for each corresponding bbox.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.AddPerPointLabels(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorComputes the class and bbox id of each point.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - labels.bboxes_3d of shape [L, 7] - labels.labels of shape [L]
This makes an assumption that each point is only in 1 box, which should almost always true in 3D. In cases where this is not true, the largest label integer and largest bbox_id will be assigned.
NOTE: Be very careful that this is performed after any modifications to the semantic labels of each point in the pointcloud. Examples of this would be operators like GroundTruthAugmentation, or DropBoxesOutOfRange.
- Adds the following features:
- lasers.points_label: [P] - integer tensor containing the class id of each
point.
- lasers.points_bbox_id: [P] - integer tensor containing box id of each
point from 0 to num_bboxes, where an id of num_bboxes indicates a background point.
- lasers.points_bbox_3d: [P, 7] - float tensor containing bounding box of
each point.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.PointsToGrid(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorBins points to a 3D-grid using custom op: ops.point_to_grid.
Expects features to have keys: - lasers.points_xyz of shape [P, 3]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
If normalizing the labels is enabled, then also expects: - labels.weights - labels.bboxes_td - labels.bboxes_td_mask - labels.bboxes_3d_mask
- Let:
gx, gy, gz = p.grid_size F = 3 + num_laser_features
- Adds the following features:
- grid_centers: [gx, gy, gz, 3]: For each grid cell, the (x,y,z)
floating point coordinate of its center.
- grid_num_points: [gx, gy, gz]: The number of points in each grid
cell (integer).
- laser_grid: [gx, gy, gz, num_points_per_cell, F] - A 5D floating
point Tensor containing the laser data placed into a fixed grid.
Modifies the bboxes in labels to also be within the grid range x/y by default.
-
_NormalizeLabels(ymin, xmin, ymax, xmax, x_range, y_range)[source]¶ Normalizes the bboxes within a given range.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors._PointPillarGridSettings[source]¶ Bases:
objectSettings for PointPillars model defined in paper.
https://arxiv.org/abs/1812.05784
-
GRID_X= 432¶
-
GRID_Y= 496¶
-
GRID_Z= 1¶
-
GRID_X_RANGE= None¶
-
GRID_Y_RANGE= None¶
-
GRID_Z_RANGE= None¶
-
-
lingvo.tasks.car.input_preprocessors.MakeGridSettings(grid_x_range, grid_y_range, grid_z_range, grid_x, grid_y, grid_z)[source]¶ Returns configured class for PointPillar grid settings.
-
lingvo.tasks.car.input_preprocessors.PointPillarGridCarSettings¶ alias of
lingvo.tasks.car.input_preprocessors.MakeGridSettings.<locals>.GridSettings
-
lingvo.tasks.car.input_preprocessors.PointPillarGridPedCycSettings¶ alias of
lingvo.tasks.car.input_preprocessors.MakeGridSettings.<locals>.GridSettings
-
class
lingvo.tasks.car.input_preprocessors.GridToPillars(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorCreate pillars from a grid of points.
- Expects features to have keys:
grid_centers: [gx, gy, gz, 3]
grid_num_points: [gx, gy, gz]
laser_grid: [gx, gy, gz, num_points_per_cell, F]
- Adds the following features:
point_count: [num_pillars]. The number of points in the pillar.
point_locations: [num_pillars, 3]. The grid location of each pillar.
pillar_points: [num_pillars, num_points_per_cell, F]. Points of each pillar.
- Drops the following features by default:
laser_grid
-
_GumbelTransform(probs)[source]¶ Adds gumbel noise to log probabilities for multinomial sampling.
This enables fast sampling from a multinomial distribution without replacement. See https://arxiv.org/abs/1611.01144 for details. A colab that demonstrates this in practice is here: http://colab/drive/1iuMt2n_r7dKPQG9T0UVMuK3fkbBayKjd
- Parameters
probs – A 1-D float tensor containing probabilities, summing to 1.
- Returns
A 1-D float tensor of the same size of probs, with gumbel noise added to log probabilities. Taking the top k elements from this provides a multinomial sample without replacement.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.GridAnchorCenters(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorCreate anchor centers on a grid.
Anchors are placed in the middle of each grid cell. For example, on a 2D grid range (0 -> 10, 0 -> 10) with a 10 x 5 grid size, the anchors will be placed at [(0.5, 1), (0.5, 3), … , (9.5, 7), (9.5, 9)].
- Adds the following features:
- anchor_centers: [num_locations, 3] - Floating point output containing the
center (x, y, z) locations for tiling anchor boxes.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.SparseCenterSelector(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorSelect centers for anchors and cells.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
If lasers.num_seeded_points of shape [] is provided, it indicates that the first num_seeded_points of lasers.points_xyz should be used as seeds for farthest point sampling (e.g., always chosen). Currently the concept of seeding is not implemented for anything but farthest point sampling.
- Adds the following features:
- anchor_centers: [num_cell_centers, 3] - Floating point output containing the
center (x, y, z) locations for tiling anchor boxes.
- cell_center_xyz: [num_cell_centers, 3] - Floating point output containing
the center (x, y, z) locations for each cell to featurize.
-
_SAMPLING_METHODS= ['farthest_point', 'random_uniform']¶
-
_FarthestPointSampleCenters(points_xyz, num_seeded_points)[source]¶ Samples centers with Farthest Point Sampling.
- Parameters
points_xyz – An unpadded tf.float32 Tensor of shape [P, 3] with per point (x, y, z) locations. We expect any padded points to be removed before this function is called.
num_seeded_points – integer indicating how many of the first num_seeded_points points in points_xyz should be considered as seeds for FPS (always chosen).
- Returns
A tf.float32 Tensor of shape [p.num_cell_centers, 3] with selected centers to use as anchors.
-
_RandomUniformSampleCenters(points_xyz)[source]¶ Samples centers with Random Uniform Sampling.
- Parameters
points_xyz – An unpadded tf.float32 Tensor of shape [P, 3] with per point (x, y, z) locations. We expect any padded points to be removed before this function is called.
- Returns
A tf.float32 Tensor of shape [p.num_cell_centers, 3] with selected centers to use as anchors.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.SparseCellGatherFeatures(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorSelect local features for each cell.
This preprocessor expects features to contain: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F] - cell_center_xyz of shape [C, 3]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Adds the following features:
- cell_points_xyz: [num_centers, num_points_per_cell, 3] - Floating point
output containing the (x, y, z) locations for each point for a given center.
- cell_feature: [num_centers, num_points_per_cell, F] - Floating point output
containing the features for each point for a given center.
- cell_points_padding: [num_centers, num_points_per_cell] - 0/1 padding
for the points in each cell.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.SparseCellCentersTopK(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorGiven selected centers and gathered points/features, apply a filter.
This preprocessor expects features to contain
cell_center_xyzand all entries in params.features_to_modify, and that the leading dimension should all be the same (num_cell_centers from SparseCenterSelector).We then modify all values in features that are specified in params.features_to_modify by sorting them with the specified sort function (specified by params.sort_by) operating on features.cell_center_xyz, and then taking the top K (specified by params.num_cell_centers) along the first dimension.
-
_REGISTERED_SORT_FUNCTIONS= ['distance']¶
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.TileAnchorBBoxes(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorCreates anchor_bboxes given anchor_centers.
This preprocessor expects features to contain the following keys: - anchor_centers of shape […base shape…, 3]
- Adds the following features:
- anchor_bboxes: base_shape + [7] - Floating point anchor box
output containing the anchor boxes and the 7 floating point values for each box that define the box (x, y, z, dx, dy, dz, phi).
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors._AnchorBoxSettings[source]¶ Bases:
objectHelper class to parameterize and update anchor box settings.
-
DIMENSION_PRIORS= []¶
-
ROTATIONS= []¶
-
CENTER_X_OFFSETS= []¶
-
CENTER_Y_OFFSETS= []¶
-
CENTER_Z_OFFSETS= []¶
-
classmethod
GenerateAnchorSettings()[source]¶ Generate anchor settings.
- Returns
anchor_box_dimensions - anchor_box_rotations - anchor_box_offsets
These can be used with the TileAnchorBBoxes preprocessor.
- Return type
A
NestedMapcontaining three lists of the same length
-
classmethod
Update(params)[source]¶ Updates anchor box settings from input configuration lists.
Given dimensions priors, rotations, and offsets, computes the cartesian product of the settings.
- Parameters
params – The KITTIAnchorExtractorBase.Params() object to update.
- Returns
Params updated with the anchor settings.
- In total there are N combinations, where each (anchor_box_dimensions[i],
anchor_box_rotations[i], anchor_box_offsets[i]) for i in range(N) is an option.
-
-
lingvo.tasks.car.input_preprocessors.MakeAnchorBoxSettings(dimension_priors, rotations, center_x_offsets, center_y_offsets, center_z_offsets)[source]¶ Returns a configured class for setting anchor box settings.
-
class
lingvo.tasks.car.input_preprocessors.SparseCarV1AnchorBoxSettings[source]¶ Bases:
lingvo.tasks.car.input_preprocessors._AnchorBoxSettingsAnchor box settings for training on Cars for Sparse models.
-
DIMENSION_PRIORS= [(1.6, 3.9, 1.56)]¶
-
ROTATIONS= [0, 1.5707963267948966, 0.7853981633974483, 2.356194490192345]¶
-
CENTER_X_OFFSETS= array([-1.5 , -0.75, 0. , 0.75, 1.5 ])¶
-
CENTER_Y_OFFSETS= array([-1.5 , -0.75, 0. , 0.75, 1.5 ])¶
-
CENTER_Z_OFFSETS= [-1.0]¶
-
-
class
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsCar[source]¶ Bases:
lingvo.tasks.car.input_preprocessors._AnchorBoxSettings-
DIMENSION_PRIORS= [(1.6, 3.9, 1.56)]¶
-
ROTATIONS= [0, 1.5707963267948966]¶
-
CENTER_X_OFFSETS= [0.0]¶
-
CENTER_Y_OFFSETS= [0.0]¶
-
CENTER_Z_OFFSETS= [-1.0]¶
-
-
class
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsPed[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsCar-
DIMENSION_PRIORS= [(0.6, 0.8, 1.73)]¶
-
CENTER_Z_OFFSETS= [-0.6]¶
-
-
class
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsCyc[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsCar-
DIMENSION_PRIORS= [(0.6, 1.76, 1.73)]¶
-
CENTER_Z_OFFSETS= [-0.6]¶
-
-
class
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsPedCyc[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PointPillarAnchorBoxSettingsCar-
DIMENSION_PRIORS= [(0.6, 0.8, 1.7), (0.6, 1.76, 1.73)]¶
-
CENTER_Z_OFFSETS= [-0.6]¶
-
-
class
lingvo.tasks.car.input_preprocessors.AnchorAssignment(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorPerform anchor assignment on the features.
This preprocessor expects features to contain the following keys: - anchor_bboxes of shape […base shape…, 7] - labels.bboxes_3d - labels.labels - labels.bboxes_3d_mask
Adds the following features:
- anchor_localization_residuals: base_shape + [7] floating point tensor of
residuals. The model is expected to regress against these residuals as targets. The residuals can be converted back into bboxes using detection_3d_lib.Utils3D.ResidualsToBBoxes.
- assigned_gt_idx: base_shape - The corresponding index of the ground
truth bounding box for each anchor box in anchor_bboxes, anchors not assigned will have idx be set to -1.
- assigned_gt_bbox: base_shape + [7] - The corresponding ground
truth bounding box for each anchor box in anchor_bboxes.
- assigned_gt_labels: base_shape - The assigned groundtruth label
for each anchor box.
- assigned_gt_similarity_score: base_shape - The similarity score
for each assigned anchor box.
- assigned_cls_mask: base_shape mask for classification loss per anchor.
This should be 1.0 if the anchor has a foreground or background assignment; otherwise, it will be assigned to 0.0.
- assigned_reg_mask: base_shape mask for regression loss per anchor.
This should be 1.0 if the anchor has a foreground assignment; otherwise, it will be assigned to 0.0. Note: background anchors do not have regression targets.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.DropLaserPointsOutOfRange(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorDrops laser points that are out of pre-defined x/y/z ranges.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Modifies the following features:
Removes or sets padding to 1 for all points outside a given range. Modifies all items in the lasers subdictionary like lasers.points_xyz, lasers.points_feature, lasers.points_padding, and optionally lasers.points_label, lasers.points_bbox_id.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.KITTIDropPointsOutOfFrustum(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorDrops laser points that are outside of the camera frustum.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F] - images.velo_to_image_plane of shape [3, 4] - images.width of shape [1] - images.height of shape [1]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Modifies the following features:
lasers.points_xyz, lasers.points_feature, lasers.points_padding, and optionally lasers.points_label, lasers.points_bbox_id so that points outside the frustum have padding set to 1 or are removed.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.RandomWorldRotationAboutZAxis(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRotates the world randomly as a form of data augmentation.
Rotations are performed around the z-axis. This assumes that the car is always level. In general, we’d like to instead rotate the car on the spot, this would then make sense for cases where the car is on a slope.
When there are leading dimensions, this will rotate the boxes with the same transformation across all the frames. This is useful when the input is a sequence of frames from the same run segment.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape […, 3] - labels.bboxes_3d of shape […, 7]
- Modifies the following features:
lasers.points_xyz, labels.bboxes_3d with the same rotation applied to both.
- Adds the following features:
world_rot_z which contains the rotation applied to the example.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.DropPointsOutOfFrustum(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorDrops points outside of pre-defined theta / phi ranges.
Note that the ranges for keep_phi_range can be negative, this is because the phi values wrap around 2*pi. Thus, a valid range that filters the 90 deg frontal field of view of the car can be specified as [-pi/4, pi/4].
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
Modifies the following features: - lasers.points_xyz removing any points out of frustum. - lasers.points_feature removing any points out of frustum.
Note: We expect a downstream processor that filters out boxes with few points to drop the corresponding bboxes.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.DropBoxesOutOfRange(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorDrops boxes outside of pre-defined x/y/z ranges (boundaries inclusive).
This preprocessor expects features to contain the following keys: - labels.bboxes_3d of shape [N, 7] - labels.bboxes_3d_mask of shape [N]
Modifies the following features: - labels.bboxes_3d_mask to mask out any additional boxes.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.PadLaserFeatures(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorPads laser features so that the dimensions are fixed.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
and optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Modifies the following features:
lasers.points_xyz and lasers.points_feature to add padding. Optionally also modifies lasers.points_label and lasers.points_bbox_id if they exist to add padding.
- Modifies/adds the following features:
labels.points_padding of shape [P] representing the padding.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.WorldScaling(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorScale the world randomly as a form of data augmentation.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - labels.bboxes_3d of shape [L, 7]
- Modifies the following features:
lasers.points_xyz, labels.bboxes_3d with the same scaling applied to both.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.RandomDropLaserPoints(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRandomly dropout laser points and the corresponding features.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
- Modifies the following features:
lasers.points_xyz, lasers.points_feature.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.RandomFlipY(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorFlip the world along axis Y as a form of data augmentation.
When there are leading dimensions, this will flip the boxes with the same transformation across all the frames. This is useful when the input is a sequence of frames from the same run segment.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape […, 3] - labels.bboxes_3d of shape […, 7]
- Modifies the following features:
lasers.points_xyz, labels.bboxes_3d with the same flipping applied to both.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.GlobalTranslateNoise(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorAdd global translation noise of xyz coordinates to points and boxes.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - labels.bboxes_3d of shape [L, 7]
- Modifies the following features:
- lasers.points_xyz, labels.bboxes_3d with the same
random translation noise applied to both.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.RandomBBoxTransform(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRandomly transform bounding boxes and the points inside them.
- This preprocessor expects features to contain the following keys:
lasers.points_xyz of shape [P, 3]
lasers.points_feature of shape [P, F]
lasers.points_padding of shape [P]
labels.bboxes_3d of shape [L, 7]
labels.bboxes_3d_mask of shape [L]
- Modifies the following features:
- lasers.points_{xyz,feature,padding}, labels.bboxes_3d with the
transformed bounding boxes and points.
-
_Foreground(features, points_xyz, points_feature, real_bboxes_3d, points_in_bbox_mask, rotation, translate_pose, transform_fn)[source]¶ Extract and transform foreground points and features.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.GroundTruthAugmentor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorAugment bounding box labels and points from a database.
- This preprocessor expects features to contain the following keys:
lasers.points_xyz of shape [P, 3]
lasers.points_feature of shape [P, F]
lasers.points_padding of shape [P]
labels.bboxes_3d of shape [L, 7]
labels.bboxes_3d_mask of shape [L]
labels.labels of shape [L]
Modifies the above features so that additional objects from a groundtruth database are added.
-
_CreateExampleFilter(db)[source]¶ Construct db example filter.
- Parameters
db – NestedMap of the following Tensors: points_mask - [N, P] - The points mask for every object in the database, where N is the number of objects and P is the maximum number of points per object. labels - [N] - int32 Label for each object in the database. difficulties - [N] - int32 Difficulty for each label in the database.
- Returns
A [N] boolean Tensor for each object in the database, True if that corresponding object passes the filter.
-
_FilterIndices(gt_bboxes_3d, db_bboxes, db_idx)[source]¶ Identify database boxes that don’t overlap with other boxes.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.FrustumDropout(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRandomly drops out points in a frustum.
All points are first converted to spherical coordinates, and then a point is randomly selected. All points in the frustum around that point within a given phi, theta angle width and distance to the original greater than a given value are dropped with probability = 1 - keep_prob.
Here, we can specify whether the dropped frustum is the union or intersection of the phi and theta angle filters.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
Optionally points_padding of shape [P] corresponding to the padding. if points_padding is None, then all points are considered valid.
- Modifies the following features:
lasers.points_xyz, lasers.points_feature, lasers.points_padding with points randomly dropped out.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
class
lingvo.tasks.car.input_preprocessors.RepeatPreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRepeat a preprocessor multiple times.
This preprocessor takes a preprocessor as a subprocessor and apply the subprocessor to features multiple times (repeat_count).
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.RandomApplyPreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRandomly apply a preprocessor with certain probability.
This preprocessor takes a preprocessor as a subprocessor and apply the subprocessor to features with certain probability.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.ConstantPreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorPreprocessor that produces specified constant values in a nested output.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.IdentityPreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorPreprocessor that passes all inputs through.
This may be useful for situations where one wants a ‘no-op’ preprocessor, such as being able to randomly choose to do nothing among a set of preprocessor choices.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.RandomChoicePreprocessor(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorRandomly applies a preprocessor with specified weights.
The input at features[p.weight_tensor_key] must be a floating point vector Tensor whose length matches the number of subprocessors to select among. The values in that Tensor are interpreted as relative weights.
For example, if p.subprocessors = [preprocessor1, preprocessor2] and the weights are [1., 2.], then preprocessor1 will be applied with probability 1/3, and preprocessor2 will be applied with probability 2/3.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.
-
-
class
lingvo.tasks.car.input_preprocessors.SparseSampler(*args, **kwargs)[source]¶ Bases:
lingvo.tasks.car.input_preprocessors.PreprocessorFused SparseCenterSelector and SparseCellGatherFeatures.
This preprocessor expects features to contain the following keys: - lasers.points_xyz of shape [P, 3] - lasers.points_feature of shape [P, F]
- Adds the following features:
anchor_centers - [num_centers, 3] - Floating point output containing the center (x, y, z) locations for tiling anchor boxes.
cell_center_xyz - [num_centers, 3] - Floating point output containing the center (x, y, z) locations for each cell to featurize.
cell_center_padding - [num_centers] - 0/1 padding for each center.
cell_points_xyz - [num_centers, num_neighbors, 3] - Floating point output containing the (x, y, z) locations for each point for a given center.
cell_feature - [num_centers, num_neighbors, F] - Floating point output containing the features for each point for a given center.
cell_points_padding - [num_centers, num_neighbors] - 0/1 padding for the points in each cell.
-
TransformFeatures(features)[source]¶ Transforms the features for one example.
- Parameters
features – A
NestedMapof tensors.- Returns
A
NestedMapof tensors corresponding.