lingvo.tasks.car.detection_3d_metrics module

Metrics for 3D detection problems.

class lingvo.tasks.car.detection_3d_metrics.TopDownVisualizationMetric(top_down_transform, class_id_to_name=None, image_height=1536, image_width=1024, figsize=None, ground_removal_threshold=- 1.35, sampler_num_samples=8)

Bases: lingvo.core.metrics.BaseMetric

Top-down detection visualization, expecting 3D laser points and 2D bboxes.

Updates to this metric is expected to be NestedMap containing:

• visualization_labels: [N, B1] int tensor containing visualization labels.

• predicted_bboxes: [N, B1, 5] float tensor containing predicted 2D bboxes each with (x, y, dx, dy, phi).

• visualization_weights: [N, B1] float tensor containing weights for each prediction. predictions with 0 weight will not be drawn.

• points_xyz: [N, P, 3] float tensor containing (x, y, z) coordinates.

• points_padding: [N, P] tensor containing 1 if the point is a padded point.

• gt_bboxes_2d: [N, B2, 5] float tensor containing ground-truth 2D bboxes.

• gt_bboxes_2d_weights: [N, B2] float tensor containing weights for each ground-truth. predictions with 0 weight will not be drawn. The ground-truth mask can be used here.

• labels: [N, B2] int tensor containing ground-truth labels.

• difficulties: [N, B2]: int tensor containing the difficulty levels of each groundtruth box.

Default parameters visualize the area around the car, with the car centered in the image, over a 32m x 48m range.

Ground-truth boxes will be drawn with color=cyan (see DrawBBoxesOnImages for details). Predicted boxes will be drawn with a color from the PIL color list, with a different color per class.

Update(decoded_outputs)

Add top down visualization to summaries.

Parameters

decoded_outputs – A NestedMap containing the fields visualization_labels, predicted_bboxes, visualization_weights, points_xyz, points_padding, gt_bboxes_2d, gt_bboxes_2d_weights, and labels.

_XYWHToExtrema(bboxes)

Convert from x, y, dx, dy to extrema ymin, xmin, ymax, xmax.

_DrawLasers(images, points_xyz, points_padding, transform)

Draw laser points.

Summary(name)

Converts the current state of this metric to a tf.Summary.

Parameters

name – A string to use as the summary value tag.

Returns

A tf.Summary proto.

_EvaluateIfNecessary(name)

Create a top down image summary, if not already created.

DrawDifficulty(images, gt_bboxes, gt_box_weights, difficulties)

Draw the difficulty values on each ground truth box.

class lingvo.tasks.car.detection_3d_metrics.WorldViewer(sampler_num_samples=8)

Bases: lingvo.core.metrics.BaseMetric

World Viewer for 3d point cloud scenes.

_MAX_HUE = 0.65

_MAX_DISTANCE_METERS = 40.0

Update(decoded_outputs)

Add point cloud mesh data to be summarized.

Parameters

decoded_outputs – A NestedMap containing the fields visualization_labels, predicted_bboxes, visualization_weights, points_xyz, points_padding, gt_bboxes_2d, gt_bboxes_2d_weights, and labels.

Summary(name)

Converts the current state of this metric to a tf.Summary.

Parameters

name – A string to use as the summary value tag.

Returns

A tf.Summary proto.

_EvaluateIfNecessary(name)

Create a mesh summary, if not already created.

class lingvo.tasks.car.detection_3d_metrics.CameraVisualization(figsize=(15, 15), bbox_score_threshold=0.01, sampler_num_samples=8, draw_3d_boxes=True)

Bases: lingvo.core.metrics.BaseMetric

Camera detection visualization.

Visualizes a camera image and predicted bounding boxes on top of the image.

Updates to this metric is expected to be NestedMap containing:

camera_images: [N, W, H, 3] float tensor containing camera image data.

bbox_corners: [N, B1, 8, 2] float tensor containing bounding box corners. For each batch (N), for each box B, there are 8 corners, each with an X and Y value.

bbox_scores: [N, B1] float tensor containing predicted box scores.

Update(decoded_outputs)

Updates this metric (e.g. accumulates statistics) from the arguments.

Summary(name)

Converts the current state of this metric to a tf.Summary.

Parameters

name – A string to use as the summary value tag.

Returns

A tf.Summary proto.

_EvaluateIfNecessary(name)

Create a camera image summary if not already created.