lingvo.tasks.car.summary module

Functions to help with plotting summaries.

These functions try to take care in ensuring that transformations on points and headings are consistent. For example, when points are transformed from the standard x/y plane (+x going right, +y going up) to an alternate formulation such as ‘flipped axes’, the heading must also be updated to account for the change in axis directions.

lingvo.tasks.car.summary._PILColorList()[source]
lingvo.tasks.car.summary.ExtractRunIds(run_segments)[source]

Extract the RunIds from the run_segments feature field.

Parameters

run_segments –

a string Tensor of shape [batch, 1] containing a text proto.

See SummaryTest.testExtractRunIds for an example.

Returns

A string Tensor of shape [batch], containing the extracted run id.

lingvo.tasks.car.summary.CameraImageSummary(frontal_images, run_segment_strings, figsize=(6, 4))[source]

Write frontal_images as tf.Summaries.

Parameters

  • frontal_images – Float tensor of frontal camera images: Shape: [batch, height, width, depth]. Expected aspect ratio of 3:2 for visualization.

  • run_segment_strings – Tensor of strings: Shape: [batch, 1]. The associated RunSegment proto for the batch.

  • figsize – Tuple indicating size of camera image. Default is (6, 4)

  • a 3 (indicating) – 2 aspect ratio for visualization.

lingvo.tasks.car.summary._CarToImageTransform()[source]
lingvo.tasks.car.summary.DrawTopDown(lasers)[source]

Draw the laser points in the top down car view.

lingvo.tasks.car.summary.MakeRectangle(l, w, theta, offset=(0, 0))[source]

Make rotated rectangle.

lingvo.tasks.car.summary.DrawHeadingTriangle(draw, x, y, heading, color, scale=25)[source]

Draw a triangle indicating heading at x, y with color.

lingvo.tasks.car.summary.DrawCircle(draw, x, y, fill, outline, circle_size=5)[source]

Draw a circle at x, y.

lingvo.tasks.car.summary.DrawBoundingBoxOnImage(image, box, display_str, color='red', thickness=4, text_loc='BOTTOM')[source]

Draw bounding box on the input image.

lingvo.tasks.car.summary.VisualizeBoxes(image, boxes, classes, scores, class_id_to_name, min_score_thresh=0.25, line_thickness=4, groundtruth_box_visualization_color='black', skip_scores=False, skip_labels=False, text_loc='TOP')[source]

Visualize boxes on top down image.

lingvo.tasks.car.summary.TransformBBoxesToTopDown(bboxes, car_to_image_transform=None)[source]

Convert bounding boxes from car coordinates to top down pixel coordinates.

Parameters

  • bboxes – A (batch, nbboxes, 4 or 5) np.float32 tensor containing bounding box xywhh in car coordinates (smooth adjusted by car pose).

  • car_to_image_transform – An optional Transform object. If None, this will be created using _CarToImageTransform.

Returns

np.array of shape (batch, nbboxes) containing the bounding boxes in top down image space.

lingvo.tasks.car.summary.DrawBBoxesOnImages(images, bboxes, box_weights, labels, class_id_to_name, groundtruth)[source]

Draw ground truth boxes on top down image.

Parameters

  • images – A 4D uint8 array (batch, height, width, depth) of images to draw on top of.

  • bboxes – A (batch, nbboxes, 4 or 5) np.float32 tensor containing bounding box xywhh to draw specified in top down pixel values.

  • box_weights – A (batch, nbboxes) float matrix indicating the predicted score of the box. If the score is 0.0, no box is drawn.

  • labels – A (batch, nbboxes) integer matrix indicating the true or predicted label indices.

  • class_id_to_name – Dictionary mapping from class id to name.

  • groundtruth – Boolean indicating whether bounding boxes are ground truth.

Returns

‘images’ with the bboxes drawn on top.

lingvo.tasks.car.summary.DrawTrajectory(image, bboxes, masks, labels, is_groundtruth)[source]

Draw the trajectory of bounding boxes on ‘image’.

Parameters

  • image – The uint8 image array to draw on. Assumes [1000, 500, 3] input with RGB value ranges.

  • bboxes – A [num_steps, num_objects, 5] float array containing the bounding box information over a sequence of steps. bboxes are expected to be in car coordinates.

  • masks – A [num_steps, num_objects] integer array indicating whether the corresponding bbox entry in bboxes is present (1 = present).

  • labels – A [num_steps, num_objects] integer label indicating which class is being predicted. Used for colorizing based on labels.

  • is_groundtruth – True if the scene is the groundtruth vs. the predicted.

Returns

The updated image array.

lingvo.tasks.car.summary.GetTrajectoryComparison(gt_bboxes, gt_masks, gt_labels, pred_bboxes, pred_masks, pred_labels)[source]

Draw a trajectory comparison of groundtruth and predicted.

Parameters

  • gt_bboxes – A [batch_size, num_steps, num_objects, 5] float array containing the bounding box information over a sequence of steps.

  • gt_masks – A [batch_size, num_steps, num_objects] integer array indicating whether the corresponding bbox entry in bboxes is present (1 = present).

  • gt_labels – A [batch_size, num_steps, num_objects] integer label indicating which class is being predicted. Used for colorizing based on labels.

  • pred_bboxes – A [batch_size, num_steps, num_objects, 5] float array containing the bounding box information over a sequence of steps.

  • pred_masks – A [batch_size, num_steps, num_objects] integer array indicating whether the corresponding bbox entry in bboxes is present (1 = present).

  • pred_labels – A [batch_size, num_steps, num_objects] integer label indicating which class is being predicted. Used for colorizing based on labels.

Returns

A np.uint8 images array that can be displayed of size

[batch_size, 1000, 500, 3].

Return type

images