lingvo.tasks.car.geometry module¶
Routines related to geometric operations on bboxes.
Bboxes and coordinates are always in the last dimension of a tensor.
2D BBoxes are represented by (ymin, xmin, ymax, xmax).
2D BBoxes can also be represented by (centroid x, centroid y, width, height).
2D coordinates are represented by (x, y).
3D coordinates are represented by (x, y, z).
-
lingvo.tasks.car.geometry._BroadcastMatmul(x, y)[source]¶ Broadcast y and matmul with x.
- Parameters
x – A tensor of shape […, b].
y – A matrix of shape [b, c].
- Returns
z[..., c], wherez[i..., :] = matmul(x[i..., :], y)- Return type
Tensor
-
lingvo.tasks.car.geometry._MakeRotationMatrix(yaw, roll, pitch)[source]¶ Create a 3x3 rotation matrix from yaw, roll, pitch (angles in radians).
Note: Yaw -> Z, Roll -> X, Pitch -> Y.
- Parameters
yaw – float tensor representing a yaw angle in radians.
roll – float tensor representing a roll angle in radians.
pitch – float tensor representing a pitch angle in radians.
- Returns
A [3, 3] tensor corresponding to a rotation matrix.
-
lingvo.tasks.car.geometry.BatchMakeRotationMatrix(yaw)[source]¶ Create a Nx3x3 rotation matrix from yaw.
- Parameters
yaw – float tensor representing a yaw angle in radians.
- Returns
A [N, 3, 3] tensor corresponding to a rotation matrix.
-
lingvo.tasks.car.geometry.CoordinateTransform(points, pose)[source]¶ Translate ‘points’ to coordinates according to ‘pose’ vector.
- pose should contain 6 floating point values:
translate_x, translate_y, translate_z: The translation to apply. yaw, roll, pitch: The rotation angles in radians.
- Parameters
points – Float shape […, 3]: Points to transform to new coordinates.
pose – Float shape [6]: [translate_x, translate_y, translate_z, yaw, roll, pitch]. The pose in the frame that ‘points’ comes from, and the defintion of the rotation and translation angles to apply to points.
- Returns
‘points’ transformed to the coordinates defined by ‘pose’.
-
lingvo.tasks.car.geometry.TransformPoints(points, transforms)[source]¶ Apply 4x4 transforms to a set of points.
- Parameters
points – A […, num_points, 3] tensor of xyz point locations.
transforms – A […, 4, 4] tensor with the same leading shape as points.
- Returns
A tensor with the same shape as points, transformed respectively.
-
lingvo.tasks.car.geometry.WrapAngleRad(angles_rad, min_val=- 3.141592653589793, max_val=3.141592653589793)[source]¶ Wrap the value of
angles_radto the range [min_val, max_val].
-
lingvo.tasks.car.geometry.TransformBBoxes3D(bboxes_3d, transforms)[source]¶ Apply 4x4 transforms to 7 DOF bboxes (change center and rotation).
- Parameters
bboxes_3d – A […, num_boxes, 7] tensor representing 3D bboxes.
transforms – A […, 4, 4] tensor with the same leading shape as bboxes_3d. These transforms are expected to only affect translation and rotation, while not scaling the data. This ensures that the bboxes have the same dimensions post transformation.
- Returns
A tensor with the same shape as bboxes_3d, with transforms applied to each bbox3d.
-
lingvo.tasks.car.geometry.PointsToImagePlane(points, velo_to_image_plane)[source]¶ Converts 3D points to the image plane.
- Parameters
points – A [N, 3] Floating point tensor containing xyz points. Points are assumed to be in velo coordinates.
velo_to_image_plane – A [3, 4] matrix from velo xyz to image plane xy. After multiplication, you need to divide by last coordinate to recover 2D pixel locations.
- Returns
A [N, 2] Floating point tensor containing points in the image plane.
-
lingvo.tasks.car.geometry.ReorderIndicesByPhi(anchor, bboxes)[source]¶ Sort bboxes based their angles relative to the anchor point.
- Parameters
anchor – A vector of (x0, y0).
bboxes – A matrix of shape [N, 4].
- Returns
A permutation of tf.range(n) which can be used to reshuffle bboxes to the sorted order. (e.g., tf.gather(bboxes, indices)).
-
lingvo.tasks.car.geometry.DistanceBetweenCentroidsAndBBoxesFastAndFurious(centroids, bboxes, masks)[source]¶ Computes the distance between centroids and bboxes.
The distance/loss is loosely following the ‘Fast and Furious’ paper by Luo et al., CVPR’18. This is just one way of calculating the distances. We will probably develop other ways.
- Parameters
centroids – […, 4]. x/y/w/h for bboxes.
bboxes – […, 4]. ymin/xmin/ymax/xmax for bboxes.
masks – […]. masks[i] == 1 means i-th entry (centroids[i] and bboxes[i]) should be considered in the distance/loss calculation.
- Returns
A […] tensor. i-th value is the distance measure of centroids[i] and bboxes[i].
-
lingvo.tasks.car.geometry.DistanceBetweenCentroids(u, v, masks)[source]¶ Computes the distance between centroids.
- Parameters
u – […, 4]. x/y/w/h for bboxes.
v – […, 4]. x/y/w/h for bboxes.
masks – […]. masks[i] == 1 means i-th entry (u[i] and v[i]) should be considered in the distance/loss calculation.
- Returns
A […] tensor. i-th value is the distance measure of u[i] and v[i].
-
lingvo.tasks.car.geometry._IsOnLeftHandSideOrOn(point, v1, v2)[source]¶ Checks if a point lays on a vector direction, or is to the left.
- Parameters
point – a tensor of shape […, 2] of points to check.
v1 – a float tensor of shape […, 2] of vertices.
v2 – a tensor of shape and type as v1. The second vertices.
- Returns
A tensor of booleans indicating whether each point is on the left of, or exactly on, the direction indicated by the vertices.
-
lingvo.tasks.car.geometry._IsCounterClockwiseDirection(v1, v2, v3)[source]¶ Checks if the path from v1 to v3 via v2 is counter-clockwise.
When v1 is equal to v2, or v2 equals v3, return true, by fiat. Tis will work when the v’s are padded vectors.
- Parameters
v1 – a float Tensor of shape […, 2], indicating the starting point.
v2 – a Tensor of same type and shape as v1, indicating the via point.
v3 – a Tensor of same type and shape as v1, indicating the ending point.
- Returns
True for all directions such that v1 to v3 via v2 is a counter clockwise
direction.
-
lingvo.tasks.car.geometry.IsWithinBBox(points, bbox)[source]¶ Checks if points are within a 2-d bbox.
The function returns true if points are strictly inside the box. It also returns true when the points are exactly on the box edges.
- Parameters
points – a float Tensor of shape […, 2] of points to be tested. The last coordinates are (x, y).
bbox – a float Tensor of shape […, 4, 2] of bboxes. The last coordinates are the four corners of the bbox and (x, y). The corners are assumed to be given in counter-clockwise order.
- Returns
If
pshape = tf.shape(points)[:-1]andbshape = tf.shape(bbox)[:-2], returns a boolean tensor of shapetf.concat(pshape, bshape), where each element is true if the point is inside to the corresponding box. If a point falls exactly on an edge of the bbox, it is also true.- Return type
Tensor
-
lingvo.tasks.car.geometry.BBoxCorners2D(bboxes)[source]¶ Extract the corner points from a 5-DOF bbox representation.
- Parameters
bboxes – A […, 5] floating point bounding box representation ([x, y, dx, dy, phi]).
- Returns
- A […, 4, 2] floating point Tensor containing
the corner (x, y) points for every bounding box.
-
lingvo.tasks.car.geometry.BBoxCorners(bboxes)[source]¶ Extract the corner points from a 7-DOF bbox representation.
- Parameters
bboxes – A [batch, num_boxes, 7] floating point bounding box representation ([x, y, z, dx, dy, dz, phi]).
- Returns
- A [batch, num_boxes, 8, 3] floating point Tensor containing
the corner (x, y, z) points for every bounding box.
-
lingvo.tasks.car.geometry.IsWithinBBox3D(points_3d, bboxes_3d)[source]¶ Checks if points are within a 3-d bbox.
- Parameters
points_3d – [num_points, 3] float32 Tensor specifying points in 3-d space as [x, y, z] coordinates.
bboxes_3d – [num_bboxes, 7] float32 Tensor specifying a 3-d bboxes specified as [x, y, z, dx, dy, dz, phi] where x, y and z is the center of the box.
- Returns
boolean Tensor of shape [num_points, num_bboxes] indicating whether the points belong within each box.
-
lingvo.tasks.car.geometry.SphericalCoordinatesTransform(points_xyz)[source]¶ Converts points from xyz coordinates to spherical coordinates.
https://en.wikipedia.org/wiki/Spherical_coordinate_system#Coordinate_system_conversions for definitions of the transformations.
- Parameters
points_xyz – A floating point tensor with shape […, 3], where the inner 3 dimensions correspond to xyz coordinates.
- Returns
A floating point tensor with the same shape […, 3], where the inner dimensions correspond to (dist, theta, phi), where phi corresponds to azimuth/yaw (rotation around z), and theta corresponds to pitch/inclination (rotation around y).