pandaset数据集下载地址：https://scale.com/resources/download/pandaset pandaset工具包代码地址：https://github.com/scaleapi/pandaset-devkit
下载数据集需要通过表格进行注册，然后会转到包含原始数据和注释的下载页面。
国内激光雷达制造商禾赛科技与人工智能数据标注平台公司Scale AI联合发布了面向L5级自动驾驶的开源商用数据集——PandaSet数据集。该数据集可用于训练机器学习模型 ，助力自动驾驶的实现。数据集首次同时使用了机械旋转和图像级前向两类激光雷达进行数据采集，输出点云分割结果，并面向科研及商业应用公开。 数据集包括48，000多个摄像头图像和16，000个激光雷达扫描点云图像(超过100个8秒场景)。它还包括每个场景的28个注释和大多数场景的37个语义分割标签。 采集数据车辆为克莱斯勒，传感器套件主要包括1个机械LiDAR，1个固态LiDAR，5个广角摄像头，1个长焦摄像头，板载GPS / IMU。
数据下载解压后，会出现103个以序列命名的文件夹，不同序列代表不同场景下的数据。 每个序列文件夹又包含4个文件夹，分别是annotations，camera，lidar，meta。 00.pkl.gz~79.pkl.gz 分别对应80帧连续帧的数据， 其格式如下：
. ├── LICENSE.txt ├── annotations │ ├── cuboids │ │ ├── 00.pkl.gz │ │ . │ │ . │ │ . │ │ └── 79.pkl.gz │ └── semseg // Semantic Segmentation is available for specific scenes │ ├── 00.pkl.gz │ . │ . │ . │ ├── 79.pkl.gz │ └── classes.json ├── camera │ ├── back_camera │ │ ├── 00.jpg │ │ . │ │ . │ │ . │ │ ├── 79.jpg │ │ ├── intrinsics.json │ │ ├── poses.json │ │ └── timestamps.json │ ├── front_camera │ │ └── ... │ ├── front_left_camera │ │ └── ... │ ├── front_right_camera │ │ └── ... │ ├── left_camera │ │ └── ... │ └── right_camera │ └── ... ├── lidar │ ├── 00.pkl.gz │ . │ . │ . │ ├── 79.pkl.gz │ ├── poses.json │ └── timestamps.json └── meta ├── gps.json └── timestamps.jsonpandaset提供了加载数据集的工具包pandaset-devkit 安装好工具包后可以直接调用API得到我们想要的数据。
需要注意的是 pandaset使用两种雷达收集数据，一种是360°旋转的雷达pandar64，另外一种是固定朝前的雷达pandarGT。这里的点云来自两个雷达的数据。如果单独选择一个传感器的数据，可以使用set_sensor方法区别两个传感器的数据。
pc0 = s002.lidar[0] print(pc0.shape) # (166768, 6) # 这里的0表示360° LiDAR s002.lidar.set_sensor(0) # set to include only mechanical 360° LiDAR # 这里的0表示第0帧的数据 pc0_sensor0 = s002.lidar[0] print(pc0_sensor0.shape) # (106169, 6) s002.lidar.set_sensor(1) # set to include only front-facing LiDAR pc0_sensor1 = s002.lidar[0] print(pc0_sensor1.shape) # (60599, 6)上面所得到的 LiDAR 点是存储在世界坐标系中，下图为360°LiDAR中某一帧的数据，坐标原点为世界坐标系中的原点位置，从图中可以看出，点云中存储的位置坐标信息ｘｙｚ是基于该坐标原点的位置，而我们想得到的是基于 LiDAR 坐标系(下图中所画的坐标系)的位置信息，因此需要将加载到的点云数据进行处理。
除了LiDAR点之外，lidar属性还针对记录的每个LiDAR帧保留lidar.poses属性，即世界坐标系中的传感器姿态。
# 序列002中第15帧的传感器姿态 pose = seq002.lidar.poses print(pose[15]) """ {'position': {'x': 0.40108437325812535, 'y': 10.644695031495647, 'z': 0.024284914159960855}, 'heading': {'w': 0.9995436171415057, 'x': -0.00202975597147578, 'y': 0.008319329500030677, 'z': -0.028969402462612804}} """我们可以利用lidar.poses进行坐标系转换
import python.pandaset as pandaset from python.pandaset import geometry # load dataset dataset = pandaset.DataSet("/media/datasets/pandaset") seq002 = dataset["002"] seq002.load_lidar().load_semseg() seq_idx = 40 # get Pandar64 points seq002.lidar.set_sensor(0) pandar64_points = seq002.lidar[seq_idx].to_numpy() pc0 = seq002.lidar[0].to_numpy() poses = seq002.lidar.poses[seq_idx] # ego_pandar64_points中所存储的坐标信息即为转换后的坐标信息 ego_pandar64_points = geometry.lidar_points_to_ego(pandar64_points[:, :3], poses)获取方式和点云数据类似，采用 seq002.cuboids方法
cuboids0 = seq002.cuboids[0] # Returns the cuboid annotations for the first LiDAR frame in the sequence print(cuboids0.columns) """ Index(['uuid', 'label', 'yaw', 'stationary', 'camera_used', 'position.x', 'position.y', 'position.z', 'dimensions.x', 'dimensions.y', 'dimensions.z', 'attributes.object_motion', 'cuboids.sibling_id', 'cuboids.sensor_id', 'attributes.rider_status', 'attributes.pedestrian_behavior', 'attributes.pedestrian_age'], dtype='object') """ cuboids0 = seq002.cuboids[0] for i, row in cuboids0.iterrows(): print(row) break """ uuid 33d21162-44d7-40c5-aa77-b53b91ca2bf7 label Car yaw -0.0541471 stationary False camera_used 4 position.x -1.222 position.y -23.322 position.z 0.42 dimensions.x 1.869 dimensions.y 4.09 dimensions.z 1.498 attributes.object_motion Moving cuboids.sibling_id - cuboids.sensor_id -1 attributes.pedestrian_behavior NaN attributes.pedestrian_age NaN attributes.rider_status NaN Name: 0, dtype: object """标签含义说明：
uuid: str Unique identifier for an object. If object is tracked within the sequence, the uuid stays the same on every frame.label: str Contains name of object class associated with drawn cuboid.yaw: str Rotation of cuboid around the z-axis. Given in radians from which the cuboid is rotated along the z-axis. 0 radians is equivalent to the direction of the vector (0, 1, 0). The vector points at the length-side. Rotation happens counter-clockwise, i.e., PI/2 is pointing in the same direction as the vector (-1, 0, 0).stationary: bool True if object is stationary in the whole scene, e.g., a parked car or traffic light. Otherwise False.camera_used: int Reference to the camera which was used to validate cuboid position in projection. If no camera was explicitly used, value is set to -1.position.x: float Position of the cuboid expressed as the center of the cuboid. Value is in world-coordinate system.position.y: float Position of the cuboid expressed as the center of the cuboid. Value is in world-coordinate system.position.z: float Position of the cuboid expressed as the center of the cuboid. Value is in world-coordinate system.dimensions.x: float The dimensions of the cuboid based on the world dimensions. Width of the cuboid from left to right.dimensions.y: float The dimensions of the cuboid based on the world dimensions. Length of the cuboid from front to back.dimensions.z: float The dimensions of the cuboid based on the world dimensions. Height of the cuboid from top to bottom.attributes.object_motion: str Values are Parked, Stopped or Moving. Set for cuboids with label values in Car Pickup Truck Medium-sized Truck Semi-truck Towed Object Motorcycle Other Vehicle - Construction Vehicle Other Vehicle - Uncommon Other Vehicle - Pedicab Emergency Vehicle Bus Personal Mobility Device Motorized Scooter Bicycle Train Trolley Tram / Subway attributes.rider_status: str Values are With Rider or Without Rider. Set for cuboids with label values in Motorcycle Personal Mobility Device Motorized Scooter Bicycle Animals - Other attributes.pedestrian_behavior: str Value are Sitting, Lying, Walking or Standing Set for cuboids with label values in Pedestrian Pedestrian with Objectattributes.pedestrian_age: str Value are Adult or Child (less than ~18 years old) Set for cuboids with label values in Pedestrian Pedestrian with Objectcuboids.sensor_id: int For the overlap area between mechanical 360° LiDAR and front-facing LiDAR, moving objects received two cuboids to compensate for synchronization differences of both sensors. If cuboid is in this overlapping area and moving, this value is either 0 (mechanical 360° LiDAR) or 1 (front-facing LiDAR). All other cuboids have value -1.cuboids.sibling_id: str For cuboids which have cuboids.sensor_id set to 0 or 1: this field stores the uuid of the sibling cuboid, i.e., measuring the same object in the overlap region, but with the other respective sensor.标签转换代码
需要注意的是:
标签信息中包含两个雷达的标签信息，对于重复区域会造成重复框问题，因此需要通过cuboids.sensor_id属性进行筛选；标签信息也需要完成坐标系转换。 cuboids0 = seq002.cuboids[0] poses = seq002.lidar.poses[0] def cuboids_to_boxes(cuboids0, poses): str_ret = '' numb_ob = 0 sensor1_num = 0 for i, row in cuboids0.iterrows(): # cuboids.sensor_id值为-1,0,1， # 对于两个雷达重复区域的框用0表示(mechanical 360° LiDAR)用1表示 (front-facing LiDAR)，其它区域用-1表示 sensor_id = row["cuboids.sensor_id"] if sensor_id == 1: sensor1_num += 1 continue # 坐标信息 box = row["position.x"], row["position.y"], row["position.z"], row["dimensions.x"], row["dimensions.y"], row["dimensions.z"], row["yaw"] # 将中心点，长宽高和航向角信息转变为８个顶点的信息 corners = geometry.center_box_to_corners(box) # 将８个顶点的坐标位置进行坐标系转换 rotate_corners = geometry.lidar_points_to_ego(corners, poses) # 将８个顶点在转换回 中心点，长宽高和航向角 s = rotate_corners p0 = [s[0][0], s[0][1], s[0][2]] p1 = [s[1][0], s[1][1], s[1][2]] p2 = [s[2][0], s[2][1], s[2][2]] p3 = [s[3][0], s[3][1], s[3][2]] p4 = [s[4][0], s[4][1], s[4][2]] p5 = [s[5][0], s[5][1], s[5][2]] p6 = [s[6][0], s[6][1], s[6][2]] p7 = [s[7][0], s[7][1], s[7][2]] x = (p0[0] + p1[0] + p2[0] + p3[0] + p4[0] + p5[0] + p6[0] + p7[0]) / 8 y = (p0[1] + p1[1] + p2[1] + p3[1] + p4[1] + p5[1] + p6[1] + p7[1]) / 8 z = (p0[2] + p1[2] + p2[2] + p3[2] + p4[2] + p5[2] + p6[2] + p7[2]) / 8 l = math.sqrt(math.pow((p1[0] - p0[0]), 2) + math.pow(p1[1] - p0[1], 2)) w = math.sqrt(math.pow((p3[0] - p0[0]), 2) + math.pow(p3[1] - p0[1], 2)) h = math.sqrt(math.pow(p4[2] - p0[2], 2)) sina = float((p0[0] - p1[0]) / l) cosa = float((p0[1] - p1[1]) / l) yaw = math.atan(sina / cosa) str = '{} {} {} {} {} {} {} {}\n'.format(row["label"], y, x, z, l, w, h, yaw) str_ret = str_ret + str numb_ob = numb_ob + 1 # print("sensor1_num:{}".format(sensor1_num)) return str_ret, numb_ob根据转换后的点云数据和标签，展示出的效果图如下：
之前提到，我们需要把基于世界坐标系的点云坐标转换成基于雷达坐标系的点云坐标，对于单个雷达这样做的目的是使得我们在设置点云检测范围时是以雷达为中心点去设置的，两个坐标系的转换可以看成是一个刚体变换。
所谓刚体变换就是只改变物体的空间位置(平移)和朝向(旋转)，而不改变其形状的变换（一种记忆方法，硬的东西：比如石头，你不能改变他的形状，只能把它旋转或者平移），可用两个变量来描述：正交单位旋转矩阵R，三维平移矢量T.
其实就是一个在世界坐标系依照顺序进行z，y，x的旋转，之后再平移的过程。 旋转矩阵和平移矢量
平移矢量我们很好理解，就是我们沿着各轴方向的平移量。 而旋转矩阵却不是很好理解，在这里我们做一个具体的说明， 旋转一共有三个自由度，即绕x,y,z旋转， 下面以动态图的形式展现绕三个轴旋转的区别：
绕X轴旋转 Roll:横滚 绕Y轴旋转 Pitch: 俯仰
绕Z轴旋转 Yaw: 偏航（航向） 根据旋转角度我们可以在各个方向上将旋转写成矩阵的形式，分别为Rz,Ry,Rx，如下图所示，而旋转矩阵即为三个自由度的旋转矩阵的乘积，即：R=RzRyRx
因此，新坐标矢量 = 旋转矩阵 x 原坐标矢量 + 平移矢量
通过上面的介绍，我们知道给出旋转矩阵和偏移量（即雷达外参）就可以进行坐标系转换了，然而有的雷达厂商给出的雷达外参并非是旋转矩阵和偏移量，而是四元数和偏移量，这个时候就需要我们将四元数转换成旋转矩阵。
下面的图介绍了什么是四元数，以及四元数和旋转矩阵的互转公式：
参考文章： https://www.cnblogs.com/21207-iHome/p/6894128.html https://blog.csdn.net/qq_15029743/article/details/90215104
