What Is a Point Cloud?
A point cloud is a collection of points in two-dimensional or three-dimensional space that represents the shape and location of objects, surfaces, or an environment. In a 3D point cloud, each point has an X, Y, and Z coordinate. Together, the points create a digital representation of the captured scene or object.
A point cloud can also store information beyond position, such as color, intensity, surface normals, timestamps, return information, or semantic labels. Point clouds are generated by sensors and computational workflows including lidar, RGB-D and time-of-flight cameras, stereo vision, mmWave radar, photogrammetry, structure from motion, and visual SLAM.
Engineers and researchers process point clouds to measure geometry, align scans, identify objects, reconstruct surfaces, build maps, analyze terrain, and develop perception and navigation systems.
How Are Point Clouds Created?
Point clouds can be measured directly by range sensors or computed from images and other data. The source affects point density, organization, attributes, noise, field of view, and coordinate system.
| Source | How the point cloud is produced |
|---|---|
| Lidars and laser scanners | Measures distance with laser light and produces spatial measurements that can form 2D scans or 3D point clouds. |
| RGB-D and time-of-flight cameras | Capture image and depth information, commonly producing organized point clouds. |
| Stereo cameras | Estimate depth from corresponding features in two or more camera views. This can then be converted into point clouds |
| mmWave radar | Can produce a point representation after radar signal processing, with attributes such as radial velocity depending on the system. |
| Photogrammetry and structure from motion | Reconstruct 3D points from overlapping images captured from different viewpoints. |
| Visual SLAM and 3D reconstruction | Generate point clouds as part of estimating camera motion and reconstructing an environment. |
| Simulation | Produces synthetic point clouds for algorithm development and testing. |
Organized and Unorganized Point Clouds
Point clouds can be represented as organized or unorganized data. The representation affects how adjacency is stored and which processing approaches are convenient.
Organized Point Clouds
An organized point cloud is arranged in rows and columns, similar to an image. Its dimensions are commonly represented as M-by-N-by-C, where M is the number of rows, N is the number of columns, and C contains coordinate or point information. Because the organization preserves a neighborhood relationship, algorithms can use the structure of adjacent measurements.
Unorganized Point Clouds
An unorganized point cloud is stored as a list of points rather than a row-and-column grid. Its dimensions are commonly represented as M-by-C, where M is the number of points and C contains coordinate or point information. Many 3D lidar scanners produce unorganized point clouds.
An unorganized point cloud can sometimes be converted to an organized representation by projecting the points into a suitable grid. The resulting organization depends on the projection and data geometry.
What is Point Cloud Processing
Point cloud processing is the set of operations used to prepare, transform, align, analyze, and interpret point-cloud data. A workflow often begins with raw measurements or reconstructed points and ends with information, geometry, maps, models, or deployable perception algorithms.
Point clouds are used in a workflow which needs a spatial representation of an object, asset, or environment. Common applications include the following.
| Application | How point clouds contribute |
|---|---|
| Mapping and geospatial analysis | Create terrain models, elevation products, maps, and environmental measurements. |
| Autonomous vehicles and ADAS | Support perception, object detection, tracking, localization, and scene understanding. |
| Robotics and navigation | Build maps, estimate position, detect obstacles, and plan motion. |
| Architecture, engineering, and construction | Capture buildings and sites for measurement, planning, progress monitoring, and digital models. |
| Industrial inspection and metrology | Compare measured geometry with a reference model and identify dimensional differences. |
| Forestry, agriculture, and utilities | Analyze vegetation, terrain, infrastructure, and clearance around assets. |
| Mining and earth sciences | Measure terrain, excavations, stockpiles, surfaces, and geological environments. |
| Digital twins and simulation | Create spatial representations and geometry for virtual environments. |
| AR, VR, and 3D content | Reconstruct objects and environments for visualization and immersive applications. |
Common Workflows and Tasks with MATLAB
Read, write, and stream point clouds
MATLAB provides readers for popular file formats like pcd, ply, pcap, las/laz, and ibeo data container. You can also stream live lidar data from Velodyne and Ouster lidar sensors.
Create synthetic lidar data
Create synthetic lidar data that imitates actual lidar sensors and test your workflow before deploying in real-world systems.
Preprocess data
Apply preprocessing algorithms like downsample, median filter, transform, extract features from, and align 3D point clouds
Calibrate lidar cameras
Find the transformation between camera and lidar in your system. You can then use this transformation to project lidar data onto camera data and vice versa.
Perform object detection and semantic segmentation
Detect objects or segment point clouds using deep learning algorithms.
Build maps and localize vehicles
Perform registration, map building, and SLAM using series of point clouds.
Deploy on CPUs and GPUs
You can deploy lidar processing workflows on your target hardware as C/C++ or CUDA codes
Live streaming lidar data from Ouster lidar sensors. Connect to lidar sensors and stream live lidar data into MATLAB.
Object tracking on point cloud data sequence. Detect, classify, and track vehicles by using sequential lidar data captured by a lidar sensor mounted on an ego vehicle.
Object tracking on point cloud data sequence. Detect, classify, and track vehicles by using sequential lidar data captured by a lidar sensor mounted on an ego vehicle.
Semantic segmentation of point clouds using SqueezeSegV2. Organized lidar data is semantically segmented into car (red), truck (purple), and background (black).(See MATLAB example)
For details, see Point Cloud Toolbox™ and Computer Vision Toolbox™.
Examples and How To
Point Cloud FAQs
A point cloud is a collection of data points in 3D space, where each point represents the X-, Y-, and Z-coordinates of a location on a real-world object’s surface, and the points collectively map the entire surface.
Point clouds are commonly produced by lidar scanners, stereo cameras, and time-of-flight cameras that capture spatial coordinates of surfaces in real-world environments.
Organized point clouds are structured into rows and columns like image data (M x N x C format) and are typically created by stereo cameras and time-of-flight cameras, while unorganized point clouds have no row-column structure (M x C format) and are typically produced by lidar sensors.
Point cloud processing is used for perception and navigation in robotics and autonomous systems, as well as in augmented reality (AR) and virtual reality (VR) applications.
Yes, Lidar Toolbox and Computer Vision Toolbox provide tools and reference applications that support point cloud processing, including capabilities for reading, writing, streaming, preprocessing, object detection, semantic segmentation, and SLAM.
MATLAB provides readers for popular point cloud file formats including pcd, ply, pcap, las/laz, and ibeo data container.
Yes, you can stream live lidar data from Velodyne and Ouster lidar sensors directly into MATLAB.
MATLAB supports preprocessing algorithms like downsample, median filter, transform, feature extraction, and alignment of 3D point clouds.
See also: 3D image processing, affine transformation, digital image processing, image analysis, image processing and computer vision, image reconstruction, image registration, image segmentation, image thresholding, image transform, object detection, RANSAC, stereo vision, SLAM (simultaneous localization and mapping), drone mapping
