bundleAdjustmentStructure

Refine 3-D points using structure-only bundle adjustment

Syntax

xyzRefinedPoints = bundleAdjustmentStructure(xyzPoints,pointTracks,cameraPoses,intrinsics)

[wpSetRefined,vSetRefined,pointIndex] = bundleAdjustmentStructure(wpSet,vSet,viewID,intrinsics)

[xyzRefinedPoints,reprojectionErrors] = bundleAdjustmentStructure(___)

[___] = bundleAdjustmentStructure(___,Name,Value)

Description

xyzRefinedPoints = bundleAdjustmentStructure(xyzPoints,pointTracks,cameraPoses,intrinsics) returns the refined 3-D points that minimize reprojection errors.

The structure-only refinement procedure is a special case of bundle adjustment with camera poses fixed during optimization. The 3-D points and the camera poses are placed in the same world coordinate system.

[wpSetRefined,vSetRefined,pointIndex] = bundleAdjustmentStructure(wpSet,vSet,viewID,intrinsics) refines 3-D points from the world point set, wpSet, and refines camera poses from the image view set, vSet. viewID specifies the IDs if the views in vSet.

[xyzRefinedPoints,reprojectionErrors] = bundleAdjustmentStructure(___) additionally returns an N-element vector containing the mean reprojection error for each 3-D world point using the arguments from the previous syntax.

[___] = bundleAdjustmentStructure(___,Name,Value) uses additional options specified by one or more name-value arguments. Unspecified arguments have default values.

Examples

Refine 3-D Points from Single Camera

Load data for initialization.

data = load('globeBA.mat');

Refine the 3-D world points.

xyzRefinedPoints = bundleAdjustmentStructure(data.xyzPoints,data.pointTracks,data.cameraPoses,data.intrinsics);

Display the refined 3-D world points.

pcshow(xyzRefinedPoints,AxesVisibility="on",VerticalAxis="y",VerticalAxisDir="down",MarkerSize=45);

Display the cameras.

hold on
plotCamera(data.cameraPoses,Size=0.2);
hold off
grid on

Input Arguments

xyzPoints — Unrefined 3-D points

M-by-3 matrix

Unrefined 3-D points, specified as an M-by-3 matrix of [x,y,z] locations.

Data Types: single | double

pointTracks — Matched points

N-element array of pointTracks objects

Matched points across multiple images, specified as an N-element array of pointTracks objects.

cameraPoses — Camera poses

table

Camera poses, specified as a table containing the two columns ViewId and AbsolutePose. You can get the cameraPoses table using the poses object function of the imageviewset object.

wpSet — 3-D world points

worldpointset object

3-D world points, specified as a worldpointset object.

vSet — Camera poses

imageviewset object

Camera poses, specified as an imageviewset object.

viewID — View identifiers of the views in vSet

N-element array

View identifiers of the views in vSet, specified as an N-element array. The viewIDs represent which camera poses to refine specifying their related views in imageviewset.

intrinsics — Camera intrinsics

cameraIntrinsics object | N-element array of cameraIntrinsics objects

Camera intrinsics, specified as a cameraIntrinsics object or an N-element array of cameraIntrinsics objects. N is the number of camera poses or the number of IDs in viewIDs. Use a single cameraIntrinsics object when images are captured using the same camera. Use a vector cameraIntrinsics objects when images are captured by different cameras.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: MaxIterations=50 sets the number of iterations to 50.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

MaxIterations — Maximum number of iterations

50 (default) | positive integer

Maximum number of iterations before the Levenberg-Marquardt algorithm stops, specified as a positive integer.

AbsoluteTolerance — Absolute termination tolerance

1.0 (default) | positive scalar

Absolute termination tolerance of the mean squared reprojection error in pixels, specified as a positive scalar.

RelativeTolerance — Relative termination tolerance

1e-5 (default) | positive scalar

Relative termination tolerance of the reduction in reprojection error between iterations, specified as a positive scalar.

PointsUndistorted — Flag to indicate lens distortion

false (default) | true

Flag to indicate lens distortion, specified as false or true. When you set PointsUndistorted to false, the 2-D points in pointTracks or in vSet must be from images with lens distortion. To use undistorted points, first use the undistortImage function to remove distortions from the images, then set PointsUndistorted.

Verbose — Display progress information

False (default) | true

Display progress information, specified as false or true.

Output Arguments

xyzRefinedPoints — 3-D locations of refined world points

M-by-3 matrix

3-D locations of refined world points, returned as an M-by-3 matrix of [x,y, z] locations.

Data Types: single | double

wpSetRefined — Refined 3-D world points

worldpointset object

Refined 3-D world points, returned as a worldpointset object.

vSetRefined — Refined camera poses

imageviewset object

Refined camera poses, returned as a imageviewset object.

pointIndex — Indices of refined 3-D world points

M-element array

Indices to wpSetRefined of refined 3-D world points, returned as an M-element array.

reprojectionErrors — Reprojection errors

M-element vector

Reprojection errors, returned as an M-element vector. The function projects each world point back into each camera. Then in each image, the function calculates the reprojection error as the distance between the detected and the reprojected point. The reprojectionErrors vector contains the average reprojection error for each world point.

References

[1] Lourakis, Manolis I. A., and Antonis A. Argyros. "SBA: A Software Package for Generic Sparse Bundle Adjustment." ACM Transactions on Mathematical Software 36, no. 1 (March 2009): 2:1–2:30.

[2] Hartley, Richard, and Andrew Zisserman. Multiple View Geometry in Computer Vision. 2nd ed. Cambridge, UK ; New York: Cambridge University Press, 2003.

[3] Triggs, Bill, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. "Bundle Adjustment — A Modern Synthesis." In Proceedings of the International Workshop on Vision Algorithms, 298–372. Springer-Verlag, 1999.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

Supports code generation in only host target platforms.

Version History

Introduced in R2020a

R2022b: Supports premultiply geometric transformation convention

Starting in R2022b, most Computer Vision Toolbox™ functions create and perform geometric transformations using the premultiply convention. Accordingly, the bundleAdjustmentStructure function now accepts cameraPoses as a table whose AbsolutePose column stores a rigidtform3d object, which uses the premultiply convention. This format is consistent with poses returned by the poses function of the imageviewset object.

Although you can still specify cameraPoses as a table whose AbsolutePose column stores a rigid3d object, this object is not recommended because it uses the postmultiply convention. You can streamline your geometric transformation workflows by switching to the premultiply geometric transformation convention. For more information, see Migrate Geometric Transformations to Premultiply Convention.