Estimate object velocities
Computer Vision Toolbox / Analysis & Enhancement
The Optical Flow block estimates the direction and speed of object motion between two images or between one video frame to another frame using either the HornSchunck or the LucasKanade method.
I/I1
— Image or video frameImage or video frame, specified as a scalar, vector, or matrix. If the
Compute optical flow between parameter is set to
Two images
, the name of this port changes to
I1.
Data Types: single
 double
 int8
 int16
 int32
 int64
 uint8
 uint16
 uint32
 uint64
 fixed point
I2
— ImageImage or video frame, specified as a scalar, vector, or matrix.
Data Types: single
 double
 int8
 int16
 int32
 int64
 uint8
 uint16
 uint32
 uint64
 fixed point
V^2
— Velocity magnitudesVelocity magnitudes, returned as a scalar, vector, or matrix.
Data Types: single
 double
 int8
 int16
 int32
 int64
 uint8
 uint16
 uint32
 uint64
 fixed point
V
— Velocity components in complex formVelocity components in complex form, specified as a scalar, vector, or matrix.
To enable this port, set the Velocity output parameter to
Horizontal and vertical components in complex
form
.
Data Types: single
 double
 int8
 int16
 int32
 int64
 uint8
 uint16
 uint32
 uint64
 fixed point
Main Tab
Method
— Optical flow calculation methodHornSchunck
(default)  LucasKanade
Select the method to use to calculate the optical flow. Options include
HornSchunck
or
LucasKanade
.
Compute optical flow between
— Compute optical flowCurrent frame and Nth frame back
(default)  Two images
Select how to compute the optical flow. Select Two images
to compute the optical flow between two images. Select Current frame and
Nth frame back
to compute the optical flow between two video frames
that are N frames apart.
To enable this parameter, set the:
Method parameter to
HornSchunck
Method parameter to
LucasKanade
and the Temporal gradient
filter to Difference filter [1
1]
N
— Number of frames1
(default)  scalarEnter a scalar value that represents the number of frames between the reference frame and the current frame.
To enable this parameter, set the Compute optical flow
between parameter to Current frame and Nth frame
back
.
Smoothness factor
— Smoothness factor1
(default)  positive scalarSpecify the smoothness factor. Enter a large positive scalar value for high relative motion between the two images or video frames. Enter a small positive scalar value for low relative motion.
To enable this parameter, set the Method parameter to
HornSchunck
.
Stop iterative solution
— Stop iterative solutionWhen maximum number of iterations is
reached
(default)  When velocity difference falls below threshold
 Whichever comes first
Specify the method to control when the block's iterative solution process stops. If
you want the process to stop when the velocity difference is below a certain threshold
value, select When velocity difference falls below threshold
.
If you want the process to stop after a certain number of iterations, choose
When maximum number of iterations is reached
. You can also
select Whichever comes first
.
To enable this parameter, set the Method parameter to
HornSchunck
.
Maximum number of iterations
— Maximum number of iterations10
(default)  scalarSpecify the maximum number of iterations for the block to perform.
To enable this parameter, set the Method parameter to
HornSchunck
and the Stop iterative
solution parameter to When maximum number of iterations is
reached
or Whichever comes first
.
Velocity output
— Optical flow outputMagnitudesquared
(default)  Horizontal and vertical components in complex
form
Specify how to output an optical flow. If you select
Magnitudesquared
, the block outputs an optical flow matrix
where each element is in the form $${u}^{2}+{v}^{2}$$. If you select Horizontal and vertical components in
complex form
, the block outputs the optical flow matrix where each
element is in the form $$u+jv$$.
Temporal gradient filter
— Filter used for temporal gradientDifference filter [1 1]
(default)  Derivative of Gaussian
Specify whether the block solves for u and v using a difference filter or a derivative of a Gaussian filter.
To enable this parameter, set the Method parameter to
LucasKanade
.
Number of input frames to buffer
— Number of input frames to buffer for smoothing3
(default)  scalarSpecify the number of input frames to buffer for smoothing. Use this parameter for temporal filter characteristics such as the standard deviation and number of filter coefficients.
To enable this parameter, set the Temporal gradient filter
parameter to Derivative of Gaussian
.
Standard deviation for image smoothing filter
— Standard deviation for image smoothing filter1.5
(default)  scalarSpecify the standard deviation for the image smoothing filter.
To enable this parameter, set the Temporal gradient filter
parameter to Derivative of Gaussian
.
Standard deviation for gradient smoothing filter
— Standard deviation for gradient smoothing filter1
(default)  scalarSpecify the standard deviation for the gradient smoothing filter.
To enable this parameter, set the Temporal gradient filter
parameter to Derivative of Gaussian
.
Discard normal flow estimates when constraint equation is illconditioned
— Discard normal flow estimatesSelect this parameter to set the motion vector to zero when the optical flow constraint equation is illconditioned.
To enable this parameter, set the Temporal gradient filter
parameter to Derivative of Gaussian
.
Output image corresponding to motion vectors (accounts for block delay)
— Output image corresponding to motion vectorsSelect this parameter to output the image that corresponds to the motion vector outputted by the block.
To enable this parameter, set the Temporal gradient filter
parameter to Derivative of Gaussian
.
Threshold for noise reduction
— Threshold for noise reduction0.0039
(default)  scalarSpecify a scalar value that determines the motion threshold between each image or video frame. The higher the number, the less small movements impact the optical flow calculation.
To enable this parameter, set the Method parameter to
LucasKanade
.
Data Types Tab
For details on the fixedpoint block parameters, see Specify FixedPoint Attributes for Blocks.
Data Types 

Multidimensional Signals 

VariableSize Signals 

To compute the optical flow between two images, you must solve this optical flow constraint equation:
$${I}_{x}u+{I}_{y}v+{I}_{t}=0$$
.
$${I}_{x}$$, $${I}_{y}$$, and $${I}_{t}$$ are the spatiotemporal image brightness derivatives.
u is the horizontal optical flow.
v is the vertical optical flow.
By assuming that the optical flow is smooth across the entire image, the HornSchunck method estimates a velocity field, $$[\begin{array}{cc}u& v{]}^{T}\end{array}$$, that minimizes this equation:
$$E={\displaystyle \iint ({I}_{x}u+{I}_{y}v}+{I}_{t}{)}^{2}dxdy+\alpha {\displaystyle \iint \left\{{\left(\frac{\partial u}{\partial x}\right)}^{2}+{\left(\frac{\partial u}{\partial y}\right)}^{2}+{\left(\frac{\partial v}{\partial x}\right)}^{2}+{\left(\frac{\partial v}{\partial y}\right)}^{2}\right\}}dxdy$$
.
In this equation, $$\frac{\partial u}{\partial x}$$ and $$\frac{\partial u}{\partial y}$$ are the spatial derivatives of the optical velocity component, u, and $$\alpha $$ scales the global smoothness term. The HornSchunck method minimizes the previous equation to obtain the velocity field, [u v], for each pixel in the image. This method is given by the following equations:
$$\begin{array}{l}{u}_{x,y}^{k+1}={\overline{u}}_{x,y}^{k}\frac{{I}_{x}[{I}_{x}{\overline{u}}^{k}{}_{x,y}+{I}_{y}{\overline{v}}^{k}{}_{x,y}+{I}_{t}]}{{\alpha}^{2}+{I}_{x}^{2}+{I}_{y}^{2}}\\ {v}_{x,y}^{k+1}={\overline{v}}_{x,y}^{k}\frac{{I}_{y}[{I}_{x}{\overline{u}}^{k}{}_{x,y}+{I}_{y}{\overline{v}}^{k}{}_{x,y}+{I}_{t}]}{{\alpha}^{2}+{I}_{x}^{2}+{I}_{y}^{2}}\end{array}$$
.
In these equations, $$\left[\begin{array}{cc}{u}_{x,y}^{k}& {v}_{x,y}^{k}\end{array}\right]$$ is the velocity estimate for the pixel at (x,y), and $$\left[\begin{array}{cc}{\overline{u}}_{x,y}^{k}& {\overline{v}}_{x,y}^{k}\end{array}\right]$$ is the neighborhood average of $$\left[\begin{array}{cc}{u}_{x,y}^{k}& {v}_{x,y}^{k}\end{array}\right]$$. For k = 0, the initial velocity is 0.
To solve u and v using the HornSchunck method:
Compute $${I}_{x}$$ and $${I}_{y}$$ by using the Sobel convolution kernel, $$\left[\begin{array}{ccc}1& 2& \begin{array}{ccc}\begin{array}{ccc}\begin{array}{ccc}1;& 0& 0\end{array}& 0;& 1\end{array}& 2& 1\end{array}\end{array}\right]$$, and its transposed form for each pixel in the first image.
Compute $${I}_{t}$$ between images 1 and 2 using the $$\left[\begin{array}{cc}1& 1\end{array}\right]$$ kernel.
Assume the previous velocity to be 0, and compute the average velocity for each pixel using $$\left[\begin{array}{ccc}0& 1& \begin{array}{ccc}0;& 1& \begin{array}{ccc}0& 1;& \begin{array}{ccc}0& 1& 0\end{array}\end{array}\end{array}\end{array}\right]$$ as a convolution kernel.
Iteratively solve for u and v.
To solve the optical flow constraint equation for u and v, the LucasKanade method divides the original image into smaller sections and assumes a constant velocity in each section. Then it performs a weighted, leastsquare fit of the optical flow constraint equation to a constant model for $${\left[\begin{array}{cc}u& v\end{array}\right]}^{T}$$ in each section $$\Omega $$. The method achieves this fit by minimizing this equation:
$$\sum _{x\in \Omega}{W}^{2}{[{I}_{x}u+{I}_{y}v+{I}_{t}]}^{2}$$
W is a window function that emphasizes the constraints at the center of each section. The solution to the minimization problem is
$$\left[\begin{array}{cc}{\displaystyle \sum {W}^{2}{I}_{x}^{2}}& {\displaystyle \sum {W}^{2}{I}_{x}{I}_{y}}\\ {\displaystyle \sum {W}^{2}{I}_{y}{I}_{x}}& {\displaystyle \sum {W}^{2}{I}_{y}^{2}}\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]=\left[\begin{array}{c}{\displaystyle \sum {W}^{2}{I}_{x}{I}_{t}}\\ {\displaystyle \sum {W}^{2}{I}_{y}{I}_{t}}\end{array}\right]$$
.
When you set the Temporal gradient filter to Difference
filter [1 1]
, u and v are
solved as follows:
Compute $${I}_{x}$$ and $${I}_{y}$$ using the kernel $$\left[\begin{array}{cccc}1& 8& 0& \begin{array}{cc}8& 1\end{array}\end{array}\right]/12$$ and its transposed form.
If you are working with fixedpoint data types, the kernel values are signed fixedpoint values with a word length equal to 16 and a fraction length equal to 15.
Compute $${I}_{t}$$ between images 1 and 2 by using the $$\left[\begin{array}{cc}1& 1\end{array}\right]$$ kernel.
Smooth the gradient components, $${I}_{x}$$, $${I}_{y}$$, and $${I}_{t}$$, by using a separable and isotropic 5by5 element kernel whose effective 1D coefficients are $$\left[\begin{array}{cccc}\begin{array}{cc}1& 4\end{array}& 6& 4& 1\end{array}\right]/16$$. If you are working with fixedpoint data types, the kernel values are unsigned fixedpoint values with a word length equal to 8 and a fraction length equal to 7.
Solve the 2by2 linear equations for each pixel using the following method:
If $$A=\left[\begin{array}{cc}a& b\\ b& c\end{array}\right]=\left[\begin{array}{cc}{\displaystyle \sum {W}^{2}{I}_{x}^{2}}& {\displaystyle \sum {W}^{2}{I}_{x}{I}_{y}}\\ {\displaystyle \sum {W}^{2}{I}_{y}{I}_{x}}& {\displaystyle \sum {W}^{2}{I}_{y}^{2}}\end{array}\right]$$
then the eigenvalues of A are $${\lambda}_{i}=\frac{a+c}{2}\pm \frac{\sqrt{4{b}^{2}+{(ac)}^{2}}}{2};i=1,2$$
In the fixedpoint diagrams, $$P=\frac{a+c}{2},Q=\frac{\sqrt{4{b}^{2}+{(ac)}^{2}}}{2}$$
The eigenvalues are compared to the threshold, $$\tau $$, that corresponds to the value you enter for the threshold for noise reduction. The results fall into one of the following cases.
Case 1: $${\lambda}_{1}\ge \tau $$ and $${\lambda}_{2}\ge \tau $$
A is nonsingular, and the system of equations is solved using Cramer's rule.
Case 2: $${\lambda}_{1}\ge \tau $$ and $${\lambda}_{2}<\tau $$
A is singular (noninvertible), and the gradient flow is normalized to calculate u and v.
Case 3: $${\lambda}_{1}<\tau $$ and $${\lambda}_{2}<\tau $$
The optical flow, u and v, is 0.
If you set the temporal gradient filter to Derivative of
Gaussian
, u and v are solved using
these steps.
Compute $${I}_{x}$$ and $${I}_{y}$$.
Use a Gaussian filter to perform temporal filtering. Specify the temporal filter characteristics, such as the standard deviation and number of filter coefficients, by using the Number of frames to buffer for temporal smoothing parameter.
Use a Gaussian filter and the derivative of a Gaussian filter to smooth the image by using spatial filtering. Specify the standard deviation and length of the image smoothing filter by using the Standard deviation for image smoothing filter parameter.
Compute $${I}_{t}$$ between images 1 and 2.
Use the derivative of a Gaussian filter to perform temporal filtering. Specify the temporal filter characteristics, such as the standard deviation and number of filter coefficients, by using the Number of frames to buffer for temporal smoothing parameter.
Use the filter described in step 1b to perform spatial filtering on the output of the temporal filter.
Smooth the gradient components, $${I}_{x}$$, $${I}_{y}$$, and $${I}_{t}$$, by using a gradient smoothing filter. Use the Standard deviation for gradient smoothing filter parameter to specify the standard deviation and the number of filter coefficients for the gradient smoothing filter.
Solve the 2by2 linear equations for each pixel using this method:
If $$A=\left[\begin{array}{cc}a& b\\ b& c\end{array}\right]=\left[\begin{array}{cc}{\displaystyle \sum {W}^{2}{I}_{x}^{2}}& {\displaystyle \sum {W}^{2}{I}_{x}{I}_{y}}\\ {\displaystyle \sum {W}^{2}{I}_{y}{I}_{x}}& {\displaystyle \sum {W}^{2}{I}_{y}^{2}}\end{array}\right]$$
then the eigenvalues of A are $${\lambda}_{i}=\frac{a+c}{2}\pm \frac{\sqrt{4{b}^{2}+{(ac)}^{2}}}{2};i=1,2$$
When the block finds the eigenvalues, it compares them to the threshold, $$\tau $$, that corresponds to the value you enter for the Threshold for noise reduction parameter. The results fall into one of the following cases.
Case 1: $${\lambda}_{1}\ge \tau $$ and $${\lambda}_{2}\ge \tau $$
A is nonsingular, so the block solves the system of equations by using Cramer's rule.
Case 2: $${\lambda}_{1}\ge \tau $$ and $${\lambda}_{2}<\tau $$
A is singular (noninvertible), so the block normalizes the gradient flow to calculate u and v.
Case 3: $${\lambda}_{1}<\tau $$ and $${\lambda}_{2}<\tau $$
the optical flow, u and v, is 0.
Block Matching  Gaussian Pyramid  opticalFlow
 opticalFlowHS
 opticalFlowLK
 opticalFlowLKDoG
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.