opticalFlowHS

Object for estimating optical flow using Horn-Schunck method

Description

Create an optical flow object for estimating the direction and speed of a moving object using the Horn-Schunck method. Use the object function estimateFlow to estimate the optical flow vectors. Using the reset object function, you can reset the internal state of the optical flow object.

Creation

Syntax

opticFlow = opticalFlowHS

opticFlow = opticalFlowHS(Name,Value)

Description

opticFlow = opticalFlowHS returns an optical flow object that you can use to estimate the direction and speed of the moving objects in a video. The optical flow is estimated using the Horn-Schunck method.

opticFlow = opticalFlowHS(Name,Value) returns an optical flow object with properties specified as one or more Name,Value pair arguments. Any unspecified properties have default values. Enclose each property name in quotes.

For example, opticalFlowHS('Smoothness',1.5)

Properties

Smoothness — Expected smoothness

1 (default) | positive scalar

Expected smoothness of optical flow, specified as a positive scalar. Increase this value when there is increased motion between consecutive frames. A typical value for 'Smoothness' is around 1.

MaxIteration — Maximum number of iterations

10 (default) | positive integer-valued scalar

Maximum number of iterations, specified as a positive integer-valued scalar. Increase this value to estimate the optical flow of objects with low velocity.

The iterative computation stops when the number of iterations equals the value of 'MaxIteration' or when the algorithm reaches the value set for 'VelocityDifference'. To stop computation only by using 'MaxIteration', set the value of 'VelocityDifference' to 0.

VelocityDifference — Minimum absolute velocity difference

0 (default) | positive scalar

Minimum absolute velocity difference, specified as a positive scalar. This value depends on the input data type. Decrease this value to estimate the optical flow of objects that have low velocity.

The iterative computation stops when the algorithm reaches the value set for 'VelocityDifference' or the number of iterations equals 'MaxIteration'. To use only 'VelocityDifference' to stop computation, set 'MaxIteration' to Inf.

Object Functions

estimateFlow	Estimate optical flow
reset	Reset the internal state of the optical flow estimation object

Examples

Estimate Optical Flow Using Horn-Schunck Method

Create a VideoReader object for the input video file, visiontraffic.avi. Specify the timestamp of the frame to read as 11.

vidReader = VideoReader('visiontraffic.avi','CurrentTime',11);

Specify the optical flow estimation method as opticalFlowHS. The output is an object specifying the optical flow estimation method and its properties.

opticFlow = opticalFlowHS

opticFlow = 
  opticalFlowHS with properties:

            Smoothness: 1
          MaxIteration: 10
    VelocityDifference: 0

Create a custom figure window to visualize the optical flow vectors.

h = figure;
movegui(h);
hViewPanel = uipanel(h,'Position',[0 0 1 1],'Title','Plot of Optical Flow Vectors');
hPlot = axes(hViewPanel);

Read image frames from the VideoReader object and convert to grayscale images. Estimate the optical flow from consecutive image frames. Display the current image frame and plot the optical flow vectors as quiver plot.

while hasFrame(vidReader)
    frameRGB = readFrame(vidReader);
    frameGray = im2gray(frameRGB);  
    flow = estimateFlow(opticFlow,frameGray);
    imshow(frameRGB)
    hold on
    plot(flow,'DecimationFactor',[5 5],'ScaleFactor',60,'Parent',hPlot);
    hold off
    pause(10^-3)
end

Algorithms

To compute the optical flow between two images, you must solve this optical flow constraint equation:

$$I_{x}u+I_{y}v+I_t=0$$

.

• $$I_x$$, $$I_y$$, and $$I_t$$ are the spatiotemporal image brightness derivatives.

• u is the horizontal optical flow.

• v is the vertical optical flow.

Horn-Schunck Method

By assuming that the optical flow is smooth across the entire image, the Horn-Schunck method estimates a velocity field, $$[\begin{array}{cc}u& v{]}^T\end{array}$$, that minimizes this equation:

$$E={\displaystyle \iint (I_{x}u+I_{y}v}+I_t{)}^{2}dxdy+\alpha {\displaystyle \iint \left\{{\left(\frac{\partial u}{\partial x}\right)}^2+{\left(\frac{\partial u}{\partial y}\right)}^2+{\left(\frac{\partial v}{\partial x}\right)}^2+{\left(\frac{\partial v}{\partial y}\right)}^2\right\}}dxdy$$

.

In this equation, $$\frac{\partial u}{\partial x}$$ and $$\frac{\partial u}{\partial y}$$ are the spatial derivatives of the optical velocity component, u, and $$\alpha$$ scales the global smoothness term. The Horn-Schunck method minimizes the previous equation to obtain the velocity field, [u v], for each pixel in the image. This method is given by the following equations:

$$\begin{array}{l}{u}_{x,y}^{k+1}={\overline{u}}_{x,y}^k-\frac{I_x[I_x{\overline{u}}^k{}_{x,y}+I_y{\overline{v}}^k{}_{x,y}+I_t]}{{\alpha }^2+I_x^2+I_y^2}\\ v_{x,y}^{k+1}={\overline{v}}_{x,y}^k-\frac{I_y[I_x{\overline{u}}^k{}_{x,y}+I_y{\overline{v}}^k{}_{x,y}+I_t]}{{\alpha }^2+I_x^2+I_y^2}\end{array}$$

.

In these equations, $$\left[\begin{array}{cc}{u}_{x,y}^k& v_{x,y}^k\end{array}\right]$$ is the velocity estimate for the pixel at (x,y), and $$\left[\begin{array}{cc}{\overline{u}}_{x,y}^k& {\overline{v}}_{x,y}^k\end{array}\right]$$ is the neighborhood average of $$\left[\begin{array}{cc}{u}_{x,y}^k& v_{x,y}^k\end{array}\right]$$. For k = 0, the initial velocity is 0.

To solve u and v using the Horn-Schunck method:

1. Compute $$I_x$$ and $$I_y$$ by using the Sobel convolution kernel, $$\left[\begin{array}{ccc}-1& -2& \begin{array}{ccc}\begin{array}{ccc}\begin{array}{ccc}-1;& 0& 0\end{array}& 0;& 1\end{array}& 2& 1\end{array}\end{array}\right]$$, and its transposed form for each pixel in the first image.

2. Compute $$I_t$$ between images 1 and 2 using the $$\left[\begin{array}{cc}-1& 1\end{array}\right]$$ kernel.

3. Assume the previous velocity to be 0, and compute the average velocity for each pixel using $$\left[\begin{array}{ccc}0& 1& \begin{array}{ccc}0;& 1& \begin{array}{ccc}0& 1;& \begin{array}{ccc}0& 1& 0\end{array}\end{array}\end{array}\end{array}\right]$$ as a convolution kernel.

4. Iteratively solve for u and v.

References

[1] Barron, J. L., D. J. Fleet, S. S. Beauchemin, and T. A. Burkitt. “ Performance of optical flow techniques.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),236-242. Champaign, IL: CVPR, 1992.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

Version History

Introduced in R2015a