An Improved Fractional-Order Variational Optical Flow Model Combining Structure Tensor

Zhu, Bin; Wang, Zhaodong; Tian, Lianfang; Guo, Jinmei; Wang, Lingjian; Bhutto, Jameel

doi:https://doi.org/10.1155/2021/9960784

Mathematical Problems in Engineering

On this page

Abstract Introduction Related Work Analysis Conclusion Data Availability Additional Points Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 9960784 | https://doi.org/10.1155/2021/9960784

An Improved Fractional-Order Variational Optical Flow Model Combining Structure Tensor

Bin Zhu,¹Zhaodong Wang,²Lianfang Tian,^3,4Jinmei Guo,¹Lingjian Wang,¹and Jameel Bhutto³

Academic Editor: Erik Cuevas

Received28 Mar 2021

Revised12 Jul 2021

Accepted20 Jul 2021

Published27 Jul 2021

Abstract

Dealing with problems of illumination changes in optical flow estimation, an improved variational optical flow model is proposed in this paper. The local structure constancy constraint (LSCC) is applied in the data term of the traditional HS (Horn & Schunck) optical flow model to substitute the brightness constancy constraint. The fractional-order smoothness constraint (FSC) is applied in the smoothness term of the HS model. Then, the detailed calculation processes from the optical flow model to the optical flow value are explained. The structure tensor in LSCC is an image feature that is constant in the illumination changes scene. The fractional differential coefficient in FSC can fuse the local neighborhood optical flow vector into the optical flow vector of the target pixel, which can improve the integrity of the motion region with the same motion speed. Combining LSCC with FSC, our improved optical flow model can obtain an accurate optical flow field with clear outline in the illumination abnormity scene. The experimental results show that, compared with other optical flow models, our model is more suitable for the illumination changes scene and can be employed in outdoor motion detection projects.

1. Introduction

Motion detection in the image sequence or video is one of the basic tasks of various image processing projects, which has been widely used in the research and engineering practice of 3D reconstruction, moving object segmentation, moving object tracking, video compression, automatic driving, and so on. The variational optical flow method is one of the most commonly used methods of motion detection. The HS optical flow model [1] is the most classical variational optical flow model. It is composed of a brightness constancy constraint and global smoothness constraint. The brightness constancy constraint requires that the intensity of pixels remains constant in the process of motion, while the smoothness constraint supposes that the motion speed of all pixels in the image changes smoothly. However, both constraints are based on ideal assumptions. In practical applications, these two constraints are often violated. Illumination changes in the scene will break the brightness constant constraint and the global smoothness constraint and lead to the blur of the edge between different motion regions.

Several other constraints on the data term were proposed to improve the performance of the variational optical flow model, such as gradient constancy constraint [2], Laplacian constancy constraint [3], and Hessian constancy constraint [4], but these constraints depend heavily on the intensity difference. Illumination invariant descriptors such as binary-based [5], real value-based [6, 7], and neighborhood-based [8, 9] were applied to substitute the brightness constancy constraint. But these descriptors fail to identify the edge between different motion regions. The structure tensor [10–12], which can embody the shape feature of the motion objective, is a local structure feature that does not change with the environmental factor. The local structure optical flow model (LSOFM) [13–15], which is established on the structure tensor feature, can improve the robustness in the illumination changes scene. But when a structure includes different motion parts, LSOFM would fail.

Because the fractional optical flow vector gradient can fuse the characteristics of the local neighborhood optical flow vector into the optical flow vector of the target pixel, the integrity of regions with the same motion speed is enhanced, so it can preserve the discontinuity of the optical flow field between different motion regions. The fractional-order variational optical flow model (FOVOFM) is proposed by Chen et al. [16], which uses the fractional-order smoothness constraint in the smoothness term of the HS model. The dual fractional order variational optical flow model (DFOVOFM) is proposed by Zhu et al. [17], which the fractional-order derivative is applied both in data term and smoothness term of the HS model. The fractional-order TV-L1 optical flow model was proposed in [18]. The fractional-order-based optical flow model neglects the structural information in image, so error would exist on the edge and some tiny motion regions.

An improved variational optical flow model is proposed in this paper. Our model is composed of LSCC and FSC. LSCC cannot only ensure the robustness of the model in scenes of illumination changes but also the contour in the image can be enhanced. FSC cannot only preserve the motion discontinuity between different motion regions but also weaken the influence of illumination changes on optical flow estimation. The model proposed in this paper combines the advantages of the two constraints and solves the illumination change problem and the motion discontinuity problem in optical flow estimation, and different motion parts in a structure can be identified.

The contribution of the paper is as follows:(1)A novel optical flow model combining local structure tensor and fractional-order deviation is designed to improve the robustness of illumination changes(2)An applicable model simplification method for our model is presented

The research of the optical flow model never stops. An optical flow model using full four-dimensional cost volume was proposed in [19], which applied structural features and stereo matching pipelines to yield high accuracy. To solve the problems caused by untextured areas, motion boundaries, and occlusions in optical flow estimation, post-hoc confidence measures [20] are applied to assess the per-pixel reliability of the flow. Occlusions have remained one of the key challenges. In [21], a symmetric optical flow model was proposed to address occlusion problems of optical flow estimation. In [22], an interpolation correspondence method is proposed to estimate the optical flow, but these methods are usually susceptible to nonrigid motion and large displacements; to solve this problem, robust interpolation of sparse correspondences is proposed in [23].

A coarse-to-fine scheme combined with an efficient random search strategy for efficient dense correspondence is proposed to estimate the optical flow in [24]. A coarse-to-fine PatchMatch with sparse seeds to sparse matches is applied to optical flow estimation in [25]. A novel differentiable neighbor-search up-sampling (NSU) module [26] is proposed to improve the coarse-to-fine technology for avoiding the optical flow error in edges, thin bars, and holes. A novel optical flow method related to the frequency domain that uses TV-wavelet regularization is applied to accurate optical flow estimation with the frequency-domain regularization in [27].

With the rapid development of artificial intelligence, deep learning-based methods have been widely employed in optical flow estimation. The FlowNet [28] demonstrated that CNNs can be used to estimate the optical flow. In [29], three major improvements were designed for the FlowNet: a subnetwork was elaborated on small displacements, a stacked architecture was applied to warp of the second image with intermediate optical flow, and a schedule of the training data. A new deep network architecture called recurrent all-pairs field transforms is designed to estimate the optical flow in [30]. A deformable cost volume neural network which can estimate the multiscale optical flow in a single high resolution was proposed in [31]. PWC-Net [32], which consists of pyramidal processing, warping, and a cost volume, is designed to estimate optical flow. The network uses the current optical flow estimate to warp the CNN features of the second image and then uses the warped features and features of the first image to construct a cost volume. The drawback of deep learning-based methods is large number of ground truth are needed, and retraining the model is needed for different application scenarios.

3. HS Optical Flow Model

The HS model can be written as follows:where represents the optical flow vector, is the optical flow value on axis, is the optical flow value on axis, represents the brightness constraint equation, represents smoothness constraint equation, and is a smoothness parameter, and its value is decided by the image quality. In a blurry image with many noises disturbing, the value of should be big and vice versa.

The brightness constraint equation based on the bright constancy constraint, which supposes that the brightness value of a pixel is constant during the motion, is as follows:

The smoothness constraint equation based on the global smoothness constraint, which supposes that the motion speed of all pixels changes smoothly, that is to say, the derivative of the speed is very small, is as follows:

4. Local Structure Constraint Equation

The structure tensor which is composed of local geometric features of image is an invariable feature to illumination changes, image rotation, image translation, and image scaling. To increase the robustness of the optical flow model, we use the local structure constancy constraint to substitute the brightness constant constraint.

4.1. Image Structure Tensor

The local structure tensor of the two-dimensional image can be expressed as follows:where represents the brightness function, is the tangent vector along axes, and is the structure tensor of a pixel. But is easily be affected by noise, so it is always enhanced by Gaussian filter :

4.2. Local Structure Constancy Constrain

The local structure constancy constraint (LSCC) is proposed in this paper to enhance the robustness of the optical flow model. LSCC is based on the assumption that the local structure in an image would not change during the motion:where represents the structure tensor function from a pixel to frame. Then, the pixel will move from to , and is the optical flow value, and represents the structure tensor of frame. Equation (6) means that the structure tensor will not change during the motion.

Using the Taylor series expansion and ignoring the minimum term, we can getwhere , , and are the derivative of the structure tensor function on axes, , , , , , , , represents the local neighborhood of the target point, is the normalization constant, and represents the weight function.

Then, can be rewritten as follows:

5. Fractional Smoothness Constraint Equation

The fractional smoothness constraint (FSC) is applied in our model to further improve the performance of optical flow estimation. We use the fractional-order derivative to substitute the integer-order derivative in the smoothness constraint equation of the HS optical flow model:where represents -order derivative of the optical flow vector on and axes, represents -order derivative of the optical flow vector on and axes, is real, , , and .

Then, can be designed as follows:

6. Improved Fractional Optical Flow Model

Combining LSCC with FSC, we construct our improved optical flow model; using the structure of the HS model, our optical flow model can be described as follows:

Using the Euler–Lagrange equation to minimize the energy function (11), a similar procedure can be seen in [14, 15]; then, we can getwhere , , , , , and are the second-order derivative of the structure tensor at , , and axes:where , , , and denote the dual fractional-order derivative of optical flow value and on and axes, respectively:where and , , , which denote the size of the neighborhood.

Then, we can getwhere represents a neighborhood of point , is equal to , and are the mean of the optical flow vector in the neighborhood, and and are the coordination of the optical flow vector on and axes of each pixel in the neighborhood.

Then, the optical flow estimation formula is as follows:where is the smoothness parameter, according to Huerst and Xie [33]. The value of parameter is determined by the image quality; when the image is dark, unidentified, blurry, or noisy, should be large. Otherwise, should be small. In our algorithm, we choose in relatively clear images, and in relatively blurry images. is the order of fractional derivation in the smoothness term, according to Chen et al. [15], and or . The accuracy in the case of is relatively higher than in the case of , while the latter case needs less iteration times.

We can obtain the accurate optical flow value by repeated iteration utilizing the Gauss–Seidel method. The iteration will stop when the accuracy reaches the requirement (the difference of the optical flow value between two adjacent iterations is less than the threshold).

7. Experimental Result and Analysis

To testify the superiority and practicability of our optical flow model, Middlebury dataset [34], MPI_Sintel dataset [35], KITTI dataset [36], and an outdoor real scene video are used in experiments. Vertical comparisons are employed to explain the model evolution process; these models include HS model [1], LSCOFM [15], FOVOFM [16], and DFOVOFM [17]. Three widely used high performance models are employed for horizontal comparison: HAST [37], MDP_Flow [38], and PH_Flow [39]. All experiments are run in a PC equipped with Intel Xeon(R) X5600, 3.4 GHZ, and 4 GB memory. The operating system is 64 byte Windows 7, and the software platform is Matlab 10.1.

Figures 1–3 show the color code map by optical flow estimation of different algorithms in the Middlebury database. They are army sequence, grove sequence, and scheflera sequence, respectively. It can be seen in these figures that, from left to right, we can get more and more clear outlines, which means the optical flow results are getting more and more accurate. But, compared to the GT, some highly-textured thin outlines disappear or become adjoined in the results of our model, such as soldier’s gun in army, the end of branches in Figure 2(e), and the rhizome parts of leaves in Figure 3(e). That is because using the fractional order derivative, the thin parts would be assimilated by the optical flow field of surrounding pixels. Table 1 shows the AEE/AAE of different algorithms in the Middlebury dataset. We can see that, from top to down, the error rate is becoming more and more small, and the error rate of our model is the lowest.

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

Figures 4 and 5 show the color code map by optical flow estimation of different algorithms in the MPI_Sintel dataset. They are Cave3 sequence and Shaman1 sequence, respectively. The results presented show that our method can acquire clear and integral motion outlines under the low contrast scene (Cave3) and illumination insufficient scene (Shaman1). HAST can embody motion details such as the end of the cone in Cave3 and the boy’s hair in Shaman1, but fails in large nontextured regions such as the tail in Cave3 and the gusset parts in Shaman1. Similar situations are shown in PH_Flow and MDP_Flow. The deficiencies of our model also exist in the thin detail parts: the end of the cone in Cave3 and details of boy’s hair are missing in Shaman1. Table 2 shows the AEE/AAE of different algorithms in the MPI_Sintel dataset. We can see in Table 2 that, in sequences with existing large displacement and occlusion problems, the accuracy of our method is lower than its components. When comes to sequences with illumination changes and illumination insufficient, the accuracy of our method is higher than other algorithms.

(a)

(b)

(c)

(d)

(e)

(a)

(b)

(c)

(d)

(e)

The KITTI dataset is a challenging real-world dataset in which most sequences are affected by illumination abnormity, such as illumination changes, illumination uneven, illumination insufficient, and so on. The experimental results are shown in Figures 6–8. They are sequence 00004, 00009, and 00011, respectively. We can see that our method obtains clear motion outlines in all image sequences. Despite the illumination uneven conditions, the tree outline can be seen clearly in 00004 and the car outline can be seen clearly in 00009. In sequence 00011 with the illumination normal scene, differences between different algorithms are small. Table 3 shows the AEE/AAE of different algorithms in the KITTI dataset. We can see that, in illumination abnormity scenes, the error rate of our model is the lowest, and the average accuracy of our model for the whole sequences in KITTI is the highest.

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

Figures 9–12 show the color code map of optical flow estimation by different algorithms in an outdoor video. We can see that compared to LSOFM, DFOVOFM has less noise affected regions, but the outline is not so clear. Our model achieves better results with the most intact object outline and least noise affected regions than other models. In Figure 10(d), different motion parts in the right sole can be seen, the optical flow field of the first half foot is pink, and the second half is yellow. The motion of one finger in the right hand in Figures 11 and 12 is different from the other part on that hand, and we can see different color regions in Figure 12(d).

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

8. Conclusion

An improved variational optical flow model is proposed in this paper to solve problems of illumination changes in optical flow estimation. The model applies LSCC and FSC in the data term and smoothness term of the HS model. Then, the detailed calculation processes from the optical flow model to optical flow value are explained. Experimental results in three public datasets and an outdoor video show that our model is superior to other algorithms in illumination abnormity scenes. The disadvantage of our model is that it cannot embody the motion of thin parts. Experimental results testify the performance of our model. Finally, we have conclusions that our model is more suitable for the illumination changes scene and can be employed in outdoor motion detection projects.

Data Availability

Middlebury dataset, MPI_Sintel dataset, and KITTI dataset can be found in references [17–19]. Our own outdoor video can be easily obtained by a normal RGB camera.

Additional Points

The model proposed in this paper has some limitations:When the difference between the RGB value of background and foreground all less than “1,” the model would fail; because of the large amount of computation, the running speed of the algorithm is 30 seconds per frame, so it cannot be used for real-time application; when the average brightness is less than 5, the optical flow estimation result of the algorithm would not be accurate.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported by the Department of Natural Resources of Guangdong Province-Offshore Wind Power Project, 2020, Key-Area Research and Development Program of Guangdong Province, 2019B020214001, Key-Area Research and Development Program of Guangdong Province, 2018B010109 001, Guangzhou Industrial Technology Major Research Plan, 20190101121006001, and Fundamental Research Funds for the Central Universities, 2018KZ05.

References

B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, no. 1, pp. 185–203, 1981.
View at: Publisher Site | Google Scholar
Z. Tu, W. Xie, J. Cao et al., “Variational method for joint optical flow estimation and edge-aware image restoration,” Pattern Recognition, vol. 65, no. 3, pp. 11–25, 2016.
View at: Google Scholar
N. Monzón, A. Salgado, and J. Sánchez, “Regularization strategies for discontinuity-preserving optical flow methods,” IEEE Transactions on Image Processing, vol. 25, no. 4, pp. 1580–1591, 2016.
View at: Publisher Site | Google Scholar
C. Zhang, Z. Chen, M. Wang, M. Li, and S. Jiang, “Robust non-local TV- L1 optical flow estimation with occlusion detection,” IEEE Transactions on Image Processing, vol. 26, no. 8, pp. 4055–4067, 2017.
View at: Publisher Site | Google Scholar
M. A. Mohamed, H. A. Rashwan, B. Mertsching, M. A. Garcia, and D. Puig, “Illumination-robust optical flow using a local directional pattern,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 9, pp. 1499–1508, 2014.
View at: Publisher Site | Google Scholar
M. Drulea and S. Nedevschi, “Motion estimation using the correlation transform,” IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, vol. 22, no. 10, pp. 3260–3270, 2013.
View at: Publisher Site | Google Scholar
R. W. Liu, L. Shi, W. Huang, J. Xu, S. C. Yu, and D. Wang, “Generalized total variation-based MRI rician denoising model with spatially adaptive regularization parameters,” Magnetic Resonance Imaging, vol. 32, no. 6, pp. 702–720, 2014.
View at: Publisher Site | Google Scholar
S. Ali, C. Daul, E. Galbrun, and W. Blondel, “Illumination invariant optical flow using neighborhood descriptors,” Computer Vision and Image Understanding, vol. 145, no. 12, pp. 95–110, 2016.
View at: Publisher Site | Google Scholar
W. Lu, J. Duan, Z. Qiu, Z. Pan, R. W. Liu, and L. Bai, “Implementation of high-order variational models made easy for image processing,” Mathematical Methods in the Applied Sciences, vol. 39, no. 14, pp. 4208–4233, 2016.
View at: Publisher Site | Google Scholar
M. Andersson and H. Knutsson, “Transformation of local spatio-temporal structure tensor fields,” in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pp. 25–31, IEEE, Hong Kong, China, April 2003.
View at: Publisher Site | Google Scholar
W. Förstner and E. Gülch, “A fast operator for detection and precise location of distinct points, corners and centres of circular features,” in International Commission Conference on Fast Processing of Photogrammetric Data, pp. 281–305, Springer, Interlaken, Switzerland, June 1987.
View at: Google Scholar
J. Shi and C. Tomasi, “Good features to track,” in Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 357–365, IEEE, Seattle, WA, USA, June 1994.
View at: Publisher Site | Google Scholar
Z. Q. Lu, X. Xie, and J. H. Pei, “A robust optical flow computation,” Journal of Electronics, vol. 5, no. 3, pp. 635–641, 2007.
View at: Publisher Site | Google Scholar
A. Bruhn, J. Weickert, and C. Schnoerr, “Lucas/kanade meets horn/schunck: combining local and global optic flow methods,” International Journal of Computer Vision, vol. 61, no. 3, pp. 211–231, 2005.
View at: Publisher Site | Google Scholar
Z. Chen, C. Zhang, X. Zhang, and M. Li, “Coarse-to-fine optical flow estimation with image structure tensor,” in Proceedings of the International Congress on Image and Signal Processing, pp. 478–485, IEEE, Hangzhou, China, December 2013.
View at: Publisher Site | Google Scholar
D. Chen, H. Sheng, Y. Chen, and D Xue, “Fractional-order variational optical flow model for motion estimation,” Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, vol. 371, Article ID 20120148, 2013.
View at: Publisher Site | Google Scholar
B. Zhu, L. F. Tian, Q. L. Du et al., “An Improved Fractional order optical flow model for motion estimation,” Mathematical Problems in Engineering, vol. 2018, Article ID 6278719, 6 pages, 2018.
View at: Publisher Site | Google Scholar
S. G. Bardeji, I. N. Figueiredo, and E. Sousa, “Optical flow with fractional order regularization: variational model and solution method,” in Proceedings of the 5th International Workshop on Analysis and Numerical Approximation of Singular Problems, pp. 188–200, Lagos, Portugal, April 2017.
View at: Google Scholar
J. Xu, R. Ranftl, and V. Koltun, “Accurate optical flow via direct cost volume processing,” in Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 5807–5815, Honolulu, HI, USA, June 2017.
View at: Publisher Site | Google Scholar
AS. Wannenwetsch, M. Keuper, and S. Roth, “Probflow: joint optical flow and uncertainty estimation,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 1182–1191, Venice, Italy, October 2017.
View at: Publisher Site | Google Scholar
J. Hur and S. Roth, “Mirrorflow: exploiting symmetries in joint optical flow and occlusion estimation,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 312–321, Venice, Italy, October 2017.
View at: Publisher Site | Google Scholar
Y. Hu, Y. Li, and R. Song, “Robust interpolation of correspondences for large displacement optical flow,” in Proceedings of the IEEE Conference Computer Vision Pattern Recognition, pp. 481–489, Honolulu, HI, USA, July 2017.
View at: Publisher Site | Google Scholar
S. Shi, D. Zhang, C. Zhang, Z. Chen, C. Feng, and B. Fan, “Large displacement optical flow estimation based on robust interpolation of sparse correspondences,” IEEE Access, vol. 8, pp. 227360–227372, 2020.
View at: Publisher Site | Google Scholar
Y. Li, Y. Hu, R. Song, P. Rao, and Y. Wang, “Coarse-to-fine PatchMatch for dense correspondence,” IEEE Transaction on Circuits System Video Technology, vol. 28, no. 9, pp. 2233–2245, 2018.
View at: Publisher Site | Google Scholar
J. Chen, Z. Cai, J. Lai, and X. Xie, “Efficient segmentation-based PatchMatch for large displacement optical flow estimation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 12, pp. 3595–3607, 2019.
View at: Publisher Site | Google Scholar
Y. Deng, J. Xiao, S. Z. Zhou, and J. Feng, “Detail preserving coarse-to-fine matching for stereo matching and optical flow,” IEEE Transactions on Image Processing, vol. 30, pp. 5835–5847, 2021.
View at: Publisher Site | Google Scholar
J. Chen, J. Lai, Z. Cai, X. Xie, and Z. Pan, “Optical flow estimation based on the frequency-domain regularization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 1, pp. 217–230, 2021.
View at: Publisher Site | Google Scholar
A. Dosovitskiy, P. Fischer, E. Ilg et al., “Flownet: learning optical flow with convolutional networks,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, December 2015.
View at: Publisher Site | Google Scholar
E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, “Flownet 2.0: evolution of optical flow estimation with deep networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470, Honolulu, HI, USA, July 2017.
View at: Publisher Site | Google Scholar
Z. Teed and J. Deng, “RAFT: recurrent all-pairs field transforms for optical flow,” 2020, https://arxiv.org/abs/2003.12039v3.
View at: Google Scholar
Y. Lu, J. Valmadre, H. Wang, J. Kannala, M. Harandi, and P. Torr, “Devon: deformable volume network for learning optical flow,” in Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 2705–2713, Snowmass, CO, USA, March 2020.
View at: Publisher Site | Google Scholar
D. Sun, X. Yang, M. Y. Liu, and J. Kautz, “PWC-net: cnns for optical flow using pyramid, warping, and cost volume,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8934–8943, Salt Lake City, UT, USA, June 2018.
View at: Publisher Site | Google Scholar
W. Huerst and W. Xie, “Weighted root mean square approach to select the optimal smoothness parameter of the variational optical flow algorithms,” Optical Engineering, vol. 51, no. 3, pp. 720–732, 2012.
View at: Google Scholar
S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski, “A database and evaluation methodology for optical flow,” International Journal of Computer Vision, vol. 92, no. 1, pp. 1–31, 2011.
View at: Publisher Site | Google Scholar
D. J. Butler, J. Wulff, G. B. Stanley et al., “A naturalistic open source movie for optical flow evaluation,” in Proceedings of the 12th European Conference on Computer Vision, pp. 611–625, Florence, Italy, October 2012.
View at: Google Scholar
A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in Proceedings of theIEEE Conference Computer Vision Pattern recognition, pp. 3354–3361, Providence, RI, USA, June 2012.
View at: Google Scholar
Y. Hu, R. Song, Y. Li, P. Rao, and Y. Wang, “Highly accurate optical flow estimation on superpixel tree,” Image and Vision Computing, vol. 52, no. 3, pp. 167–177, 2016.
View at: Publisher Site | Google Scholar
X. Li, J. Jiaya, and Y. Matsushita, “Motion detail preserving optical flow estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1744–1757, 2012.
View at: Publisher Site | Google Scholar
J. Yang and H. Li, “Dense, accurate optical flow estimation with piecewise parametric model,” in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition, pp. 1019–1027, Boston, MA, USA, June 2015.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Bin Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Mathematical Problems in Engineering

An Improved Fractional-Order Variational Optical Flow Model Combining Structure Tensor

Abstract

1. Introduction

2. Related Work

3. HS Optical Flow Model

4. Local Structure Constraint Equation

4.1. Image Structure Tensor

4.2. Local Structure Constancy Constrain

5. Fractional Smoothness Constraint Equation

6. Improved Fractional Optical Flow Model

7. Experimental Result and Analysis

8. Conclusion

Data Availability

Additional Points

Conflicts of Interest

Acknowledgments

References

Copyright