Skip to content

Python implementation of Typhoon algorithm: dense estimation of 2D-3D optical flow on wavelet bases.

License

Notifications You must be signed in to change notification settings

pderian/PyTyphoon

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PyTyphoon

Python implementation of Typhoon motion estimator: dense estimation of 2D/3D optical flow on wavelet bases, primarily aimed at fluid motion estimation.

Important remarks

At the moment, the wavelet-based data DFD term (Dérian et al., 2013) only is provided: the high-order regularizers (Kadri-Harouna et al., 2013) are not included in this implementation.

The reference implementation used in (Dérian et al., 2015) and (Dérian et al., 2017) is written in C++ and GPU-accelerated with CUDA, and contains the high-order regularizers. It is the property of Inria (FR) and the CSU Chico Research Foundation (Ca, USA), and can be licensed from these institutions. This Python implementation is not the same as the reference for many reasons, and it is obviously much slower.

Requirements

Tested with Anaconda Python 3.6.1, Numpy 1.12.1, Scipy 0.19.1, PyWavelet 0.5.2.

Usage

The Typhoon class can be imported from other modules/scripts to perform estimations as needed.

The script can also work as a standalone estimator in simple cases, e.g.:

python pytyphoon.py -i0 path/to/im0.jpg -i1 path/to/im1.jpg -wav 'db3' --display

will solve the problem for image pair (im0.jpg, im1.jpg) and wavelet Daubechies-3. See python pytyphoon.py -h for the complete list of parameters.

How does it work?

Typhoon solves a dense variational optical flow problem, that is to say: (i) it provides one motion vector at every pixel of input images and (ii) it estimates the entire vector field altogether.

To do so, it looks for the motion field which minimizes the displaced frame difference (DFD):

DFD

This is achieved by minimizing the following functional:

DFD functional

where the integral is taken over the image. The functional above is non-linear with respect to the motion field. This has the advantage of better handling large displacement, but complicates the minimization process.

For the non-linear minimization to succeed, the solution should lie reasonable "close" to the first guess. This is where wavelets bases come into play: by providing a multiscale representation of the motion field, they enable to estimate the motion iteratively from its coarsest scales to the finests.

The minimization is handled by L-BFGS, which is efficient memory-wise and only requires the functional value and its gradient.

Demos

These demos are shipped with the project.

(2D) Synthetic particle images

Simple 2d estimation using synthetic particle images (256x256 pixels) originally created for the FLUID project (image database #1). Run:

python pytyphoon.py --demo particles

Particle results

(3D) Homogeneous shift

Simple 3d estimation using synthetic images (64x64x64 pixels) obtained by filtering random normal noise at various scales. The displacements are integer shifts along each of the 3 dimensions. Run:

python pytyphoon.py --demo 3dshift

3dshift results

(3D) Column vortex

Simple 3d estimation using synthetic images (96x96x96 pixels) obtained by filtering random normal noise at various scales. The displacement field is a column vortex (first two axes) with an updraft increasing linearly along the third axis. Run:

python pytyphoon.py --demo 3dvortex

3dshift results

References

  • (Dérian et al., 2017) Dérian, P. & Almar, R. "Wavelet-based Optical Flow Estimation of Instant Surface Currents from Shore-based and UAV Video". IEEE Transactions on Geoscience and Remote Sensing, Vol. 55, pp. 5790-5797, 2017.
  • (Dérian et al., 2015) Dérian, P.; Mauzey, C. F. and Mayor, S. D. "Wavelet-based optical flow for two-component wind field estimation from single aerosol lidar data". Journal of Atmospheric and Oceanic Technology, Vol. 32, pp. 1759-1778, 2015.
  • (Dérian et al., 2013) Dérian, P.; Héas, P.; Herzet, C. & Mémin, E. "Wavelets and Optical Flow Motion Estimation". Numerical Mathematics: Theory, Method and Applications, Vol. 6, pp. 116-137, 2013.
  • (Kadri-Harouna et al., 2013) Kadri Harouna, S.; Dérian, P.; Héas, P. and Mémin, E. "Divergence-free Wavelets and High Order Regularization". International Journal of Computer Vision, Vol. 103, pp. 80-99, 2013.

Todo

  • NetCDF for output results? in standalone mode;
  • support of masks;
  • some regularizers;
  • alternative penalization functions;
  • divergence-free wavelets;
  • ...

About

Python implementation of Typhoon algorithm: dense estimation of 2D-3D optical flow on wavelet bases.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages