Dynamic Textures: Modeling the Temporal Statistics

In nature there are plenty of scenes that originate video sequences showing temporal “repetition,” intended in a statistical sense. One could think of a flow of water, a fire, or a flow of car traffic or people walking. This kind of visual processes are now referred to as dynamic textures. (Doretto et al., 2003; Soatto et al., 2001) propose to study dynamic textures as stochastic processes that exhibit temporal stationarity, and introduce the use of linear dynamic systems for modeling their second-order statistical properties. They derived procedures for learning and simulating a dynamic texture model, and demonstrated its effectiveness in several cases using prediction error methods. The formalization is technically sound, and the model has been used in the literature to tackle many other problems by several other authors.

References

1. IJCV

   Dynamic textures Doretto, G., Chiuso, A., Wu, Y. N., and Soatto, S. International Journal of Computer Vision, 2003. abstract bibTeX pdf

   Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind etc. We present a characterization of dynamic textures that poses the problems of modeling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system identification to capture the “essence” of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of second-order stationary processes, we identify the model sub-optimally in closed-form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to infinite length with negligible computational cost. We present experimental evidence that, within our framework, even low-dimensional models can capture very complex visual phenomena.

   @article{dorettoCWS03ijcv,
     abbr = {IJCV},
     author = {Doretto, G. and Chiuso, A. and Wu, Y. N. and Soatto, S.},
     title = {Dynamic textures},
     journal = {International Journal of Computer Vision},
     year = {2003},
     volume = {51},
     pages = {91--109},
     number = {2},
     bib2html_pubtype = {Journals},
     bib2html_rescat = {Dynamic Textures, Visual Motion Analysis, Visual Motion Recognition},
     file = {dorettoCWS03ijcv.pdf:doretto\\journal\\dorettoCWS03ijcv.pdf:PDF;dorettoCWS03ijcv.pdf:doretto\\journal\\dorettoCWS03ijcv.pdf:PDF}
   }
   

2. ICCV

   Dynamic textures Soatto, S., Doretto, G., and Wu, Y. N. In Proceedings of IEEE International Conference on Computer Vision, 2001. Oral abstract bibTeX pdf

   Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind but also talking faces, traffic scenes etc. We present a novel characterization of dynamic textures that poses the problems of modelling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system identification to capture the �essence� of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of secondorder stationary processes we identify the model in closed form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to infinite length with negligible computational cost. We present experimental evidence that, within our framework, even low dimensional models can capture very complex visual phenomena.

   @inproceedings{soattoDW01iccv,
     abbr = {ICCV},
     author = {Soatto, S. and Doretto, G. and Wu, Y. N.},
     title = {Dynamic textures},
     booktitle = {Proceedings of IEEE International Conference on Computer Vision},
     year = {2001},
     volume = {2},
     pages = {439--446},
     address = {Vancouver, BC, Canada},
     month = jul,
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Dynamic Textures, Visual Motion Analysis},
     file = {soattoDW01iccv.pdf:doretto\\conference\\soattoDW01iccv.pdf:PDF;soattoDW01iccv.pdf:doretto\\conference\\soattoDW01iccv.pdf:PDF},
     wwwnote = {Oral}
   }