Dynamic Textures: Joint Modeling of Spatial and Temporal Statistics

In analyzing visual processes there may be portions of videos that can be modeled as dynamic textures, which means that they exhibit temporal stationarity. In addition to that, within a single frame they may also exhibit repetitions of the same patterns, like in image textures, which means that the visual process is spatially stationary as well. Therefore, it makes sense to design models that can capture the structure of the joint spatial, and temporal statistics, for the purpose of enabling recognition and segmentation. (Doretto et al., 2004) introduces a model for this kind of dynamic textures, which combines a tree representation of Markov random fields, for capturing the spatial stationarity, with linear dynamic systems, for capturing the temporal stationarity of the visual process. The effectiveness of the model is demonstrated by showing extrapolation of video in both space and time domains. The framework sets the stage for simultaneous segmentation and recognition of spatio-temporal events.

References

1. ECCV

   Spatially homogeneous dynamic textures Doretto, G., Jones, E., and Soatto, S. In Proceedings of European Conference on Computer Vision, 2004. Oral abstract bibTeX pdf

   We address the problem of modeling the spatial and temporal second-order statistics of video sequences that exhibit both spatial and temporal regularity, intended in a statistical sense. We model such sequences as dynamic multiscale autoregressive models, and introduce an efficient algorithm to learn the model parameters. We then show how the model can be used to synthesize novel sequences that extend the original ones in both space and time, and illustrate the power, and limitations, of the models we propose with a number of real image sequences.

   @inproceedings{dorettoJS04eccv,
     abbr = {ECCV},
     author = {Doretto, G. and Jones, E. and Soatto, S.},
     title = {Spatially homogeneous dynamic textures},
     booktitle = {Proceedings of European Conference on Computer Vision},
     year = {2004},
     volume = {2},
     pages = {591--602},
     address = {Prague, Czech Republic},
     month = may,
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Dynamic Textures, Visual Motion Analysis},
     file = {dorettoJS04eccv.pdf:doretto/conference/dorettoJS04eccv.pdf:PDF},
     owner = {doretto},
     timestamp = {2013.10.18},
     wwwnote = {Oral}
   }