Dynamic Texture Segmentation

Summary

Segmenting the image plane of video sequences is often one of the first steps towards the analysis of video. A lot of effort has been spent on developing image segmentation techniques based on cues such as color, or texture. Similarly, there are several methods for segmenting image motion based on optical flow, or motion features. On the other hand, there might be cases where segmentation based on photometry, or motion (dynamics) alone might be insufficient, because the motion of the object, segmented based on photometry, is incorrect, or because the color of the object, segmented based on motion, is wrong. (Doretto et al., 2003) addresses, for the first time, the problem of segmenting video based on the joint photometry and dynamics of the scene. The approach aims at solving a variational optimization problem that looks for the region boundaries and the dynamic texture models that can optimally represent the video data inside each region. The result is an algorithm that can group regions with the same spatio-temporal statistics. Extensions of this approach have been successfully used for applications such as traffic monitoring, and medical image analysis for segmenting organs in motion.

References

1. ICCV

   Dynamic texture segmentation Doretto, G., Cremers, D., Favaro, P., and Soatto, S. In Proceedings of IEEE International Conference on Computer Vision, 2003. abstract bibTeX pdf

   We address the problem of segmenting a sequence of images of natural scenes into disjoint regions that are characterized by constant spatio-temporal statistics. We model the spatio-temporal dynamics in each region by Gauss-Markov models, and infer the model parameters as well as the boundary of the regions in a variational optimization framework. Numerical results demonstrate that, in contrast to purely texture-based segmentation schemes, our method is effective in segmenting regions that differ in their dynamics even when spatial statistics are identical.

   @inproceedings{dorettoCFS03iccv,
     abbr = {ICCV},
     author = {Doretto, G. and Cremers, D. and Favaro, P. and Soatto, S.},
     title = {Dynamic texture segmentation},
     booktitle = {Proceedings of IEEE International Conference on Computer Vision},
     year = {2003},
     volume = {2},
     pages = {1236--1242},
     address = {Nice, France},
     month = oct,
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Dynamic Textures, Visual Motion Segmentation},
     file = {dorettoCFS03iccv.pdf:doretto\\conference\\dorettoCFS03iccv.pdf:PDF;dorettoCFS03iccv.pdf:doretto\\conference\\dorettoCFS03iccv.pdf:PDF},
     owner = {doretto}
   }