Editing Dynamic Textures by Changing Model Parameters

The operation that by processing video data allows producing new video data in Computer Graphics is known as Video-Based Rendering (VBR). Developing new VBR techniques is important because they allow to quickly synthesize new photo-realistic scenes (because of the origin of the data) without having to develop and simulate a synthetic model of the scene. The difficulty is in developing editing techniques that, once applied to the original data, can produce the desired perceptual effect. (Doretto & Soatto, 2002) and (Doretto & Soatto, 2003) represent a VBR approach that allows learning dynamic texture models from video, and then simulating them for synthesizing/rendering new unseen videos. Modeling the spatial stationarity enables the synthesis not only in time, but also in space (so the frame size can grow) (Doretto et al., 2004). It is shown how dynamic texture model parameters can be edited (changed) online, and mapped to meaningful perceptual changes, such as the spatial frequency content, the speed, the time axis, or the intensity of the visual process. This means that from a video sequence of sea waves one could, for instance, produce a new video with a rougher or smoother sea movement, according to the desire.

Plenoptic Modeling for Viewpoint Editing

(Doretto & Soatto, 2003) further extends the previous VBR approach by introducing a model of the spatio-temporal statistics of a collection of images of dynamic scenes as seen from a moving camera. The joint modeling of the moving vantage point together with the statistics of the scene motion is obtained by introducing a time-variant linear dynamical system. The resulting algorithms could be useful for video editing where the motion of a virtual camera can be controlled interactively, as well as for performing stabilized synthetic generation of video sequences.


  1. ECCV
    Spatially homogeneous dynamic textures Doretto, G., Jones, E., and Soatto, S. In Proceedings of European Conference on Computer Vision, 2004. Oral abstract bibTeX pdf
  2. CVPR
    Editable dynamic textures Doretto, G., and Soatto, S. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. abstract bibTeX pdf
  3. Towards plenoptic dynamic textures Doretto, G., and Soatto, S. In Proceedings of the 3rd International Workshop on Texture Analysis and Synthesis, 2003. abstract bibTeX pdf
  4. Editable dynamic textures Doretto, G., and Soatto, S. In Conference Abstracts and Applications of SIGGRAPH ’02, 2002. Oral bibTeX pdf