Dynamic Shape and Appearance: Joint Shape, Appearance, and Dynamics Modeling

3 minute read

Rather then attempting to model the temporal image variability of dynamic textures by capturing only how image intensities (appearance) vary over time, one could try to describe it by modeling how the shape of the scene varies. Both representations have advantages and limitations. For instance, the temporal variations of sharp edges are better captured by shape variation; however, this one cannot be used when a directional motion component is present, and appearance is the alternative. Therefore, exploiting the benefits of jointly modeling shape and appearance is very important, as it has been demonstrated for single images, but the extension to dynamic scenes (motion) was missing. (Doretto & Soatto, 2006; Doretto, 2005) address this issue, and propose to explain stationary image variability by means of the joint variability of shape and appearance akin to a temporal generalization of the well-known Active Appearance Models (AAMs). The issues of how much image variability should be modeled by shape, how much by appearance, how they vary over time (motion), and how appearance, shape and motion merge together, are addressed. The approach is capable of learning the temporal variation of higher-order image statistics, typical of videos containing sharp edge variation.

References

  1. TPAMI
    Dynamic shape and appearance models Doretto, G., and Soatto, S. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006. abstract bibTeX pdf
  2. CVPR
    Modeling dynamic scenes with active appearance Doretto, G. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. Oral abstract bibTeX pdf