Providing a semantic interpretation of the actions and interactions between the actors in a scene enables behavior analysis for automated decision-making. This is still a challenging problem, especially when performed online. The problem is aggravated when the track of each actor is fragmented and not linked, due to occlusions, traffic, and tracking errors (Chan et al., 2006) (Chan et al., 2006). In the online settings, behavior might depend on the interactions between pairs of actors, which ultimately constitute the building blocks of more complex group behavior (Motiian et al., 2013). A major challenge is to detect and recognize online such pairwise interactions in a causal fashion (Siyahjani et al., 2014), and by leveraging multiple camera sensors whenever available (Motiian et al., 2017). Correctly interpreting behavioral traits can also be used to characterize identity, and can be an important discriminator at large standoff distances.

Human Interactions

2 minute read

Recognizing human interactions from video is an important step forward towards the long-term goal of performing scene understanding fully automatically. Rece...

Aerial Video Analysis

8 minute read

In aerial video moving objects of interest are typically very small, and being able to detect them is key to enable tracking. There are detection methods tha...


  1. TCSVT
    Online Human Interaction Detection and Recognition with Multiple Cameras Motiian, S., Siyahjani, F., Almohsen, R., and Doretto, G. IEEE Transactions on Circuits and Systems for Video Technology, 2017. abstract bibTeX pdf
  2. ICME
    Online Geometric Human Interaction Segmentation and Recognition Siyahjani, F., Motiian, S., Bharthavarapu, H., Sharlemin, S., and Doretto, G. In Proceedings of IEEE International Conference on Multimedia and Expo, 2014. abstract bibTeX pdf
  3. ISVC
    Pairwise Kernels for Human Interaction Recognition Motiian, S., Feng, K., Bharthavarapu, H., Sharlemin, S., and Doretto, G. In Advances in Visual Computing, 2013. Oral abstract bibTeX pdf html
  4. CVPR
    Joint recognition of complex events and track matching Chan, M. T., Hoogs, A., Bhotika, R., Perera, A., Schmiederer, J., and Doretto, G. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006. abstract bibTeX pdf
  5. ICPR
    Event recognition with fragmented object tracks Chan, M. T., Hoogs, A., Sun, Z., Schmiederer, J., Bhotika, R., and Doretto, G. In Proceedings of the International Conference on Pattern Recognition, 2006. abstract bibTeX pdf