Virtual Friend: Tracking and Generating Natural Interactive Behaviours in Real Video

Yue Zheng, Yulia Hicks, Darren Cosker, David Marshall and Jonathan Chambers

Abstract

In this paper, we present a novel approach to generating a variety of complex behaviour responses for a “virtual friend” in three dimensional (3D). The model used to tracking and generating is Hidden Markov Model (HMM). It trained on real motion capture (MoCap) data collected from real people. In tracking stage, we used Annealed Particle Filtering to tracking whole human body, since it is capable of recovering whole human body motion efficiently. In generating stage, we use a dual HMM to represent temporal constraints and use them to generate interactive behaviour for a “virtual friend” character corresponding to the given motion of another character. In this paper, we describe our model, tracking and generating method. We also present experiments that show our virtual friend.

Downloads

• PDF (120 KB)
• push_clip.mov - The behaviour of the virtual human in this clip is generated based on the interaction of the person on the right.
  

Citation

• Z.Zheng, Y. A. Hicks, D. Cosker, D. Marshall and J. A. Chambers, “Virtual Friend: Tracking and Generating Natural Interactive Behaviours in Real Video”, In Proc. of IEEE International Conference on Signal Processing (ICSP), Gullin, China, 2006.

Related Work

• Z.Zheng, Y. A. Hicks, D. Cosker, D. Marshall and J. A. Chambers, Generating 3D Interactive Behaviours, IET CVMP 2006