MINUTES OF PAVR TECHNICAL MEETING
Expertise centre for Digital Media, Diepenbeek, Belgium, February 18th 1997

Present
University of Bath Phil Willis (chair)
University of Glasgow John Patterson
Ecole des Mines, Nantes Jean-Daniel Fekete
EPFL, Lausanne Tom Molet
University of Geneva Pascal Volino
TU, Vienna Michael Gervautz
UJF, Grenoble Jean-Dominique Gascuel, Agata Opalach
INRIA, Rennes Stephane Donikian
UIB, Mallorca Ramon Mas
LUC, Diepenbeek Frank van Reeth, Eddy Flerackers

Apologies
Siemens, Munich Christoph Maggioni

  1. Welcome, introductions and departure times
    Constraints on departure times were noted, for scheduling the end of the meeting.

  2. Minutes of previous meeting
    Minutes of the inaugral meeting of December 6th 1996, at the University of Bath UK, were accepted. There were no matters arising.

  3. Management of the project
    In the PAVR proposal, as it was accepted by the EC, a list of work packages, tasks and resulting deliverables were identified. The meeting considered whether these were still valid, and updated them where necessary. Firstly, the task list was checked; secondly the deliverable list was checked; thirdly the work package list was checked.

    Task contribution list:

    T1 `Specification of the system': All (stays as it is)

    T2 `Extraction of 3D objects from still and moving images': Bath, Glasgow, UIB (added)
    [John Patterson elucidated that the Turing Institute will contribute to this task. Phil Willis gave some background on how the involvement of industry changed during the project definition. ERC (a conglomerate of Siemens, Bull and ICL) was at first a partner in the PAVR consortium. This was attractive to the EC, but ERC had to pull out when they were effectively shut down by their parent companies Bull, ICL and Siemens. The Turing Institute would come in, but the EC said they were too small. Ultimately, it was decided that one of the partners in ERC should join in - it became Siemens.]

    T3 `Realistic animation and simulation of multi-body systems' to become `Physically based animation and simulation of multi-body systems' UFJ, INRIA, EPFL, TU-Vienna, UIB (added), UG (added) (EMN removed)

    T4 `Motion capture through magnetic sensors and modelling EPFL, LUC (added)

    T5 `Construction, manipulation, movement of 3D objects immersively' LUC, EPFL (UG removed)

    T6 `Interface for guiding virtual moving objects by speech' EMN (stays as it is)

    T7 `Facial/emotional animation' UG, Bath (stays as it is)

    T8 `Visual analysis of face regions' Sie, Bath (stays as it is)

    T9 `Collision of rigid and deformable objects' UFJ, INRIA, EPFL (added), TU-Vienna (added), UIB (added), UG
    It was agreed that T9 and T3 should have the same partner list.

    T10 `Deformation of objects' UG, UFJ, EPFL, INRIA (added)

    T11 `Distribution across broadband networks' LUC, Bath (stays as it is)

    T12 `Workshop organisation' Sie (stays as it is)

    T13 `Morphing' Bath, EPFL, Glasgow, UG
    This task is added to enable a new regrouping of related deliverables (D12-D15), which are were spread in less structured manner over various tasks and WPs.

    Deliverables list:

    D0: stays as it is

    D1: moved from end year 1 to end year 3 (as ERC would do a lot of the work)

    D2: moved from end year 2 to mid year 3 (a question was asked whether Siemens might deliver some input here.
    § Siemens to comment

    D3: stays as it is (It is a substantial workload, so Michael Gervautz as representative of the responsible partner (TU-Vienna) urges that input from other partners will definitely be needed). The question is asked: if some partner develops a library, do other partners HAVE to use it. Phil Willis answers: No; it is the final intention to demonstrate the PAVR platform, in which various complementary models and libraries can be presented.

    D4: stays as it is; Jean-Daniel Fekete mentions that EMN can contribute to this deliverable as well

    D5: stays as it is;

    D6: stays as it is;

    D7: moved from end year 2 to end year 4; It is unlikely that a commercial system will come out of the research; a work-bench will be worked out that operates on a common interface. It was mentioned that, for example, if someone in the consortium has a specific way to deform objects, it can be used in the consortium; we do not have the role of defining a `Euro-standard' on deformation.

    D8: the suggestion was made to move this deliverable also to the end of year 4. As it should not be the intention to move all the deliverables to the end of the project (we need some integration time as well), the move from end of year 2 to mid year 4 was accepted.

    D9: stays as it is; Given the close relation (also in terms of internal data structures) between the work in D3, D7, D8 and D9, it was suggested that these deliverables be united in one work package (see the following section: Work Package list)

    D10: stays as it is;

    D11: stays as it is; the term `and integration' at the end of the deliverable title was deleted.

    D12: stays as it is; Phil Willis briefly mentioned that the work on the animation manager is aimed high-level, full-length feature animation management, as well as on the lower level (e.g. sequencing). Work regarding model sheets is to be expected as well.

    D13: moved from end of year 2 to end of year 3

    D14: stays as it is;

    D15: stays as it is;

    D16: stays as it is; the question was raised if the workshop wasn't too soon in the project plan (the advantage of being really soon is that we are pushed, at the disadvantage of having not so much to show). It was agreed that the workshop would be scheduled around project month 18. Phil Willis will contact Siemens about their opinion in this context.
    § PJW/Siemens

    D17-D19: As Siemens is not present, these three deliverables stay as they are, for the time being. The statement was raised that maybe EPFL could contribute to D18 (Siemens will be contacted in this respect in due time).
    § EPFL/Siemens

    Work Package list:

    Given the above changes and suggestions, a new WP list was defined:

    WP  Topic                 Tasks        Deliverables       Co-ordinator
    
    WP1 Specification         T1           D0                 Bath
    WP2 Capture technologies  T2, T4       D1, D5, D11        EPFL
    WP3 Immersive interaction T5, T6       D2, D4             EMN
    WP4 Facial interaction    T7, T8       D6, D17, D18, D19  Sie
    WP5 Dynamic simulation    T3, T9, T10  D3, D7, D8, D9     UJF
        (^new topic/name)     
    WP6 Networks              T11          D10                LUC
        (^new topic/name)
    WP7 Workshop organisation T12          D16                Sie
    WP8 Keyframing            T13          D12, D13, D14, D15 Glasgow
        (^new WP)
    
    With respect to D0, it was asked what the planned schedule will be; The suggestion is to try to draft the specification by the June meeting.

    After review of the WP-list, it was noted that UIB are nowhere in the proposal regarding Task-, Deliverable- or WP-membership! Ramon Mas mentioned that UIB is best included in T2 and T3 (as done in this document); for an indication to which Deliverables UIB will be contributing, Ramon Mas said it will be forwarded in the near future.
    § Ramon Mas

  4. Review of appointments
    A quick table-round was undertaken to see which appointments are available at the various partner's sites.

    UG: 1 Ph.D. and 1 Post-doc
    EMN: 1 Ph.D.
    LUC: 1 Post-doc
    TU-Vienna: 1 Ph.D. and 1 Post-doc
    UIB: 1 Ph.D. (already hired) and 1 Post-doc
    EPFL: all positions are occupied
    INRIA: 1 Ph.D. (as of sept.) and 1 Post-doc
    UJF: 1 Post-doc
    Glasgow: 1 Post-doc and 1 Ph.D. (originally 2 Post-docs)
    Bath: 1 Post-doc and 1 Ph.D.

    Phil Willis pointed out that at most 40% of the money can used for Ph.D. positions, so everyone should try to recruit at the Post-doc level as much as possible, at least one where two posts are available.

    What can be done to get the recruitment running?

    Jean-Daniel Fekete suggested to get a precise description of the available jobs:

    • description of the work to be done (technical scope)
    • salary range
    • link to common `rules' (age limit, etc.) for the job
    • facilities available (`try to be attractive')
    • tourist information
We should try to find newsgroups on the WWW for identifying posts.
§ ALL

It was suggested placing advertisements in technical journals; this could be expensive, but since we do it for the 10 sites, it can be cost-effective.

In any case, we need to show our results at the review with the EC, so recruiting is a matter of real urgency.

It was agreed to write at each site the pro forma job description, to put it on a WWW-page by MID MARCH. The URL of this page should be sent to Phil Willis, so he can link to it from the PAVR home page.
§ ALL

In case anyone encounters a potentially interesting bulletin board or email list, please forward it to the PAVR email list.
§ ALL

  • Project demonstrator
    Before discussions on the demonstrator were started, a quick table- round was undertaken to find out what kind of development machines are available at the various PAVR sites. It turns out that all have SGI machines (and some PCs), with the exception of Glasgow, where high- end PCs (and Macs) are available (no SGI yet).

    Discussion then turned to the tools to be used in PAVR. Stephane Donikian mentioned that documentation is available on various VR toolkits:

    • WorldToolkit (Sense8) supports various graphics libraries (OpenGL and Direct3D, so both SGI and PC are supported), and it can be connected to various I/O devices. Its price depends on the architecture: PC +- 4000 USD, on ONYX it will be much more expensive.
    • MR-toolkit (University of Alberta) free-of-charge, UNIX based, various I/O devices supported, modeling tools (objects and animation) are available, software libraries in C and C++ are available. The URL of the WWW-site regarding MR-Toolkit will be sent to the PAVR email list.
      [http://www.cs.ualberta.ca/~graphics/MRToolkit.html]
      
      The question was asked whether VRML2.0 (with C++ and JAVA binding) could be useful (as it could be used in PC viewers as well as on SGI).

      A number of good PC plug-in boards are available, supporting OpenGL as well as Direct3D (e.g. development could be done on PC, with demonstration on SGI).

      The suggestion was made to look at MAYA (Alias/Wavefront). It is not available yet (some sites applied for beta-testing, but nobody has received anything yet); some questions are raised with respect to the real-time capabilities of MAYA.

      Phil Willis remarked that PAVR is not only about VR, but also about Animation. As the needs for the various research tasks in PAVR could be incompatible, it was suggested to seek integration at the work package level. In this context, the question was raised what the EC would expect from PAVR:

      1. do we need to demonstrate the various components at work (without direct simultaneous use; i.e. further integration could be done `with more development work');
      2. do we need to provide collective papers (or is this merely a `desirable' side-effect);
      3. what will be the measureables with respect to the Training and Mobility of the researchers (a lot of software or a lot of researchers moving around). Phil Willis will contact the EC to find out what the EC finds important regarding the deliverables.
        § PJW

    This ended the first meeting day.

    On the second meeting day, the group split in two parts, to discuss the various integrational issues for the demonstration platform. The respective leaders in these two groups will provide a short summary of what has been discussed.

    [Note: report from first group attached. Second group's report will follow in 10 days]


    APPENDIX A: SUMMARY OF REVISED WORK-PLAN

    TASKS

    T1 Specification of the system
    (All partners)
    T2 Extraction of 3D objects from still and moving images
    (Bath, Glasgow, UIB)
    T3 Realistic animation and simulation of multi-body systems
    (UJF, EPFL, UG, INRIA, TU-Vienna, UIB)
    T4 Motion capture through magnetic sensors and modelling
    (EPFL, LUC)
    T5 Construction, manipulation, movement of 3D objects immersively
    (LUC, EPFL)
    T6 Interface for guiding virtual moving objects by speech
    (EMN)
    T7 Facial/emotion animation
    (UG, Bath)
    T8 Visual analysis of face regions
    (Sie, Bath)
    T9 Collision of rigid and deformable objects
    (UJF, EPFL, UG, INRIA, TU-Vienna, UIB)
    T10 Deformation of objects
    (UG, UJF, EPFL, INRIA)
    T11 Distribution across broadband network
    (LUC, Bath)
    T12 Workshop organisation
    (Sie)
    T13 Morphing
    (Bath, EPFL, Glasgow, UG)

    MILESTONES

    Mid year 1   D0
    End year 1   D5, D17
    Mid year 2   D16
    End year 2   D4, D14, D18
    End year 3   D1, D2, D6, D11, D13, D9, D19
    End year 4   D3, D7, D8, D10, D11, D12, D15
    
    DELIVERABLES

    D0 Specification (All)
    D1 Library of extraction of 3D objects from photographs and sequences of images (Bath, Glasgow)
    D2 Interface for guiding virtual moving objects by speaking in a microphone (EMN)
    D3 Library for simulation of multi-body systems (TU-Vienna)
    D4 Demonstrator 3D direct manipulation (LUC)
    D5 Demonstrator of motion capture through magnetic sensors (EPFL)
    D6 Facial animation (Bath, UG)
    D7 Deformations of objects (UJF)
    D8 Collision of rigid objects (INRIA)
    D9 Collision of deformable objects (EPFL, UG)
    D10 Demonstrator of broadband network distributed animation (LUC, Bath)
    D11 Generative models from capture motion (EPFL)
    D12 Demonstrator of animation sequence manager (Bath)
    D13 Automatic morphing of 3D objects (EPFL, UG)
    D14 Demonstrator of animation sequence genetics (Glasgow)
    D15 Demonstrator of image-analysis based morphing and warping (Glasgow)
    D16 Workshop (Sie)
    D17 Eye detection (Sie)
    D18 Dynamic head gesture recognition (Sie)
    D19 Face and hand gesture control of VR animations (Sie)

    WORK PACKAGES

    WP     Topic              Tasks        Deliverables       Co-ordinator
    WP1 Specification         T1           D0                 Bath
    WP2 Capture technologies  T2, T4       D1, D5, D11        EPFL
    WP3 Immersive interaction T5, T6       D2, D4             EMN
    WP4 Facial interaction    T7, T8       D6, D17, D18, D19  Sie
    WP5 Dynamic simulation    T3, T9, T10  D3, D7, D8, D9     UJF
        (^new topic/name)     
    WP6 Networks              T11          D10                LUC
        (^new topic/name)
    WP7 Workshop organisation T12          D16                Sie
    WP8 Keyframing            T13          D12, D13, D14, D15 Glasgow
        (^new WP)
    

    APPENDIX B: SUMMARY OF GROUP 1 DISCUSSION

    Group (WP2/WP3/WP5)

    Topics: Capture technologies (WP2) Immersive interaction (WP3) Dynamic simulations (WP5)

    People:

    Stephane Donikian (IRISA)
    Jean-Dominique Gascuel (UJF)
    Michael Gervautz (TU Vienna)
    Ramon Mas (UIB)
    Tom Molet (EPFL)
    Agata Opalach (UJF) (note taker)
    Bruno Rassaerts (LUC)
    Pascal Volino (UG)

    The discussion was chaired by Dominique Gascuel. Its main purpose was to consider options for close interaction between the teams, in particular ways of creating a software platform in which varied expertise of the PAVR teams could be demonstrated and common results obtained. The chairman first asked the teams around the table to summarise their contributions to the network and their expectancies:

    UJF: Experience in layered deformable implicit models consisting of a dynamic skeleton (a rigid solid, an articulated structure, a particle system) and an implicit layer that coats it. The implicit layer is used to process collisions between objects and model resulting deformations. Interested in testing their models against motion obtained from motion capture, interaction of the implicit and explicit models, rigid and deformable models.

    IRISA: Experience with interaction between several kinds of dynamic objects (articulated, particle, mass-spring-damper etc) and their motion control by executing C++ code generated by solving equations of motion in Matlab/Simulink. Real time performance is an aim. Interested in developing efficient ways of collision detection between various models, incorporating motion capture and defining new high level control paradigms.

    EPFL: Main focus on motion capture for human figures and data input from capturing devices. Currently controllers are only developed for specific EPFL hardware. Interested in the manipulation of recorded motion and its high level editing.

    UIB: Experience in the control of an articulated human figure, its position and its balance. Interested in a further improvement of their models: higher level control and more efficient algorithms.

    TU Vienna: Experience in rigid multi-body dynamics. Interested in multi-model simulations and *motion* levels of detail based on a constrained computation time for a frame.

    LUC: Experience in the virtual reality technology and 3D modelling. Interested in developing an immersive 3D modeller.

    GU: Experience in dynamic simulations of polygonal meshes (cloth in particular), self-collision detection and response modelling using space subdivision. The next generation of these algorithms will use coarser models but with run in real time. Interested in a generalisation of this collision model to other kinds of objects and developing methods for collision detection in a multi-body environment.

    The chairman proposed to identify partners that might be interested in working together on a common task and deliver a demonstrator of their cooperation. The following pairs were identified:

    UJF-GU: collisions between implicit models and polygonal meshes
    TU Vienna-GU: collisions and levels of detail (geometry and motion)
    UJF-EPFL: motion capture used in a simulation environment
    IRISA-TU Vienna: levels of detail and multi-model simulations
    IRISA-EPFL: motion capture and humanoid control
    IRISA-UJF: implicit and parametric model interaction
    LUC-EPFL: immersive modelling
    UIB-EPFL: motion capture for driving articulated structures
    UIB-UJF: implicit coating for humanoid articulated structures

    The final discussion topic was an initial exploration of potential harnesses that would allow the PAVR teams to work together in a uniform environment. The options identified were as follows:

    1. VRML 2.0 + Java/C++ API (?VIPERE = SGI Inventor+Performer+VRML2.0)
    2. MR Toolkit
    3. Maya (Alias Wavefront)
    4. World Toolkit
    5. Autodesk
    6. SoftImage
    The decision has been taken to investigate them further since not enough technical detail about them were known to make a choice. All partners will gather more information about the harnesses, in particular:
    • IRISA and TU Vienna will summarise the advantages and disadvantages of using VRML 2.0 (option 1).
    • EPFL will examine the possibility of using MR Toolkit as a method of transfering their motion capture results (option 2).
    • UIB and UJF will contact Alias Wavefront to obtain information about the expected release date of Maya (option 3).

    §Further details will be provided to all partners and the discussion will be continued over electronic mail.
    APPENDIX C: SUMMARY OF GROUP 2 DISCUSSION


    § to come from JWP