Virtual Agent for dialogue system research

Our laboratory is participating in the Avatar Symbiotic Society Project which is a part of Moonshot Research and Development program, and conducting research and development of spoken dialogue systems and remote avatar interaction, using a high presence and high life-like CG agent (CG Cybernetic Agent; CG-CA) system. The “Rubica” and “Gene” below were developed as the official avatars for the project, in collaboration with our lab and Ishiguro Lab, Osaka University.

Here is a list of CG-CAs developed by our laboratory so far.

Rubica

Rubica

“Rubica” is a high-definition 3D CG human-like agent designed to work fine for both an automated spoken dialogue system and a remote avatar system. With its truely neutral appearance that can be interpreted as either male or female and the rich facial and body expressions for conversation, it gives users a real sense of a digital human when used as an automated spoken dialogue system, while the operator’s presence is felt when used as a remote avatar controlled by a human operator. This portable 3D-CG avatar system makes it possible to apply spoken dialogue and remote avatar interaction technology to a wide range of social situations.

Gene

Gene

“Gene” ( /dʒénə/ ) is an cartoon-style CG agent with a neutral appearance and a generic design. It is equipped with rich expressiveness with numerous emotional expressions and physical calculations. It’s life-like behavior and fully tailored contemporary anime-like expression will make users a rich sense of presence. It is a full body 3-D model with 116 morphs, 221 bones, and 87 physical rigid bodies, and can express a cartoon-style life-like expressions using facial expressions and the entire body, driven by our multi-modal dialogue system MMDAgent-EX. This model has a simple design to be accepted by general users in real life. It is lightweight and runs smoothly at high frame rates even on mobile devices, thus can be used in a wide range of social situations.

Gene’s 3D model is available on GitHub under CC-BY 4.0 license. Note that the copyright holder still has trademark and design rights of this model. You are permitted to use its trademark and design for:

  • Academic purpose (publications and releases), and
  • Personal non-commercial usage (posts to SNS or at an event). For other commercial use, please contact us.

Uka

Uka

Uka is a CG-CA designed to be used primarily in chat dialogues and sympathetic dialogue tasks. The design is neutral, with a male-leaning image of a listener who listens attentively to what the user has to say. The character’s personality is not overly strong, with Japanese-style clothing and earth tones used to convey a sense of calmness. The body parts (ears and tail) are intended to be used for non-verbal expression of emotions and internal states. 49,951 vertices, 77,907 faces, 301 bones, 148 shapes, and 164 rigid bodies make up this medium-sized model.

Uka’s 3D model is available on GitHub under CC-BY 4.0 license.

Information

Other tips