Virtual Agent for dialogue system research

Our laboratory is participating in the Avatar Symbiotic Society Project which is a part of Moonshot Research and Development program, and conducting research and development of spoken dialogue systems and remote avatar interaction, using a high presence and high life-like CG agent (CG Cybernetic Agent; CG-CA) system. The “Rubica” and “Gene” below were developed as the official avatars for the project, in collaboration with our lab and Ishiguro Lab, Osaka University.

Here is a list of CG-CAs developed by our laboratory so far.

Rubica

Rubica

“Rubica” is a high-definition 3D CG human-like agent designed to work fine for both an automated spoken dialogue system and a remote avatar system. With its truely neutral appearance that can be interpreted as either male or female and the rich facial and body expressions for conversation, it gives users a real sense of a digital human when used as an automated spoken dialogue system, while the operator’s presence is felt when used as a remote avatar controlled by a human operator. This portable 3D-CG avatar system makes it possible to apply spoken dialogue and remote avatar interaction technology to a wide range of social situations.

Gene

Gene

“Gene” ( /dʒénə/ ) is an cartoon-style CG agent with a neutral appearance and a generic design. It is equipped with rich expressiveness with numerous emotional expressions and physical calculations. It’s life-like behavior and fully tailored contemporary anime-like expression will make users a rich sense of presence. It is a full body 3-D model with 116 morphs, 221 bones, and 87 physical rigid bodies, and can express a cartoon-style life-like expressions using facial expressions and the entire body, driven by our multi-modal dialogue system MMDAgent-EX. This model has a simple design to be accepted by general users in real life. It is lightweight and runs smoothly at high frame rates even on mobile devices, thus can be used in a wide range of social situations.

Information

Other tips