Lead Inventor:
Shree K. Nayar, Ph.D.
Speech-Enabled Human Face Avatars with Realistic Facial Motion
While computer graphics technology has progressed dramatically over the past couple years, there remains a pressing need to develop human face avatars, which can realistically present and animate a person's face on screen. The current available approaches to this problem have several limitations, such as being not completely accurate in both animation and appearance, and requiring large amounts of seed data to produce a believable presentation. Accordingly, it has been difficult to create an avatar that looks and sounds as if it was produced by a human face that is being recorded by a video camera. Thus, creating speech-enabled avatars of faces that provide realistic facial motion from text or speech inputs represents a worthwhile endeavor.
Using Motion Synthesis Algorithm, Text or Speech Input, and Image of Face to Create Speech-Enabled 2D or 3D Avatar
This technology is a framework for creating speech-enabled 2D or 3D avatars from just a photograph or a single stereo image of a face respectively. The avatar contemplated can be animated using text or speech input and a novel motion synthesis algorithm. The approach proposed here can significantly enhance the user experience and create new modes of interactive applications for users.
Applications:
Sufficiently accurate speech-enabled avatars can have multiple applications, predominately related to the user experience in various contexts:
• Communications: This technology can be used in video conferencing applications and generating avatars for social networking or web profiles.
• Advertising: This approach can create interacting avatars used in web-based or physical ads.
• Gaming: This method can be used to provide better gaming experience by creating more realistic avatars.
• Information Retrieval: The product can be used as an interface to extract information from a kiosk at a number of locations
Advantages:
• The technology described here can be used to develop more realistic avatars, which can be animated using text or speech input
• This method can be applied to 2- or 3-D environments
Patent Status: Patent Pending (WO2008/141125) ~ see link below.
Licensing Status: Available for Licensing and Sponsored Research Support
Publications: D. Bitouk, S. K. Nayar; Creating a Speech Enabled Avatar from a Single Photograph;
Proceedings of IEEE Virtual Reality; Mar, 2008.