An issue for our healthcare facilities as well as many others around the country is better understanding the needs of patients. If a patient has experienced a form of trauma, talking to a doctor face-to-face may be difficult. This is where the NVIDIA Omniverse can help doctors extract vital information that they would otherwise miss. Talking with a non-threatening animated avatar (shown in this NVIDIA Audio2Face demo) may help a patient open up and allow them to speak openly. This is also especially helpful for children who are afraid of interacting with unfamiliar people. With Audio2Face, an avatar mesh can be created using standard modelling tools (such as Blender or Unreal Engine) and imported into the Audio2Face app. Once imported, a live recording can be started and the avatar’s facial movements imitate the natural movements of our faces.

audio to face

NVIDIA is planning to add native language translation to this application (shown in the video below). Once this happens, we can move forward into building an application to house this technology and allow it to be user-friendly.

In the meantime, however, we have already begun initial trials in translational training. Using the NVIDIA code provided here, we are testing different models and methods to produce the most efficient and accurate results. Currently, we are using the Google Neural Machine Translation (GNMT) version 2 model to train a neural network to translate German text into English with other languages being trained in the future.