Researchers create virtual therapist with webcam and game sensor

Researchers with the University of Southern California Institute for Creative Technologies are working on a virtual therapist that appears lifelike and is aimed to help those who need some type of counseling but aren't yet ready or able to see a live worker. In addition, the digital shrink, because of the way it is designed, can be used to monitor the minute details of a person's body language over time, helping a live counselor monitor progress.

The virtual therapist is designed to look as much like a real human as possible, with realistic body movements and facial expressions similar to those you'd observe from a live in-person counselor. The therapist "observes" its client by means of a webcam and a gaming sensor that is mounted above the display, with the person's various movements, facial expressions, and other such shifts being recorded in relation to the corresponding question.

The information that is gathered from the "clients" are fed into a computer where the virtual therapist's software lies. Such data is used to aid the software in guiding the virtual shrink in how it should approach clients and what questions it should ask, and how it should interpret the body language of clients that respond to the questions.

For example, a certain tone of voice and facial movement, such as an aversion of eyes or a brief smile, all indicate different mental aspects of the client, and can help the virtual therapist pin-point whether the person is suffering from depression, anxiety, or other such disorders. One big purpose of the virtual therapist is in PTSD cases, helping soldiers address the issue and proceed towards live counseling.

The center responsible for the work spends a lot of time collecting data from hundreds of military personnel, helping provide the data needed for the software to eventually identify the signs of PTSD. Other experiments being carried out by the researchers include the creation of a 3D human face hologram, and virtual full-size human projections that interact with real humans.