Researchers at the University of Rochester have developed software that can turn any computer or smartphone with a camera into a personal mental health monitoring device.

In a paper to be presented at the American Association for Artificial Intelligence conference in Austin, Texas, Professor of Computer Science Jiebo Luo and his colleagues describe a computer program that can analyze “selfie” videos recorded by a webcam as the person engages with social media.

The team was able to measure a user’s heart rate simply by monitoring very small, subtle changes in the user’s forehead color.The novel approach quietly observes user’s behavior while he or she uses the computer or phone as usual. The program is deemed “unobtrusive; it does not require the user to explicitly state what he or she is feeling, input any extra information, or wear any special gear.” For example, the team was able to measure a user’s heart rate simply by monitoring very small, subtle changes in the user’s forehead color.

The software goes through a number of “clues” to determine things like heart rate, blinking rate, eye pupil radius, and head movement rate. At the same time, it also analyzes both what the users posted on Twitter, what they read, how fast they scrolled, their keystroke rate and their mouse click rate. Not every input is treated equally though: what a user tweets, for example, is given more weight than what the user reads.

The program currently only considers emotions as positive, neutral or negative, with extra sensitivity coming at a later stage. Right now, this is a demo software and no “app” exists yet…