Happiness tracking software could gauge mood in photos

[Nipuna]

SMILE – you're on camera! If you want a quick way to pick out the happiest snaps from a wedding or judge the changing mood of a crowd, now there's software that can do it for you.

Developed by Abhinav Dhall at the Australian National University in Canberra and colleagues, the software analyses all the faces in a photo to give the shot an overall "mood score".

The team used face tracking software to analyse the smiles of the faces in a group by noting the positions of nine spots on the face such as the corners of the mouth and eyes. A machine learning algorithm, trained on photos that had been pre-labelled by humans, then used this data to give each face a smile intensity score.

The team also programmed the system to incorporate information from volunteers, who assessed how important the intensity of any individual's smile was to the overall mood score of a photo. Those who were standing near the centre of a picture were given a stronger weighting, for example, while partially obscured faces were less influential. When asked to gauge the happiness level of a photo, the system only deviated from the opinion of a human by around 7 per cent. The software was presented at the Conference on Multimedia Retrieval in Dallas, Texas, last week.

Dhall says the aim is to be able to assess the overall mood of a group from a single shot. By looking at a sequence of frames in a video, it could even gauge the mood of a crowd in real time. "If the mood score goes down over the time, we can assume that the group are getting angry," says Dhall. It could also be used to view albums on Facebook by arranging photos so the happiest ones are shown first, for example.

Ranael Kaliouby says the work fits well with what she is doing with her emotion analysis company, Affectiva. "I love the application – namely, understanding and modelling the overall mood of a crowd," she says. "It gets us one step further to applying emotion measurement technologies in the real world."

This article appeared in print under the headline "Get your grin on to light up the happiness tracker"