Students from HSE University in Nizhny Novgorod Create Web Service for Recognising Emotions

© iStock

A team of students from HSE University in Nizhny Novgorod and Minin University have created a mobile application and a web service for recognising emotions in photographs. This solution can be useful in marketing, education, personnel management—any areas where the quality of interpersonal communication matters.

The development of modern technologies makes it possible to realise the task of recognising emotions in photographs. Such technologies are widely used in marketing to help improve targeted advertising: an installation with a camera can change the commercial depending on the mood, gender, and age of people passing by.

It is difficult to overestimate the importance of emotion recognition in other areas such as education, where it can be used (among other things) to study the emotional state of students in a classroom. The ability to recognise emotions can be useful for training professionals who work with people, and can improve the level of emotional intelligence and reflection of people in general.

As a result of joint work on the project, fourth-year student of the Bachelors in Applied Mathematics and Informatics (which from 2023 has been replaced by the Bachelor's in Computer Science and Technologies) at HSE University in Nizhny Novgorod Vladislav Zagoskin and fourth-year student of Minin University Elena Pestova have developed a web application that allows users to upload photos for emotion recognition and obtain an analysis of the types of emotion identified: anger, disgust, sadness, contempt, joy, fear, or surprise.

In addition, the students have created a mobile application with a convenient interface that allows users to take photos and transfer data to the server via HTTP protocol. The application is a simulator that uses photographs of people’s faces to identify one of seven basic emotions, and as a result, get a total number of correct answers. The app features descriptions of the characteristic markers of emotions. It can be useful to students in fields where effective communication is important (psychology, teaching, personnel management, etc.).

‘This app can be helpful to a wide range of users who struggle with the recognition of emotions,’ the project authors believe.

Elena Shadrina

Head of the project, Deputy Dean of the Faculty of Informatics, Mathematics and Computer Science

It is nice when an IT solution finds an application in the form of a useful service, and when the thesis of a student of the Applied Mathematics and Informatics programme is not just a graduation paper, but a work of interdisciplinary applied research at the intersection of mathematics, informatics, and computer science. Many thanks to our colleagues Andrey Savchenko and Egor Churaev for the technical advice—there is a reason why they say that HSE University is one big family where you will always get help.

Vladislav Zagoskin

Graduate of the Bachelor’s in Applied Mathematics and Informatics

The courses of our Applied Mathematics and Informatics programme were very useful to me when creating the application, especially the courses in machine learning and neural networks. And my supervisor Elena Shadrina introduced me to the scientists of the Laboratory of Algorithms and Technologies for Networks Analysis at HSE University in Nizhny Novgorod Egor Churaev and Andrey Savchenko, who are also engaged in emotion recognition. They recommended a model from which I can make my own unit. In addition, they advised the approach to training the model and how to use an already-trained model.

Elena Pestova

Fourth-year student of the Bachelor’s in Psychology of Management and Internet Communications at Minin University

I loved working on the project. It was interesting to select photos depending on the emotion depicted. As a result, I began to better distinguish between shades of emotions and their recognisable markers myself.

I worked directly on the theoretical part of our emotion recognition application. Exploring various theories of emotion has been an incredible pleasure for me. In the end, I am satisfied with the result. It will help psychology students to learn to determine emotions more accurately, which will undoubtedly allow them to help people better in general!