The process of creating a game that can replace a visit to a speech therapist

Abubakir Nunuly (Beasty Crew)

At the very beginning, we decided on the theme and design of our game, since the game is for children, the main characters of the game are funny monsters. Then, our team found the necessary exercises for the development of speech in children, so two different game modes appeared in our game. In the first mode, the child must repeat different emotions behind the monsters, and in the second mode, the child must reproduce the sounds and words that he heard. If a child does repetition successfully, he gets points for which he can buy different masks of cute monsters in the store, and otherwise the AI reads and writes data that it uses to find the optimal words and emojis.

If we move on to the development of the game, then we used the Unity game engine to create “Beasty”. We have connected “Unity+OpenCV” and “UnityFacialExpressionRecognition” assets to find and recognize the child’s emotions, as well as IBM Watson Cloud to our Unity project for recognizing the child’s voice, words and sounds.

We are not going to abandon this project, as we are sure that it will help many children get rid of speech problems.

Our group: Abubakir, Zhanyia, Temirlan, Zhansaya.