Innovative AI research by Institute of Digital Games scholars wins prestigious NVIDIA Award

IDG PhD students Nemanja Rasajski and Chintan Trivedi won the NVIDIA Best Paper Award at the 2024 European Conference on Computer Vision, where their innovative approach addressed the one of the big challenges of AI - namely having AI models understand gameplay across multiple games. The PhD students presented BehAVE, an AI tool developed together with Prof. Antonios Liapis, Prof. Georgios N. Yannakakis (Institute of Digital Games), and Dr Kostas Makantasis (Faculty of ICT), that can describe in text format what is happening on the screen based on video output only. And while state-of-the-art video encoders were able to do this, BeHAVE is groundbreaking because it can transfer what it learns from one game to apply it to another, even across genres. It is able to do this regardless of different visual styles and without access to the game engine.

BehAVE opens up new possibilities for applications such as behaviour recognition, AI-driven gameplay analysis, and video understanding because it allows for data collection without requiring access to a video game studio’s game engine which is often highly restricted. Not only does it offer a practical solution to the data collection challenge in video games, it also steps into the domain of general artificial intelligence.

The research was supported by Project OPtiMaL and funded by Xjenza Malta through the SINO-MALTA Fund 2022.