Researchers from Nvidia, Stanford, Caltech, and several other institutions unveiled the AI model NitroGen, which can interface with virtual worlds across various video games. Nvidia’s Head of AI, Jim Fan, characterized the neural network as “an open-source foundation model trained to play over 1000 games.” This research also holds significant importance for other domains, such as simulation construction and robotics.
One could say this study represents an attempt to create something akin to a “GPT for actions.” Overall, the work marks a breakthrough in large language models, employing proven technology for massive-scale training beyond the fields of language and computer vision. The description of the scientific paper for this research also states that “creating generalist agents capable of operating in unknown settings has long been considered the Holy Grail of AI research.”
It is interesting that NitroGen is based on the GROOT N1.5 architecture, which was originally developed for robotics. Its application in the video game realm showcases potential utility for robotics, offering advantages to robots operating in unpredictable scenarios. The researchers successfully adapted NitroGen to games with fundamentally different mechanics and physics. For training the AI model, they utilized over 40,000 hours of publicly available gameplay recordings shared by various streamers. Particularly helpful were videos where creators superimposed an image of their controller over the broadcast, demonstrating the buttons pressed during gameplay in real-time.
After completing the training, the researchers conducted a series of tests which indicated that the algorithm performs well in games across different genres, including RPGs, platformers, racing games, battle royales, and others. Although the study’s outcomes are encouraging, Fan stated that the scientists still have a substantial amount of work ahead. The current iteration of NitroGen is deliberately focused on rapid control or what Fan calls “gamer instinct.” Currently, all studies related to NitroGen are publicly accessible. Thus, scientists are giving all interested enthusiasts of video games, robotics, and large language models the opportunity to experiment, fully utilizing their capabilities.
About the Author
