Generative AI can create text, audio, images, and 3D assets. The games industry combines these media – and could therefore benefit greatly.
At least that’s what venture capitalist Andreessen Horowitz (a16z) assumes, making him the second major Silicon Valley investor after Sequoia Capital to embrace the potential of generative AI. In a blog post, the company specifically addresses the benefits of generative AI for the gaming industry, where it expects the technology to have the most significant impact.
Magic potions made by machines
The basic thesis of investors: games are the most complex form of media and therefore have a particularly high barrier to entry into development. Generative AI lowers this barrier by reducing the complexity of asset production while increasing quality, quantity, and speed.
“There hasn’t been such revolutionary technology for games since real-time 3D,” write authors James Gwertzman and Jack Soslow. “There hasn’t been such revolutionary technology for games since real-time 3D.”
Early conversations with developers showed they were excited about the “dramatic reduction in time and cost”, they say. One developer says he’s cut the time it takes to complete a single piece of concept art from three weeks to an hour. The venture capitalist expects similar savings rates across the entire production pipeline.
The developer “Emm” shows concrete examples on Twitter. He focuses on producing 2D assets with Stable Diffusion. It does this by refining Stable Diffusion with sample assets using the Dreambooth API. This allows it to generate game-specific and consistent graphics. With Scenario_gg, it plans to offer a platform for AI-generated gaming assets from December.
.@Scenario_gg allows you to create endless cohesive content, similar to what I’ve described in my recent articles (swords, chests, books, characters, potions, maps and more)
Game creators can create amazing, high-quality assets in hours (compared to days or weeks with a traditional process) pic.twitter.com/14JNvBmhxF
— Emm (@emmanuel_2m) November 17, 2022
While the productive use of generative AI for concept art, 2D assets, and text is already underway, 3D objects are a step further. Early approaches include Google’s Dreamfusion research project and Nvidia’s GET3D, which is open-source.
More games, new games and micro studios
The following graphic from a16z shows an overview of commercial projects and startups working on generative AI that game studios might be interested in. Despite all the excitement, generative AI for games is in its infancy, writes a16z. In addition to large foundational models like Stable Diffusion, AI tools developed specifically for the industry could add value.
The venture capitalist expects the use of generative AI to become a professional discipline in its own right. Additionally, if generative AI lowers barriers to entry, it could lead to riskier and more creative game projects.
Another possible outcome of the generative AI revolution could be “micro game studios” built on AI technology that have only one or two employees but still create extensive and elaborate games.
This would in turn lead to an even higher number of games – the problem of marketing games would thus be exacerbated – but perhaps also to the development of new ideas and genres of games.
For example, a16z mentions the Arrowmancer role-playing game, in which players can experience new adventures with endless AI-generated characters.
An even better example is the text-based AI RPG Roguelite, where all locations, characters, enemies and objects are generated by GPT-3 and VQGAN-CLIP (see video). A pioneering project in the field of AI-generated games is the text and image adventure AI Dungeon.
Andreessen Horowitz sees potential stumbling blocks in copyright issues. The big game studios, in particular, are unlikely to make their resources freely available to train or refine publicly available AI models.
Stable Diffusion, for example, is formed with datasets that incorporate unauthorized and copyrighted material. Stable Diffusion co-inventor Emad Mostaque recently announced that future training datasets should only include licensed material and offer an opt-out option. Solutions are therefore probably already on the horizon.
Andreessen Horowitz sees the impact of generative AI primarily in the creative realm. Game programmers should have greater job security despite increased code automation with tools like Copilot. Code generation always requires more human intervention and therefore has less impact on productivity, according to the company.
AI already offers another advantage for games: in graphics rendering. Currently, the most popular example is Nvidia’s DLSS, which uses AI scaling to achieve significantly more performance with the same hardware. Wider dissemination of these methods with increasing efficiency could lead to even faster development of sophisticated real-time 3D graphics.