The Future Of Gaming: What You Need To Know

In early 2019, Netflix CEO Reed Hastings told shareholders what he thought was the company’s stiffest competition. It wasn’t HBO, Disney+ or Amazon. It wasn’t cable television or movie theaters either. In his estimation, the biggest threat to Netflix’s continued dominance in entertainment was the video game Fortnite.

“It used to be ‘what to watch’ and now it’s ‘whether to watch,’” wrote Matthew Ball, former head of strategy at Amazon Studios. “And the answer is increasingly ‘no, I’m going to play a game.’” 

Just how often are people choosing to reach for a game controller instead of a TV remote? An overview of the numbers paints an illuminating picture. In 2019, the gaming industry generated $120 billion in revenue, and experts predict it could reach $200 billion within two years; 100 million viewers tuned in to watch players compete in the World Championship of the game League of Legends — a larger haul than the telecast of the Super Bowl; and by 2021, it’s projected that 2.7 billion people — about one-third of the global population — will be gamers.

Big tech companies are seizing the moment: Apple, Google and Amazon are all developing gaming products. In the fall of 2019, Apple released Apple Arcade, a game subscription service. Soon after, Google launched Stadia, which allows users to stream major game titles directly from the Cloud rather than fussing with downloads or a physical console. And Amazon — which in 2014 acquired Twitch, the most-popular platform to watch gamers play — is rumored to soon launch their own game-streaming service. These newcomers to the space will contend with gaming mainstays like Sony (Playstation), Nintendo (Switch) and Microsoft (Xbox) for market share.

The rising tide lifts all boats. Mobile gaming raked in more than half of all gaming dollars spent last year. Virtual reality and augmented reality games, while comparatively niche, continue to climb in revenue, too.

So what’s next? Culturally, it seems that gaming has broken out of niche corners of the world and will only continue to become more mainstream. But what tech innovations are shaping the future of video games, and how will they influence the gaming experience? 
 

ARTIFICIAL INTELLIGENCE IN VIDEO GAMES

Artificial intelligence is no longer the stuff of science fiction; it’s lodged itself into the fabric of our everyday lives — from Gmail’s Smart Compose to self-driving cars to facial recognition software. Video games are no exception.

AI AND NON-PLAYER CHARACTERS

The idea of AI has been expressed in gaming for decades — most prominently in non-player characters (NPCs), like the colorful ghosts in Pac-Man or the innocent bystanders in Grand Theft Auto. 

Characters like these are typically programmed with what designers call a “finite state machine.” In the plainest possible terms, this means that NPCs follow a script, or an if-then statement. “If the game is on, then chase Pac-Man,” the ghosts’ script might say. “And if Pac-Man eats a Power Pellet, then run away from him.”

In recent years, gamemakers have taken a more sophisticated approach to NPCs. For example, some are now programmed with a behavior tree, which enables them to perform more complex decision-making. The enemy aliens in Halo 2, for example, have the ability to work together and coordinate their attacks, rather than heedlessly beeline into gunfire one by one like they’re in a cheesy action movie.

Still, NPCs can only do what is written in their code. Their behavior, however intelligent it seems, is still determined in advance by the game’s designers. 
 

Even if we could give NPCs minds of their own and let them run free in games, odds are that their autonomy would result in a less fun experience for the player.

In the future, could we expect to see more advanced AI appear in commercial games? It’s possible, according to experts; but not all are convinced it’s coming anytime soon.

“You can try and build a really cool, comprehensive AI system which is about letting a character behave in all kinds of ways the designer hasn’t anticipated,” Mitu Khandaker, PhD, an assistant arts professor at NYU’s Game Center, told Built In. “But if there’s too much of that, there’s no guarantee about which way the story will go and whether it’s going to be any fun.”

In other words, even if we could give NPCs minds of their own and let them run free in games, odds are that their autonomy would result in a less fun experience for the player. A rogue NPC could decide to shirk its duty to help the player advance toward the next level, or take the player on a nonsensical quest where nothing happens. Giving NPCs the ability to behave in ways unanticipated by the game’s designer may sound like an amusing Westworld-like experiment. But if it leads to an inferior gaming experience, why bother?

In addition to presenting game design challenges, free-range NPCs may be a non-starter when considered from a purely economic perspective, as well.

“Games are a pretty conservative industry, in terms of the willingness that publishers or studios have to take risks,” Khandaker said. “Because there is such a great history in terms of design for what does work in games, there’s a real sense of wanting to keep doing that same thing.”

Putting more sophisticated NPCs in games may be possible. But if it costs a lot of money, and fails to improve the player’s experience, studios lose an incentive to make it happen. The sales numbers speak for themselves, they might conclude. Players are content with what’s already out there.

Still, some designers persist in NPC enhancements, especially in figuring out ways to make NPCs more believable. 

Khandaker has spent much of her career figuring out how to build digital characters who speak naturally and behave more like real-life friends than bots.

She points to the proliferation of digital assistants like Alexa and Siri as proof of concept that humans are ready to embrace virtual friends.

“I do see a future where, within ten years, whether it’s through mixed reality headsets or looking at AR through our phones, we’ll have this concept of, ‘Oh, I hang out occasionally with this NPC who remembers me and who I have this conversation with.’”

AI AND CONTENT GENERATION

Artificial Intelligence isn’t just part of the gameplay experience. It’s part of the game-making experience. For several years now, designers have been using AI to help them create games. 

AI can generate game assets, which frees up designers from painstakingly drawing each individual tree in a forest or rock formation in a canyon. Instead, designers can offload that work to computers by using a technique called procedural content generation, which has become fairly standard practice in the industry. 

Procedural content generation is also used to create game levels — sometimes randomly — so the player can enjoy a fresh experience each time.

The 2016 game No Man’s Sky took this technique to the extreme. The entire open-world environment of the game is procedurally generated, meaning that the player traverses through a world that was not sketched out ahead of time by the game’s creators.

Some gamemakers also rely on neural networks to tailor-make game levels for players through a process NYU professor Julian Togelius calls experience-driven procedural content generation. For example, in 2009 researchers collected player data for Super Mario, quantifying each player’s preferences as they played. Maybe a level had too many jumps and not enough sewers, or coins were hard to reach and bad guys were too easy to defeat. Researchers fed player data to a computer. Once the computer digested the information, it spat out new levels that reflected the player’s preferences.

While AI generating game assets and, in some cases, entire levels, the livelihoods of human designers aren’t in jeopardy — at least not yet.

“For the foreseeable future, we will not have AI systems that can design a complete game from scratch with anything like the quality, or at least consistency of quality, that a team of human game developers can,” Togelius wrote in his 2018 book Playing Smart.

Academics and game designers alike are still trying to implement AI systems that will control the game in such a way that is fun for the player.

One such academic is Rogelio Cardona-Rivera, PhD, an assistant professor at the University of Utah who introduced himself by saying “I primarily care about building Star Trek’s the Holodeck, or the technology responsible for powering Westworld.” (The Holodeck and Westworld are two prominent pop culture touchstones that have captured the imaginations of technologists with their depictions of AI and VR.)
 

Cardona-Rivera envisions a future in which AI acts as a game master that calls the shots for a human player.

“Imagine what it would mean to have an AI ‘director’ who’s looking at what you’re doing and directing the unfolding experience for you,” he said. “That’s kind of like what my research is trying to do and what a lot of interesting work in the field — not just me — is trying to do.”

Until they figure that out, we’ll continue to see human designers and computer algorithms working together to create the next generation of video games.

AI AND ANALYTICS

AI may not yet be up to the task of creating entire high-quality games from scratch, but it can certainly provide valuable feedback to the game’s designers, who can fine-tune their creations on the fly. It’s quite common.

“It would be hard to find a commercially released game that does not ‘phone home’ to the developer with information on how it’s being played,” Togelius wrote.

Games routinely collect data on how a player experiences a game. This information is fed into an algorithm and is ultimately used by humans to tweak games based on its predictions of what players will like. Sound weird? Think Netflix nudging you toward your next show, Spotify building you a custom playlist or Amazon recommending a product based on your past browsing history.

“Games are so much about what creates a compelling experience that people will keep coming back to,” Khandaker said. “That’s basically what AI is in service of.”
 

VIRTUAL REALITY VIDEO GAMES

For decades, virtual reality has tantalized gamers with the prospect of a fully immersive experience. But the technology has yet to deliver on this promise.

Tech companies are looking to change that. Huge firms like Facebook, Google, Microsoft and Sony — and startups like Magic Leap — have invested considerable resources to develop VR hardware and games.

VR is still a niche category when compared to the rest of the gaming industry. And despite its buzzy status, it continues to give many consumers pause.

“Right now we’re sort of in this trough of disillusionment about VR,” Kevin Mack, a VR game developer, told Built In. “There was a lot of hype around it in 2015 and 2016, and then the whole world sort of got butt-hurt that their first-generation VR headset didn’t instantly morph into the Holodeck.”

When it comes to VR, we’re still in the early days. There are promising developments on the horizon. But first, a few challenges need to be addressed.

WHAT’S HOLDING VR BACK

For the future of consumer VR to be more than a niche hobby for people with high disposable income — or a once-in-a-while theme park experience — several hardware challenges will need to be overcome. Namely, the bulky headsets.

Most VR headsets weigh well over a pound and must be strapped tightly to a user’s face. It’s not terribly comfortable. You get sweaty and after a half-hour of play your energy is sapped.

This experience chafes against that mode of playing which is typical of gaming enthusiasts — spending hours comfortably sunk into a couch. If VR hardware can’t align with the preferences of gamers, will it be able to survive? Until companies slim down their VR headsets — and price points — most gamers, save the early adopters and tech enthusiasts, will continue to balk.

“I think that the future of VR is more through social VR.”

Companies are busy making VR more consumer-friendly, and it’s only a matter of time before both the weight and price of headsets drop. But even when those hurdles are cleared, the fact that the typical VR experience is so socially isolating might limit its upside.

“[VR] is a solitary experience. It’s a thing that you’re doing on your own and it’s a thing that you choose to do to the exclusion of anything else,” Mack said. He enjoys playing VR games, but if someone else is around, he thinks twice before strapping the headset on.

“I still generally wouldn’t really wear one that much at home if my girlfriend is there too,” he said. “Because I feel like I was completely cutting myself off from the social environment.” 

Though he recognizes the limitations, Mack remains optimistic about VR’s future.

“VR I think will remain niche, but it could potentially turn into a big niche,” he said. “I think we’re going to see some very impressive stuff and very compelling stuff come down the pipe in the next couple of years.”

Khandaker, too, is hopeful about VR’s role in gaming. She just doesn’t think it’s going to look like people alone in their homes playing through a headset, so much as a co-located experience that multiple people share in.

“I think that the future of VR is more through social VR,” she said.

THE RISE OF HYPERREALITY

To understand what social VR is, one only needs to look at the VOID, a company specializing in “hyperreal” experiences, or location-based entertainment (LBE), which blends together virtual reality and physical reality.

Here’s how it works. Players gather in a physical space, like the inside of a warehouse, and strap on helmets and computers that double as backpacks. They proceed to participate in a virtual activity together, a mission. But unlike the typical VR experience, where everything takes place inside the goggles, hyperreality game players will notice that what they see in their headsets actually corresponds to the physical space of the room. So they can run around, duck and in some cases, even reach for a virtual object and feel it — in the form of a strategically placed physical prop. The interplay between virtual reality and tactile feedback creates a uniquely immersive experience.

The VOID makes games in which players assume roles in Star Wars, Avengers and Jumanji. Even the backpack you wear, used to store the computers that power the game, is baked into the storyline.

“It’s a kind of where VR meets AR. You are touching something real and augmenting it in some way, whether you’re overriding your eyes or not,” Cardona-Rivera said. He called the hyperreality experience magical.

“I see that as kind of like the future. But what exactly that will entail, I don’t know,” he said.

Mack, who’s been busy working on a hyperreality game in Los Angeles, sees this type of entertainment as an on-ramp experience that introduces everyday people to the possibilities of virtual reality — even if it doesn’t instantly convert them into enthusiasts.

“I think for a lot of people it’s going to be their first exposure to VR,” he said. “It’ll be like a theme park experience. But you know, they’re not necessarily going to go and follow that up and start putting headsets in in their homes.”
 

AUGMENTED REALITY VIDEO GAMES

In the summer of 2016, parks and plazas swarmed with smartphone-wielders on Pokémon-catching missions.

The masses were playing Pokémon Go, an augmented reality mobile game in which virtual images — in this case, colorful critters called Pokémon — overlay a person’s view of the real world. 

The game, which is free to download and play, has since generated over $3 billion in sales, mostly from in-app purchases.

AR BRINGS PEOPLE OUT — AND TOGETHER

The technology that powered Pokémon Go wasn’t new. That sort of geo-based AR gameplay was already around. It took a well-established brand (Pokémon) to get consumers to give it a try. And once they played it, they loved it.

But the long-term success of Pokémon Go is due only in part to its beloved IP. There are plenty of other games and books and movies in which people can spend time with Ash Ketchum and Pikachu. The real secret sauce is the game’s blend of virtual and real, the interplay between digital characters and physical locations.

That’s partly why AR is taking off faster than VR: People have an appetite for games that interact with reality, not remove them from it.

“I think the entertainment experiences in AR aren’t going to try to be immersive experiences,” Mack said. “When I was playing [Pokémon Go], I would go to specific places just because there was a Pokémon there. And that’s a powerful social driver.”

Further out into the neighborhood — rather than deeper inside goggles — was the x-factor that led to the network effect that propelled Pokémon Go into a multi-billion-dollar phenomenon. Its success will no doubt inspire more game studios to try to capitalize on the consumer demand for games that blend the virtual with the real.

“I could totally see a game where you’re playing hide and seek or some kind of laser tag,” Mack said. “It’s a natural fit at that point.”

Cardona-Rivera likewise predicts that, in the short term at least, AR will prove to be more fertile ground for game designers than VR. 

“Instead of trying to simulate reality altogether, I think designers might find complementing reality a more trackable design challenge,” he said. “And then we might see some of the lessons from AR folded back into VR.” 

MOBILE 5G: THE AR GAME CHANGER

If you’ve ever used your phone to stream a movie or play an online game, you’re likely well-acquainted with the spinning wheel indicating your content is loading. That’s because internet speeds are slow. It’s kept mobile gaming’s ceiling lower than it could be.

To remedy this, telecommunications companies are in the early stages of rolling out 5G for mobile devices. Soon, smartphones will be much faster, and laggy game experiences will be a thing of the past.

This is not just great news for mobile games in general, which currently chew through a phone’s computational power. It’s also great for AR games specifically, which, without real-time data input, aren’t worth playing.

With 5G capabilities, you will be able to pull up an AR game, look through your screen and — with lightning speed — get instantaneous data on the world around you.

For example, Mack said, “Imagine that people are in a social AR experience, and they created custom avatars for themselves, so anybody else who’s using the same application to overlay their experience sees this person in whatever costume they designed for themselves … You’re going to need to have some fast way to make that data available.”

The technology behind 5G promises to answer that. This breakthrough will not only make mobile gaming better, it will likely create a network effect in which more and more people get into mobile gaming — especially AR — due to an increase in cultural buzz about the high-quality experience of AR gaming.
 

HIGH-FIDELITY GRAPHICS IN VIDEO GAMES

Have you ever revisited an old video game that you enjoyed when you were younger — at which point you swore the graphics were incredible, the peak of realism — only to recoil in confusion, wondering how you ever thought that the splotchy pixelated shapes, barely recognizable as an approximation of a human face, were as realistic as games could get? The answer is unknown. But the fact is you did.

Mack echoes a similar sentiment.

“I remember when Skyrim first came out, looking at it and saying, ‘My God, what are we going to be looking at in five years that makes this look primitive?’ And now when you play Skyrim, it kind of does.” 

In the pursuit of ultra-realistic graphics, video games have come a long way. Are we there yet?

Recently, the company Nvidia created a graphics card that enables video games to feature something called ray tracing. It’s a technique that allows light in games to behave more naturally.

In the past, things like shadows and reflections and lens flares were essentially painted onto objects within the game. This gave the illusion that light was coming from the sun or moon and reacting as it would when it hit a surface. With ray tracing, an algorithm basically allows it to do just that, but for real this time.

The technology is in its nascent stages, but it’s expected to be a game changer. Graphics will only get better. That is, if companies want to pursue it.

The way Mack sees it, there are two distinct routes game developers can take when it comes to graphics. 

One approach is what you see happening in major AAA games, which is to hire tons of visual artists and technicians to supply vast amounts of art for high-fidelity graphics. That means big budgets, big teams and increasingly realistic graphics, down to every last speck of dirt. 

The other approach is to produce a more stylized — in some cases cartoonish — aesthetic for your game. That way, the costs stay down but the game still looks cool and dodges the criticism that it doesn’t look realistic! Mack said this approach is becoming more and more common in the mobile VR space.
 

TOWARD A COMMUNAL EXPERIENCE

The future of entertainment is gaming. And the future of gaming is community.

Just look at Pokémon Go and Fortnite, the two biggest game sensations of the past five years. The former projects to have over 67 million active users in the U.S. in 2020. That’s the combined populations of California and Texas regularly leaving their home to visit public spaces alongside other gamers. The latter boasts roughly 250 million registered users, which means a quarter-billion people are hanging out at what journalist Keith Stuart described as a “digital Third Place” that more closely resembles a skate park than an arena for competition.

Augmented reality will continue to unlock more, not less, social interactions, because encountering the real world is what distinguishes AR from VR. The quality of an augmented reality game hinges, first and foremost, on the presence of reality.

When it comes to VR games, people are seeking out eventized versions of it with their friends, coworkers and dates, treating otherwise-isolating tech like past generations treated bowling alleys and ropes courses. When VR incorporates embodied elements — the ability to feel real-world objects with your hands, or use your legs to walk in a game rather than a joystick — players are spellbound.

If the current trends and future forecasts of the gaming industry clue us into anything about ourselves, it’s that our desire to escape is far outpaced by our desire to connect.