Summary
- Nvidia showcased impressive AI capabilities in gaming, allowing for realistic interactions and conversations between virtual characters, enhancing immersion in video games.
- Chat with RTX, an AI-powered chatbot, can be trained using custom datasets stored locally, providing up-to-date and personalized responses without relying on the internet.
- Nvidia demonstrated iStock, a feature that generates AI images using a clean dataset, ensuring no licensing issues. Users can also create more complex images using Nvidia Picasso, with assets that are always free to use.
AI is the buzzword of the year at CES 2024, and pretty much every company has something to say about it. However, hearing about things and getting to see them in person is a very different experience, and Nvidia had some really cool stuff to talk about at the event. Of course, the company announced new RTX Super cards, but what's even more interesting is what you can do with them.
Nvidia gave us a tour of some of the products and features it has at CES 2024, and I walked away very impressed with some of the stuff I saw.
Talking to AI characters in games
Going behind the limits of scripted interactions
One of the first demos I saw at Nvidia's showcase was of Nvidia ACE, a feature that aims to implement proper AI capabilities in videogames, making it possible for characters to react and respond to anything the player says, but also have unique conversations between themselves. The demo was done in partnership with a company called Convai, and it was really cool.
First, we could start by entering a prompt for any topic we wanted, and the two virtual characters in the game environment would be talking between themselves about the topic we chose. In the demo I saw alongside fellow journalists, we had the characters in a ramen shop talk about ravioli, and aside from the silliness of the topic, the conversation flowed pretty naturally.
But it is a video game, so it's not just about interactions between the virtual characters. When we approached the characters in the game, we were able to speak to them using a microphone, and whatever we talked about, the characters in the game would respond accordingly. The demo took place in a ramen shop, so you could ask for a bowl of ramen in different flavors. We also tried asking for a different color of bowl, and the character responded that they didn't have different colored bowls.
We tried bringing up a few topics, like if either of the characters had a desktop PC at home. One of them responded that they didn't have one, but that the other character probably would since they were fans of all things tech. It was crazy how natural the conversation felt, and this is the kind of stuff we can expect to see in games in the coming years. The teams at the show said we'll probably see this kind of stuff in indie games first, since AAA releases have much longer development cycles, but it will be really cool to see either way. A cool thing about this is that developers can allow characters to talk about anything, but they can also put in guardrails to safeguard against certain topics being brought up.
Offline chatbots with private data
Provide your own datasets and process them without the cloud
Another really cool AI demo Nvidia was showing off was related Chat with RTX. The idea behind Chat with RTX is that typical LLMs will only be able to answer with information that was in their training dataset, so they may not always have the most up-to-date info. Plus, they often require a connection to the internet to function properly.
Chat with RTX was interesting because you can provide your own dataset for the chatbot to respond to. You simply point the chatbot to where you stored the data you want to use, and it will be able to respond based on the information you provided. In the demo I saw, Nvidia had the bot pointing to documents containing the company's press releases, which meant the bot could tell you all about the latest announcements. You can point it to any folder on your PC, though, and it will scan all kinds of text files to get you the information you need. It even scans sub-folders, so you can create an organized repository of information to have your questions answered at a moment's notice. This experience runs completely locally on an RTX GPU, too, so you don't have to worry about that data being sent to a server anywhere.
Another option Nvidia showed off was pointing the bot to a YouTube video. Chat with RTX can access the transcript for the video and answer any questions about the contents of the video. For example, if you don't want to watch a long video to find out what someone thinks of a certain product, you can ask the chatbot to scan it for you. I really liked this demo and found it pretty useful, and I'm told it'll be available for end users in the near future.
Generating safe-for-use AI images
Training with a clean dataset
One other demo Nvidia showed off was about AI-generated images, and the risks that sometimes come with them since they might not always be safe for use from a licensing standpoint. Nvidia was showing off iStock, a new feature from Getty Images that generates images using a clean dataset with fully safe images to use, so you can be sure no assets in your images will have licensing issues. You can change settings like generating images with different aspect ratios or choose a resolution. iStock even includes a prompt builder, if you're not sure how to provide the right input for AI to generate a good image.
Nvidia did have its own app, called Nvidia Picasso, where you can use this API from Getty Images to create more complex images. Not only do you generate a single image, but you can mask parts of the image and generate images to fill in those parts For example, in the demo we saw, there was an image of a snow-covered forest, and our guide asked Nvidia Picasso to generate a woman that was standing in a specific part of the image. Then, with a different mask, he also asked the app to generate new clothing for the woman. All the assets generated were always free to use, so the final result causes no legal issues. I asked, and Getty Images is also aware that many companies would want to add their own training data for the data set, so they can use images licensed to them specifically. Nothing has really come of that yet, but you can see how that could be useful once it becomes possible.
It is worth noting that this all runs in the cloud, though, so this isn't leveraging RTX GPUs for the AI assets.
You can use GPUs for gaming, too
And there's a good feature for up-and-coming Twitch streamers, too
In the era of AI, it almost feels like gaming is an afterthought, but you can, in fact, play games on the latest Nvidia GPUs, and the company was showing that off at CES 2024. For starters, I got a look at a ton of partner cards based on the new GeForce RTX 4070 Super, 4070 Ti Super, and 4080 Super. There are a ton of designs, but some of my favorites include models from Zotac, thanks to the curvy design that's pretty unique, and the Colorful iGame RTX 4070 Ti Super. Even without RGB, this is a beautiful card thanks to the sort of iridescent look.
As for gaming demos, we saw what you'd expect, like a comparison between the Nvidia GeForce RTX 2070 and 4070 Super in a game like The Finals. Even without frame generation, the new card was delivering well over twice the frame rate, so it's a big upgrade. Nvidia pointed out that only about 30% of customers are on the RTX 30 series or newer, so that's why the company compared to an older, more popular card.
A demo that I found particularly interesting was showcasing ray reconstruction (DLSS 3.5), and the kind of quality enhancements you can get from it. While it was very hard to capture in a photo, you can clearly see some benefits to the image quality when using the ray reconstruction denoiser. In this Alan Wake 2 screenshot, for example, you can see the shadows from the curtains preserve a lot more detail when using ray reconstruction compared to native rendering. We saw some similar things with light reflections in Cyberpunk 2077, too.
Other demos were less about showing off specific features and more about just having fun. There was a driving sim experience with Forza Horizon 5 on a massive 49-inch super ultrawide monitor, which was pretty fun if you want an immersive experience. There were also some gaming laptops, despite there not being any new Nvidia hardware for laptops at the show.
One last thing Nvidia showed off was a new capability for enhanced Twitch streaming in OBS Studio. If you're not a Twitch partner or affiliate, you might know that you can only provide streams in one resolution, so you have to choose between offering a high-quality stream or a stream that's actually accessible to most viewers. Nvidia GPUs now offer a feature that allows you to encode multiple streams in different levels of quality, so you can provide all those quality options for viewers even if you're not a big-time streamer yet. And even if you are, this real-time encoding done locally can actually offer lower latency than the cloud transcoding offered by Twitch to its affiliates. Granted, this will use more resources on your PC, but Nvidia GPUs are plenty powerful to where it might not matter to you.
AI is getting interesting
If you've seen a lot of my writing, you know I haven't found the uses for AI to be particularly interesting, but I was really impressed by what Nvidia was showing off here. I think the most practical tech is the ability to create more realistic interactions with NPCs in videogames, and I can see it making all kinds of games so much more immersive. Games that emphasize character interactions can really be taken to a whole other level with this kind of capability.
I also do like the idea of chat with RTX, though. I can imagine saving all my documents related to a specific topic in one folder and have the app surface relevant information for me without relying on the cloud. It's probably a more niche use case, but it's still neat, and there's potential for extensions to increase the versatility of the tool by building it into an app so it can access your information within that app, for example.
AI seems to be here to stay, and seeing how companies are exploring its potential does make it all the more exciting.