Meet NVIDIA Xavier: A new brain for self-driving, AI, and AR cars

NVIDIA isn't letting up on its goal of being at the heart of as many driverless cars as possible and it's NVIDIA Xavier, its newest autonomous machine processor, it plans to do that with. Unveiled at CES 2018 today, Xavier is the world's most powerful System-on-Chip (SoC), so NVIDIA claims, and comes at the tail-end of $2bn in R&D by the company.

It's not, of course, NVIDIA's first ride in this particular self-driving rodeo. The company has already been working with automakers and researchers on autonomous AI, most recently with the NVIDIA DRIVE PX2.

Xavier, though, will be fifteen times more energy efficient than the previous-generation SoC used in the DRIVE PX2 platform. With upward of nine billion transistors, NVIDIA is arguing it's two years ahead of anything its competitors are doing in terms of performance. A next-gen chip gets a next-gen platform, too.

It'll be the heart of the NVIDIA DRIVE Pegasus AI platform, announced three months ago at GTC Europe. Indeed, two Xavier SoCs will be found inside the new computer, along with two next-generation GPUs also of NVIDIA's design. It'll be capable of 320 trillion operations per second (TOPS) of processing performance. "This is basically the requirement that we're seeing from our customers to process all the information that's coming in," Danny Shapiro, senior director of automotive at the chip-maker, says.

Over 25 different companies are already using NVIDIA's chipsets to work on autonomous robo-taxis, the company claims, and they're planning to transition to Pegasus. There are some clear reasons why: for a start there's the size, which is roughly akin to that of a license plate.

The heart of Xavier is a custom, 8-core CPU, paired with a new 512-core Volta GPU. There'a also a new deep learning accelerator, new computer vision accelerators, and new 8K HDR video processors. It's sampling to the first customers this quarter, but NVIDIA points out that any existing software or algorithms developed for previous DRIVE platforms will run with no modifications required.

That's DRIVE AV, but NVIDIA has more in mind for Xavier than just autonomous vehicles. At CES 2018 today, it's also announcing DRIVE IX and DRIVE AR, two more elements of its artificial intelligence for vehicles. They'll be based on Xavier too.

DRIVE IX is NVIDIA's vision of an intelligent vehicle experience, bringing AI into the user-experience (UX) of the car or robo-taxi, and using that as a smart assistant for the driver and passengers. It'll be focused on pulling in as much data as possible from sensors inside and outside of the vehicle, understanding who is approaching it, who's inside, what they're doing, and how they're trying to communicate with the AI and the overall driving experience.

For example, an AI powered by DRIVE IX could spot someone approaching with their arms full of groceries and automatically unlock the car and open the trunk. Inside, it could use gaze-tracking and head position to control the interface, or recognize if the driver is getting drowsy or distracted. NVIDIA even expects it to be able to read lips, helping voice interface systems to monitor who is talking even amid other conversations.

"Even if the car isn't fully self driving, we'll be able to create a much safer experience," NVIDIA's Shapiro suggests. "We'll be able to understand the hazards around them." By spotting an approaching bicycle in the blind-spot, for instance, a DRIVE IX powered car could temporarily prevent the door from opening and causing a collision.

DRIVE AR, meanwhile, is NVIDIA's push for Augmented Reality in automotive settings. "It's going to be expected from all future computer platforms," Shapiro points out, "and the car is no different."

Exactly how that might look inside a car will depend on the particular automaker, of course. However, NVIDIA is expecting anything from alerts and hazards being flagged, through to points-of-interest and infotainment interfaces. You could even have an AI co-pilot, shadowing your driving and offering insights and safety warnings though not actively operating the vehicle itself.