Google Cloud AI chips introduce 8th generation TPUs to boost model training

Google Cloud has officially announced its eighth-generation custom-built AI chips, which are divided into two distinct categories. The TPU 8t is designed particularly for AI model training, while the TPU 8i is dedicated totally to inference, processing user prompts after a model has been trained. 

These low-power chips provide significant upgrades, offering training speeds that are three times faster and an eighty percent improvement in performance per dollar spent.

Furthermore, Google asserts that its system can allow over one million TPUs to operate together in a single cluster, providing customers with considerably more computing power while consuming less energy and reducing costs.

Despite these massive hardware advancements, Google is not launching a full-scale assault against industry leader Nvidia just yet. Similar to other major cloud providers like Microsoft and Amazon, Google currently uses its custom TPUs to supplement existing Nvidia-based systems within its infrastructure rather than completely replacing them.

In fact, Google has officially promised that its cloud services will definitely feature Nvidia’s highly anticipated Vera Rubin chip later this year. While hyperscalers building custom artificial intelligence chips might eventually reduce their reliance on Nvidia, betting against the nearly five trillion dollar chip giant remains highly unprofitable in today’s market.

Demonstrating that collaboration remains essential in today’s tech world, Google announced a new engineering partnership with Nvidia.

The two companies are working together to improve computer networking, aiming to make Nvidia systems run more efficiently in Google Cloud. Their joint efforts focus on upgrading Falcon, a networking software technology developed by Google and open-sourced through the Open Compute Project, an influential organization focused on data center hardware.

If this collaboration succeeds, Google’s continued growth as a leading AI cloud provider will likely create more profitable opportunities for Nvidia in the future.