Last week, Google announced the open-sourcing of TorchTPU, its long-anticipated version of the PyTorch framework. With this step, Google joins Amazon in an effort to break Nvidia's strong position in the artificial intelligence industry, where the vast majority of AI applications are primarily designed to run on Nvidia's chips, known as GPUs.
PyTorch is a widely popular open-source tool that AI developers and startups use to create, train, and deploy artificial intelligence models. It is estimated to be used by up to 85% of all AI applications. PyTorch was originally developed by Meta (then Facebook) in 2016, long before it was widely anticipated how significant an impact Large Language Models (LLMs) would have on the world. At that time, Nvidia's GPUs were already the standard hardware for AI research, which is why many of PyTorch's features were designed to work specifically with them. In 2022, Meta transferred the governance of PyTorch to the Linux Foundation, a non-profit organization managing many key open-source projects.
Google's version, named TorchTPU, allows AI applications to run on Google's proprietary AI chips, known as Tensor Processing Units (TPUs). It also significantly eases the migration of existing PyTorch applications to TPUs and, by extension, to Google Cloud. Developers can achieve this with a simple configuration change, without needing to rewrite their application code.
Amazon made a similar announcement, introducing its own version of PyTorch, called TorchNeuron. TorchNeuron enables porting AI applications to Amazon's internally developed AI chip, Trainium, and to AWS, also without rewriting code. Amazon further stated that it is going even further in its efforts to loosen PyTorch's dependency on Nvidia. It is contributing code to the main PyTorch project that separates various functionalities from Nvidia's chips, thereby making it more hardware-agnostic by default.
Need a hand with your website or app?
Get in touch and I will reply within 24 hours with a proposal for the next step.
However, Google may surpass Amazon in some respects. It states that it directly collaborated with Meta, Hugging Face, Inferact (creator of vLLM), RadixArk (creator of SGLang), and others to ensure that TorchTPU's functionalities and code libraries work well with these popular AI development tools, just like the original PyTorch.
Furthermore, the search giant launched a specialized tool, a suite of TPU agents, designed to assist developers in migrating their applications to Google's chips and cloud and in debugging during the process. It even updated its popular LiteRT tool, which runs AI on phones, laptops, and embedded devices, to better integrate with AI applications built on PyTorch.
None of this means that AI application developers will automatically abandon Nvidia hardware en masse simply because they can now easily move to alternative chips and cloud platforms. Nevertheless, Amazon and Google are clearly striving to end the advantage Nvidia has enjoyed to date.