Elon Musk’s newest AI venture, xAI, has successfully brought its supercomputer “Colossus” online.
Revealed over the Labor Day weekend, Colossus is specifically designed to train xAI’s large language model (LLM), Grok, which is set to compete with OpenAI’s GPT-4.
Musk, renowned for his swift execution and lofty ambitions, declared that Colossus is “the most powerful AI training system in the world.”
He also shared plans to double its capacity in the coming months by acquiring 50,000 of Nvidia’s advanced H200 chips, which boast twice the power of the H100 models currently in use. The supercomputer is based in Memphis, a location chosen by xAI this past June.
The rapid completion of this project is noteworthy, especially given the intense competition among tech giants like Microsoft, Google, and Amazon to secure Nvidia’s Hopper series AI chips amid the ongoing AI boom.
Musk has pledged to spend $3-4 billion on Nvidia hardware this year for both xAI and Tesla, with Tesla providing its AI chips to give xAI a head start.
Colossus will be instrumental in training Grok-3, Musk’s third-generation AI model, which is anticipated to become the most powerful AI globally when it launches in December.
An early beta version of Grok-2, trained on 15,000 Nvidia H100 chips, was recently released and is already being recognized as one of the most capable AI models.
The project has sparked concerns in Memphis, particularly over the strain on the city’s resources, as Colossus demands up to 1 million gallons of water daily for cooling and could draw as much as 150 megawatts of power.
However, Musk’s focus on speed and scale has been a driving force behind xAI’s operations, as he highlighted in a recent conversation with podcaster Lex Fridman.
To further strengthen xAI’s capabilities, the company secured $6 billion in a Series B funding round in May, supported by prominent investors such as Andreessen Horowitz, Sequoia Capital, Fidelity, and Saudi Arabia’s Kingdom Holding.
Musk has also hinted at plans to propose a $5 billion investment from Tesla into xAI, potentially accelerating the company’s growth even further.
