Mark Zuckerberg revealed during Meta’s recent earnings call that training the next iteration of their foundational language model, Llama 4, will demand a tenfold increase in computing power compared to its predecessor, Llama 3. This projection underscores Meta’s commitment to staying ahead in the competitive AI landscape by scaling its infrastructure ahead of anticipated needs.
Zuckerberg expressed a forward-thinking approach, stating, “The amount of computing needed to train Llama 4 will likely be almost 10 times more than what we used to train Llama 3, and future models will continue to grow beyond that.” He acknowledged the unpredictability of future trends but emphasized the importance of preemptively building capacity to avoid falling behind.
Meta’s latest release, Llama 3.1 405B, marks a significant upgrade with 405 billion parameters, making it the company’s largest open-source model to date. This follows the earlier release of Llama 3, which had 80 billion parameters. The evolution of these models reflects Meta’s ambition to lead in AI innovation.
Susan Li, Meta’s CFO, also highlighted the company’s strategic focus on expanding data center projects and AI training capacity. She indicated that this expansion would significantly boost capital expenditures in 2025, a move aligned with the company’s ongoing investments in servers, data centers, and network infrastructure. In Q2 2024, Meta’s capital expenditures surged by nearly 33% to $8.5 billion, driven by these investments.
Training large language models is a costly endeavor. For comparison, OpenAI reportedly spends $3 billion on model training and an additional $4 billion on renting servers from Microsoft. Meta’s approach involves building a flexible infrastructure to manage its growing AI needs, allowing for efficient allocation of resources between generative AI training and core ranking and recommendation systems.
Despite these ambitious plans, Meta does not anticipate immediate revenue contributions from its generative AI products. During the earnings call, Li noted that while India remains the largest market for Meta AI’s chatbot, the financial impact of these AI products is not expected to be substantial in the near term.
Source: TechCrunch