Musk Says Grok’s Training Stack Is Now Built in C and C++

Elon Musk has revealed that the software stack used to pre-train xAI’s Grok models is now built internally using C and C++.

Musk said in a post on X that Grok 4.8, which he described as a 2.5-trillion-parameter model, is expected to complete pre-training this week before moving into reinforcement learning.

Musk said xAI’s reinforcement learning infrastructure remains more fragmented, describing it as “a legacy pile of many different third-party software systems” that the company plans to replace over time.

The disclosure has sparked discussion among engineers about whether lower-level programming can deliver efficiency gains for large-scale AI training.

Microsoft Elevates Rust to Tier-1 Language, Unifies Rust and C++ Development on Windows

The move is part of Microsoft’s broader effort to expand Rust across its software stack, including firmware, drivers, kernels, hypervisors, microservices and applications.

The Left ShiftThe Left Shift Bureau

Google software engineer Patrick C Toulme suggested that xAI’s C++ stack could be contributing to its model development speed through higher model FLOPs utilisation (MFU).

Musk also said the stack uses assembly code and direct calls to NVIDIA GPU accelerators, allowing the software to optimise more closely for the underlying hardware.

Andrej Karpathy Says AGI Decades Away, Calls AI Agents ‘Slops’

Karpathy’s comments come amid a frenzy of AI agent launches, with companies touting autonomous digital assistants that can browse, code, and execute complex workflows.

The Left ShiftThe Left Shift Bureau

Elon Musk Now Wants to End GitHub’s Dominance

Just as Cursor is moving into Git hosting, GitHub suffered one of its most disruptive outages of the year, taking down or degrading the website, API, Actions, pull requests and Copilot for developers worldwide.

The Left ShiftThe Left Shift Bureau