AMD’s Ryzen AI NPUs Can Now Run LLMs Locally on Linux — Here’s What That Means

AMD’s neural processing units are finally doing something useful on Linux. After months of driver work and community effort, Ryzen AI NPUs — the dedicated AI accelerators baked into AMD’s latest laptop and desktop processors — can now run large language models locally, no cloud required. It’s a milestone that matters more than it might seem at first glance.

Phoronix reports that the AMD XDNA driver stack for Linux has matured enough to support LLM inference on Ryzen AI 300-series hardware using the NPU directly. The work builds on AMD’s open-source XDNA driver, which targets the NPU silicon inside Ryzen AI chips — specifically the XDNA and XDNA 2 architectures found in Strix Point and later processors. These NPUs deliver up to 50 TOPS (tera operations per second) of AI compute, a number AMD has been marketing aggressively but that, until recently, had little practical meaning for Linux users.

That’s changed.

The key enabler here is the integration with the Ryzen AI Software platform and the open-source tooling around it, including support through ONNX Runtime and the Vitis AI framework. Developers have been working to pipe LLM workloads through AMD’s NPU using these tools, and the results are now tangible enough to benchmark. Phoronix’s Michael Larabel ran tests showing local LLM inference functioning on the NPU hardware under Linux, a first for this class of AMD silicon in the open-source world.

So why should you care? Because the industry is barreling toward on-device AI, and the operating system that runs most of the world’s servers and a growing share of developer workstations has been left behind in the NPU race. Intel has had NPU support on Linux for its Meteor Lake and Lunar Lake chips for a while now, largely through its OpenVINO toolkit. Qualcomm’s Snapdragon X Elite, with its 45 TOPS Hexagon NPU, has also been making Linux inroads, though driver maturity remains uneven. AMD was the laggard. Not anymore.

The practical implications are straightforward. Running LLMs locally means inference without sending data to a remote server. Privacy-sensitive workloads stay on the machine. Latency drops. And for developers building AI-powered applications on Linux, having NPU acceleration means they don’t have to burn GPU cycles — or battery life — on tasks the NPU can handle more efficiently.

But let’s be honest about the limitations. NPUs in their current form aren’t replacing GPUs for training or heavy inference. A 50 TOPS NPU is useful for running smaller models — think quantized versions of Llama, Mistral, or Phi-class models — not for spinning up a 70-billion-parameter behemoth. The performance ceiling is real. What NPUs offer is efficiency: lower power draw for sustained, lighter AI workloads. The kind of thing that makes sense in a laptop running all day.

AMD’s XDNA driver has been part of the upstream Linux kernel since version 6.7, when the initial DRM driver landed. Subsequent kernel releases have added functionality and stability improvements. The user-space stack, though, has been the bottleneck. Getting the firmware, compiler toolchain, and runtime libraries aligned for practical workloads took time. The fact that LLMs are now running through this stack represents real progress in the software layer, which has historically been AMD’s weak spot compared to NVIDIA’s CUDA dominance.

There’s a competitive angle here too. Microsoft has been pushing its Copilot+ PC initiative hard, requiring 40+ TOPS of NPU performance for the branding. That initiative is Windows-centric. Linux support for the same hardware means the open-source community isn’t locked out of the AI PC trend — a trend that AMD, Intel, and Qualcomm are all betting billions on.

And the timing matters. With AMD’s Ryzen AI 9 HX 375 and similar chips shipping in mainstream laptops from Lenovo, ASUS, and HP, the installed base of XDNA 2 hardware is growing fast. Every one of those machines has an NPU sitting idle under Linux unless the software catches up. Now it’s catching up.

For Linux developers and sysadmins evaluating AI workloads on client hardware, the takeaway is simple: AMD’s NPU is no longer vaporware on your OS. It works. It runs models. It’s not going to replace your NVIDIA A100 for serious inference, but that was never the point. The point is local, efficient, private AI on the hardware you already own. And on Linux, that’s finally real.