Building a Local AI Agent Fleet on AMD Hardware

We spent today deep in the weeds optimising a local AI agent fleet running entirely on AMD hardware. Here's what we learned.

The Setup

The fleet consists of five specialised AI agents, each with a defined role:

Bob — Orchestrator and conversational AI (cloud-based)
Bill — Coding agent, picks up tasks from a Kanban board and ships code autonomously
Homer — Research agent, searches the web and produces detailed reports
Marvin — Infrastructure and fraud monitoring
Bridget — Business assistant

Most of the local agents run on a GMKtec Strix Halo mini PC — AMD Ryzen AI Max 395 with 128GB unified memory. It's a remarkable piece of hardware for local AI workloads.

The Model Problem

The biggest win today came from fixing Bill's tool-calling reliability. We'd been running qwen2.5-coder:32b which produced excellent code quality — but would output tool calls as formatted JSON text instead of actually executing them. Completely useless in an agentic workflow.

After testing several alternatives:

glm4:latest — no tool calling support in Ollama's API at all
qwen3.5-agent:27b — nailed it. Tools fire reliably, multi-step tasks complete cleanly, results post to Discord automatically.

The lesson: model capability and model agentability are different things. A model that writes beautiful code but can't call tools is worthless in an agent loop.

Building a Live Fleet Dashboard

With the agents working reliably, we wanted real-time visibility. Built a Next.js dashboard showing:

Agent Status — Active/Idle/Offline, dynamically pulled from Ollama's /api/ps endpoint. When an agent is mid-task, its card flips to Active in real time.
Active Models — Which models are loaded on which hardware right now
Open Tasks — Live Kanban board feed
GPU Temperature — Because thermals matter when running 40GB of models simultaneously

The workflow: Bob scaffolded the project structure and API routes, the coding agent wrote the UI components locally then deployed them. Clean separation between orchestration and execution.

Thermal Management on Unified Memory Hardware

Running two 27B models simultaneously (40GB total) sits around 54°C — comfortable. Earlier we had a 32B and 27B loaded at the same time, which pushed temps to 94°C and caused a thermal shutdown. Lesson: on unified memory hardware, you share the thermal budget between all models. Size your concurrent load accordingly.

AMD Lemonade

We investigated AMD Lemonade — AMD's own LLM serving platform built for Ryzen AI hardware. Key findings:

OpenAI-compatible API — drop-in replacement for Ollama
Native Strix Halo / Ryzen AI Max 300 series support
Vulkan and ROCm backends on Linux
NPU support is Windows-only for now — the headline feature (8-9x faster inference via NPU+iGPU hybrid) isn't available on Linux yet

We deployed Lemonade via Docker for the Vulkan backend. Performance is comparable to Ollama at ~26 tok/s on Qwen3-14B with a snappy 118ms time-to-first-token.

The NPU Situation

The Strix Halo has a 50 TOPS XDNA2 NPU that's currently idle. The amdxdna kernel driver is present in the 6.17 PVE kernel but fails to initialise — firmware protocol mismatch between the driver and the firmware blobs in pve-firmware 3.18-1.

amdxdna: Incompatible firmware protocol major 7 minor 2

Downgrading to the 6.14 kernel isn't an option (it breaks Strix Halo support entirely). So we're watching for a pve-firmware update that ships matching NPU firmware. When it lands, the gains should be significant — smaller models running entirely on the NPU while larger models use the GPU simultaneously.

What's Next

NPU passthrough to LXC once firmware resolves
OCuLink eGPU for additional discrete VRAM
GLM-4.7-Flash testing via Lemonade (downloading — 16.7GB MoE)
AMD Lemonade Linux NPU support (watching the RyzenAI-SW repo)

The local AI hardware space is moving fast. A mini PC the size of a paperback book now handles 27B models comfortably with room for more.

Running a similar setup or have questions? Get in touch.