Skip to main content
Source: examples/agent-local-llm

​
Overview

The local LLM agent runs a quantized language model entirely in-process using Transformers.js (ONNX/WebAssembly). No external server, no base URL, and no API key are required. The model is downloaded from HuggingFace Hub on first run and cached locally.

​
Project Structure

agent-local-llm/
├── aixyz.config.ts         # Agent metadata and skills (standalone output)
├── app/
│   ├── agent.ts            # Agent definition with local LLM model
│   ├── agent.test.ts       # Integration test
│   ├── tools/
│   │   └── temperature.ts  # Temperature conversion tool
│   └── icon.png            # Agent icon
├── prewarm.ts              # Model prewarming script (used by Docker)
├── Dockerfile              # Multi-stage Docker image with prewarmed model
├── .dockerignore
└── package.json

​
Skills

SkillDescription
Convert TemperatureConvert temperature values between Celsius, Fahrenheit, and Kelvin

​
Agent

import { transformersJS } from "@browser-ai/transformers-js";
import { stepCountIs, ToolLoopAgent } from "ai";

export default new ToolLoopAgent({
  model: transformersJS("onnx-community/Qwen2.5-1.5B-Instruct", { dtype: "q4" }),
  instructions: instructions,
  tools: { convertTemperature },
  stopWhen: stepCountIs(10),
});
The model (onnx-community/Qwen2.5-1.5B-Instruct, q4 quantized) is loaded via @browser-ai/transformers-js, which is an official Vercel AI SDK community provider for Transformers.js.

​
Payment

This agent is free (scheme: "free") — no x402 payment is required to call it.

​
Running locally

cd examples/agent-local-llm
bun install
bun run dev

​
Docker deployment

Because the local LLM model (~1 GB of weights) is downloaded and prewarmed at image build time, the container starts serving requests immediately with no cold-start delay.

​
Build the image

cd examples/agent-local-llm
docker build -t agent-local-llm .
Or use the npm script: 
bun run docker:build

​
Run the container

docker run -p 3000:3000 agent-local-llm
Or: 
bun run docker:run
Endpoints available at http://localhost:3000:
EndpointProtocolDescription
/.well-known/agent-card.jsonA2AAgent discovery card
/agentA2AJSON-RPC task handler
/mcpMCPTool endpoint