Local LLM Agent

Source: examples/local-llm

Overview

The local LLM agent runs a quantized language model entirely in-process using Transformers.js (ONNX/WebAssembly). No external server, no base URL, and no API key are required. The model is downloaded from HuggingFace Hub on first run and cached locally.

Project Structure

local-llm/
├── aixyz.config.ts         # Agent metadata and skills (standalone output)
├── app/
│   ├── agent.ts            # Agent definition with local LLM model
│   ├── agent.test.ts       # Integration test
│   ├── tools/
│   │   └── temperature.ts  # Temperature conversion tool
│   └── icon.png            # Agent icon
├── prewarm.ts              # Model prewarming script (used by Docker)
├── Dockerfile              # Multi-stage Docker image with prewarmed model
├── .dockerignore
└── package.json

Skills

Skill	Description
Convert Temperature	Convert temperature values between Celsius, Fahrenheit, and Kelvin

Agent

app/agent.ts

import { transformersJS } from "@browser-ai/transformers-js";
import { stepCountIs, ToolLoopAgent } from "ai";

export default new ToolLoopAgent({
  model: transformersJS("onnx-community/Qwen2.5-1.5B-Instruct", { dtype: "q4" }),
  instructions: instructions,
  tools: { convertTemperature },
  stopWhen: stepCountIs(10),
});

The model (onnx-community/Qwen2.5-1.5B-Instruct, q4 quantized) is loaded via @browser-ai/transformers-js, which is an official Vercel AI SDK community provider for Transformers.js.

Payment

This agent is free (scheme: "free") — no x402 payment is required to call it.

Running locally

cd examples/local-llm
bun install
bun run dev

Docker deployment

Because the local LLM model (~1 GB of weights) is downloaded and prewarmed at image build time, the container starts serving requests immediately with no cold-start delay.

Build the image

cd examples/local-llm
docker build -t local-llm .

Or use the npm script:

bun run docker:build

Run the container

docker run -p 3000:3000 local-llm

Or:

bun run docker:run

Endpoints available at http://localhost:3000:

Endpoint	Protocol	Description
`/.well-known/agent-card.json`	A2A	Agent discovery card
`/agent`	A2A	JSON-RPC task handler
`/mcp`	MCP	Tool endpoint

Templates

Basic

Advanced

Local LLM Agent

Overview

Project Structure

Skills

Agent

Payment

Running locally

Docker deployment

Build the image

Run the container

​Overview

​Project Structure

​Skills

​Agent

​Payment

​Running locally

​Docker deployment

​Build the image

​Run the container

Overview

Project Structure

Skills

Agent

Payment

Running locally

Docker deployment

Build the image

Run the container