Klad: A Paint App Built for AI Coding Agents

The developer behind the Klad painting application released it as a source-available tool built specifically for AI coding agents to control, as showcased on Hacker News. The app exposes a local WebSocket API that agents like Cursor’s Composer or Claude Code can use to execute drawing commands programmatically.

Tool: Klad | License: Source-available (BSL) | API: Local WebSocket
Language: Go | Platforms: macOS, Windows, Linux

How the Agent-Driven API Works

Klad functions as a standard local paint application but runs a WebSocket server on localhost. Instead of manual mouse input, an AI coding agent sends JSON-formatted commands to this server to manipulate the canvas. The command set includes primitives like stroke, fill, clear, and undo. This turns visual art generation into a code-driven workflow where the agent writes the drawing instructions.

The architecture is deliberately simple. The agent needs only to establish a WebSocket connection and push command objects, making it compatible with any AI assistant that can write and execute code. Early testers on the HN thread reported integration with Cursor IDE’s agent within minutes.

Benchmarks: Latency and Agent Compatibility

While not a traditional benchmark, the critical metrics are connection latency and command execution speed. Since communication is local, response times are sub-10ms, making the feedback loop between agent and canvas near-instantaneous. The more relevant "benchmark" is agent capability: successful use depends entirely on the coding agent’s ability to reason spatially and generate correct JSON commands.

The HN discussion (38 points, 14 comments) highlighted this as the main challenge. One comment noted, “Getting an agent to draw a coherent shape is harder than it looks,” underscoring that this is a test of agentic reasoning, not raw application speed.

Pros, Cons, and Immediate Trade-offs

Pros:

• Agent-First Design: The entire tool is built for this single use case, reducing configuration friction.
• Lightweight and Local: A native Go application with no external dependencies or cloud costs.
• Simple API: The limited command set is easier for agents to learn and use correctly.

Cons:

• Very Basic Features: Lacks layers, advanced brushes, text tools, or most features expected by human artists.
• Agent Reliability is Key: The utility is 100% dependent on the coding agent’s often-unreliable spatial reasoning.
• Niche Use Case: Useless as a daily driver for manual digital art creation.

Alternatives for AI-Driven Image Generation

Klad occupies a unique niche between full image generators and scriptable design tools. For developers building agentic workflows, the alternatives fall into two categories.

For scripted drawing, Aseprite with its Lua API is far more powerful but requires more complex agent output. For direct image generation, calling a Stable Diffusion API (like Replicate or Together.ai) or automating ComfyUI is the standard approach, but it doesn’t allow for incremental, stroke-by-stroke editing.

Who Should (and Shouldn’t) Use Klad

Use Klad if: You are an AI agent researcher or developer testing an agent’s ability to execute precise, multi-step spatial tasks. It’s a compelling sandbox for evaluating reasoning and planning in a visual domain with immediate, observable results.

Skip Klad if: You need a practical tool for generating assets or artwork. If you want high-quality images, use a diffusion model API. If you need scriptable vector graphics, use Puppeteer or Playwright to control a browser-based SVG editor.

Bottom line: Klad is less a production art tool and more a specialized benchmark or prototype environment for AI coding agents.

Klad’s release signals a growing focus on building tools that accept agent commands as a primary interface. Its value isn’t in its feature set, but in providing a clean, minimal testbed for the next challenge after code generation: getting AI to reliably manipulate the visual world.