Building ElevenAgents with Claude Code

This shows a flow of assembling an ElevenAgents voice agent in half a day in Claude Code using only natural language prompts. It first creates the persona, voice, and LLM, then layers on a knowledge base, workflow, tools, guardrails, and tests in order, ultimately connecting it to an actual phone number.

It also presents use cases such as an agent that briefs overnight CI failures, a multilingual support line, and an interactive agent on a portfolio site that speaks like the owner. The fastest model achieves about 75ms latency, and a turn-taking model handles pauses and interruptions in real time, reducing the awkwardness of existing voice agents. Eleven v3 TTS can even express tone shifts, laughter, and sighs.

Setup proceeds in the following order:

• Install the ElevenLabs skill in Claude Code.
• Connect the API key using the setup-api-key skill.
• Restrict the key to agents-write only and set a daily spending cap.

After that, it creates an agent using Qwen-3.6 as the LLM, and indexes documents, URLs, and README.md as the knowledge base. RAG finds and inserts relevant documents, and automatically re-indexes when sources change to keep them in sync.

The workflow is divided into nodes such as greeting → intent classification → KB answer or handoff to a human agent → wrap-up. Tools are attached such as the client tool show_help_article, webhook tool get_weather, and built-in tools end_call, language_detection, each handling frontend UI manipulation, server API calls, and call termination respectively.

Finally, guardrails are placed in platform_settings, applied redundantly alongside the system prompt to create a defense layer that operates outside the LLM. After enabling focus and prompt injection protection, custom rules are added to block pricing claims, speculation about unreleased features, and write access to the billing system. Testing wraps up with a response test verifying greeting tone, a tool call test confirming the show_help_article call, and a simulation test that hands off pricing questions to sales.