Self-Improving Software (5 min read)

An automatic improvement platform where coding agents build, fix, and verify software has been introduced. The core idea is binding the agent development lifecycle together with five prompts: Create, Improve, Extend, Hill Climb, and Review.

The reason the platform improves itself is that the execution environment was designed from the start to fit this loop. Agent code, traces, sessions, logs, eval suites, and the live service are all gathered in one place, and every core action can be invoked via API or cURL/bash. Sessions and traces are stored in Postgres, and the platform runs on local Docker, keeping the test-review loop at around 5 seconds.

• Create: Claude Code searches the Agno docs via MCP to scaffold a new agent, registers it in app/main.py, then restarts the container to smoke-test it. It takes 5-10 minutes to go from prompt to agent.
• Improve: Generates 8-12 probes from the INSTRUCTIONS and runs them across the golden path, edge cases, tool selection, and even prompt injection. Rules are tightened or loosened to match failure patterns, agents/<slug>.py is fixed, and failed probes are re-run for up to 5 rounds.
• Extend: New tool additions, prompt edits, and bug fixes are applied in small increments. Each change is verified with a smoke test, and the Agno docs are referenced again to align the implementation.
• Hill Climb: Runs evals to diagnose failures, fixes issues within scope based on the cases, rubrics, and expected tool calls in evals/cases.py, then reruns the full suite to catch regressions.
• Review: Finds drift across docs, code, and config, aligns file registration, env vars, and doc paths, and fixes minor inconsistencies on the spot.

Improve catches out-of-distribution failures, while Hill Climb prevents regressions in in-distribution cases. Running both together keeps raising agent quality.

This structure fits especially well with agent platforms. Since actions, data, and iteration tools are all in one place, end-to-end testing and fixes are possible, and as a result the platform itself becomes the first target for improvement. agent-platform-railway is presented as a starter codebase that lets you run this loop directly on local Docker or on Railway.