Title: NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
Executive summary:
The Problem: Today's AI models generally learn in a straight line: they are trained, deployed, and then their knowledge freezes. To unlock the next frontier of AI capability - "Recursive Self-Improvement" (RSI) - a system needs a reliable mechanism to observe its own actions, evaluate its performance, and automatically convert that experience into its next round of training data. While this self-improving concept is highly sought after, building the actual engineering pipeline to close this feedback loop in the real world has remained incredibly complex and largely theoretical.
The Breakthrough: NeoHorse-1 provides a working prototype for an AI system that actively drives its own upgrades. It introduces a smart "routing" system that directs user requests to a pool of different models based on task difficulty. As it works, it records the entire interaction - including tool usage, logical reasoning, and how the task was routed. This data is rigorously filtered, evaluated, and then used by a "teacher" model to train smaller models in a three-stage curriculum. Crucially, NeoHorse-1 uses its own evaluation scores to dictate its next training mixture, creating a closed loop where what the system learns to do directly dictates what it studies next.
Why This Matters: This continuous, self-improving process yields massive performance gains without requiring a massive increase in model size or computing scale. Across 11 complex benchmarks testing coding, tool usage, and autonomous agent behavior, this post-training method elevated a highly efficient 4-billion-parameter (4B) model's performance to nearly match that of a much larger 9B base model. It proves that smarter, self-guided training loops can punch far above their weight class.
Business Impact: For enterprises and AI builders, this paper offers a practical blueprint for the ultimate "data flywheel." It points to a future where deploying an AI agent is just the starting line. Businesses can leverage these self-improvement architectures to build autonomous enterprise agents, customer service bots, and specialized copilots that organically adapt and improve from daily interactions. It offers a tangible path to deploying smaller, cheaper AI models that get increasingly capable over time, reducing reliance on expensive, static foundation models.
Generated by Gemini