Request for Startups: New Verticals for Reinforcement Learning

In 2025, RL techniques began to elevate the capabilities of AI agents to exciting new thresholds. Probably the biggest AI agent story last year – coding agents – is in part a story about RL. (Of course, fundamental model improvements, more compute, better evals, and other innovations also have contributed to the explosion of progress in the past year.)

RL creates an important new layer in the AI startup stack. Because if reliable agents depend on domain-specific environments, tasks, and reward signals, then the scarce asset is no longer just raw data. It is the training infrastructure: the “environments” that richly simulate how actual people in the real world do their job and interact with software, which in turn captures expert workflows in fields like medicine, law, and finance, and translates them into something an AI model can understand. 

Frontier labs like Anthropic and OpenAI will build some of these environments themselves, but the breadth of real-world domains creates room for specialized external suppliers.

The Opportunity for Startups

Software 1.0 easily automates what you can specify.
Software 2.0 easily automates what you can verify.
— Karpathy

AI data is one of the fastest-growing markets in history. Frontier labs and hyperscalers are expected to spend roughly $185 billion on AI opex and capex this year, with tens of billions allocated to data. A general rule of thumb is that for every $1 a lab spends on compute, 10 cents will be spent on data. The data labeling subset of this market has already birthed at least three decacorns in Scale AI, Mercor, and Surge AI. 

Within the broad AI data market, the RL post-training category has begun to dominate frontier research budgets. Frontier labs are investing hundreds of millions of dollars in a category that barely existed a year ago. The labs have a seemingly unlimited budget for high-quality RL to bridge the gap between general and domain-specific expertise and to enable AI tools to take action in the real world. Unsurprisingly, many startups have been founded over the past year to service the insatiable appetite for RL environments.

In recent months, Village Global has made investments in various domain-specific RL companies, each with unique training methods.

The broad lesson is that as AI systems move from generating language to taking action, the bottleneck shifts. The scarce input is no longer just model size or internet text. It is access to environments where domain expertise can be translated into feedback loops machines can learn from. 

The next act of the modern AI revolution is a massive opportunity for startups to build the environments that will affect how the next generation of AI models learn. 

As agentic models move into enterprise software, finance, analytics, operations, and even robotics,  there will be more and more demand from the model companies for domain-specific RL environments to support increasingly nuanced real-world actions, and dozens of extremely valuable startups will be founded to serve this opportunity.

If you’re building in this area, we’d love to hear from you.