Cover | Graphic by Pencil News

Another AI business is on fire.

On October 8, AI evaluation platform Arena announced the completion of a $200 million (approximately 1.34 billion yuan) Series B funding round, co-led by Lightspeed Venture Partners and Khosla Ventures, reaching a valuation of $3.1 billion (approximately 20.7 billion yuan).

In January of this year, Arena's valuation was only $1.7 billion. In just nine months, its value has surged by 80%.

Even more astonishing is its revenue growth. Arena disclosed in June that its commercial services, launched just eight months earlier, had already exceeded $100 million in annualized revenue (approximately 670 million yuan).

The company's business is unique: it creates, tests, and grades AI models like ChatGPT and Claude.

Similar stories are unfolding. In September, Stanford-affiliated startup Snorkel AI raised $350 million in funding, reaching a valuation of $3.5 billion. Domestically, reports emerged that Alibaba is considering leading a $300 million funding round for AI evaluation company UniPat, with a valuation of $2.5 billion, potentially involving Tencent and Sequoia China.

AI evaluation, once confined to university labs, has suddenly become a hot business in the capital market.

- 01 - A $3.1 Billion Unicorn from Berkeley Lab

Arena's story begins at the University of California, Berkeley.

In 2023, Berkeley PhDs Weilin Jiang and Anastasios Angelopoulos launched Chatbot Arena to address a question: Which model—ChatGPT, Claude, or Gemini—is smarter?

At the time, the large model industry was obsessed with showcasing performance. Whenever a new model was released, vendors would present test results in math, programming, and knowledge quizzes to prove their superiority.

But these exams had a problem: high scores didn't necessarily mean users found the models useful.

So, the Berkeley team came up with an idea: let ordinary people be the examiners.

On the Chatbot Arena website, users input a question, and the system randomly assigns two anonymous models to answer. Users don't know which answer comes from OpenAI or Anthropic—they simply choose the better one.

As votes accumulated, the platform gradually formed an AI model leaderboard.

This approach quickly gained popularity in Silicon Valley. Whenever a large model company released a new product, it aimed not only to showcase its own test scores but also to rank high on Arena's leaderboard.

What started as a research project gradually became a vital evaluation platform for the AI industry.

In 2025, the team founded Arena Intelligence as a commercial entity and began accepting venture capital. By January 2026, Arena had completed a $150 million Series A funding round, reaching a valuation of $1.7 billion.

Just nine months later, Lightspeed Venture Partners and Khosla Ventures co-led a $200 million Series B funding round, with participation from Salesforce Ventures, Dell's investment arm, and a16z.

What did capital see in Arena? The answer lies in the user data it has accumulated.

As of June this year, Arena had over 10 million monthly active users, generating more than 700 million conversations and 82 million human votes.

These users not only helped Arena create test questions for free but also graded the answers.

For example, when asking AI to write a business plan, why do users prefer Claude over GPT? Is it due to clear structure, accurate information, or content that better meets practical needs?

For AI vendors, this information is invaluable.

In the past, large model companies relied on internal engineers to design tests. Now, Arena has obtained millions of real-world questions and user judgments on different answers.

More importantly, this feedback is continuous.

In September 2025, Arena officially launched its paid product, AI Evaluations. The public leaderboard remained free, but model companies could purchase detailed evaluation services to understand why they lost, which tasks they performed poorly on, and how to improve.

The free leaderboard attracted users, while the evaluation data and analysis services behind it generated revenue.

This model quickly proved successful.

In January, Arena's annualized revenue was around $30 million; by June, it had exceeded $100 million. However, this annualized revenue was projected based on the current business scale, not actual full-year revenue.

Even so, it took only eight months for Arena to surpass $100 million in annualized revenue after launching its paid services.

Today, Arena is no longer satisfied with just ranking large models.

On October 8, alongside its new funding round, it launched the Alignment Index to detect whether AI agents perform unauthorized operations, misinterpret user intent, or falsely claim completion of unfinished tasks.

In a study covering 27 models and approximately 90,000 real agent conversations, Arena found that even advanced AI could delete files without authorization or claim success without completing checks.

From judging whether AI can chat to assessing whether it can work reliably, Arena is expanding its business scope.

- 02 - VCs Are Scrambling for 'AI Examiners'

Arena is not alone.

On September 22, another Silicon Valley AI company, Snorkel AI, announced the completion of a $350 million Series E funding round, reaching a valuation of $3.5 billion, nearly tripling from $1.3 billion in May last year.

This round was co-led by Insight Partners and S32, with participation from renowned firms like Greylock, Lightspeed, and GV.

Snorkel also originated from Stanford's research ecosystem. Founded in 2019, it initially helped enterprises automate training data processing.

But it soon discovered that AI companies' needs had changed.

In the past, model training required large volumes of text, images, and manually annotated information. These tasks could be outsourced.

Now, as AI becomes smarter, simple data is no longer sufficient.

For example, if an AI company wants its model to learn to analyze mergers and acquisitions, it cannot just feed it thousands of contracts—it also needs professionals like lawyers and financial analysts to design complex cases and judge whether the model identifies key risks.

Snorkel began expanding its business from selling data processing software to providing high-quality training data, professional tasks, and evaluation environments.

The result was explosive revenue growth.

According to the company, after launching its new data services in September 2025, its business grew rapidly. By September this year, its annualized revenue had exceeded $375 million, up more than 17-fold from a year earlier.

A company providing AI with data and exam materials has already achieved annualized revenue in the hundreds of millions of dollars.

Chinese capital is also paying attention to this sector.

In late September, 36Kr reported that Alibaba is considering investing in UniPat, potentially leading a $300 million funding round with a post-money valuation of about $2.5 billion. Tencent and Sequoia China are also interested in participating.

Founded in late 2025, UniPat focuses on AI model training, evaluation, and real-world task data.

Its services can be understood as helping AI companies build an exam system.

When a large model company develops an agent, it traditionally needs its own testing team to design tasks and engineers to check whether the AI completes the work. UniPat attempts to productize this process.

Zhu Jun (pseudonym), an AI agent project leader at a major tech company, told Pencil News that in the past, developing an agent required the testing team to design tasks and judge completion levels. UniPat has packaged part of this work into a product.

There's an economic rationale behind this.

Suppose a company plans to launch an AI programming tool. Building an in-house evaluation team would require hiring engineers, developing test environments, designing questions, and verifying answers.

If a third party can do this faster and cheaper, the company has a reason to pay.

Moreover, this business has another characteristic: as AI models continuously upgrade, exams must be retaken.

Today, GPT releases a new version; next month, Claude launches a new model; a few weeks later, another new product emerges. Each upgrade requires vendors to test new capabilities and check for regressions in old functions.

Besides Arena, Snorkel, and UniPat, more companies are already earning money from related businesses.

A notable example is Mercor.

Founded in 2023, this U.S. startup specializes in helping AI companies recruit professionals like programmers, doctors, lawyers, and financial experts to design tasks, review answers, and provide training feedback for large models.

According to Sacra, Mercor's annualized total revenue reached approximately $2 billion in June this year.

Another company, Surge AI, has already proven that this industry can be profitable.

British media previously reported that Surge AI generated over $1 billion in revenue in 2024 and was profitable. It primarily provides high-quality training data and human feedback, not just model evaluation.

These companies have different businesses but share one commonality: instead of investing heavily in training general-purpose large models like OpenAI, they help large models become better and more reliable.

Some design exam questions, some grade answers, some recruit experts, and some turn errors into new training data.

The more complex AI becomes, the more professional services it requires.

This resembles the gold rush. Some mine for gold; others sell shovels. Today, as AI model companies compete to be first, a group of companies providing evaluation, training, and data services have already found ways to make money.

- 03 - Who Pays for AI Examiners?

As companies grading AI become more valuable, Zhang Yizhe, founding partner of Yonggui Fund, raises a critical question: Who pays for them?

He believes AI's development resembles that of the internet and mobile internet, with each part of the supply chain having its value. Evaluation could find its commercial niche, but the size of the business depends on customer demand.

So far, payers fall into two main categories.

The first is large model companies.

In recent years, OpenAI, Anthropic, Google, and others have invested heavily in training models. Whenever they release a new product, they want to prove it outperforms competitors.

But by 2026, merely demonstrating strong math or programming skills is no longer enough. Especially with the rise of agents, AI now performs tasks for humans rather than just answering questions.

Suppose a financial company asks AI to analyze five public company financial reports.

The AI might generate a polished report in minutes, with complete numbers and fluent language. But if it misses a major liability or mixes financial metrics from different companies, the report becomes unusable. Worse, users may not know where it went wrong.

Similarly, an AI programming tool might generate tens of thousands of lines of code daily but introduce new bugs when modifying a feature.

High model exam scores don't mean it can truly work for humans.

In 2025, OpenAI launched GDPval, covering nine industries and 44 professions, with 1,320 professional tasks. Exam designers and reviewers averaged over 14 years of experience.

Such evaluations now require AI to deliver real work results, not just answer multiple-choice questions.

This means exam costs have changed.

In the past, a math question needed only the correct answer; now, a complex contract or financial analysis requires experts with years of experience to judge.

Designing these tests costs money, judging results costs money, and improving models after errors costs money.

The second category of payers is enterprises planning to adopt AI.

This market may be even more noteworthy.

A bank using an AI customer service agent needs to know if it will leak customer information; a software company purchasing an AI programming tool needs to know how many real tasks it can complete; a manufacturing company introducing intelligent agents must first ensure they won't perform dangerous operations unauthorized.

Enterprises truly care about success rates, stability, and cost.

In a September study, Arena tested 21 model-and-programming-tool combinations. For some tasks, the same model achieved similar success rates but had cost differences of up to fivefold.

This means enterprises choosing the wrong model or tool could pay several times more for the same work.

If evaluation companies can help clients save this money, they have clear commercial value.

However, challenges loom amid the capital frenzy.

The first is credibility.

In 2025, a paper titled 'The Leaderboard Illusion' questioned Arena's leaderboard mechanism.

Researchers found that before Llama 4's official release, Meta had tested up to 27 unreleased model versions. They argued that if vendors can repeatedly test and choose the best version to disclose, the leaderboard may suffer from selection bias.

Arena disputed some claims in the paper and responded by clarifying rules for multi-version testing, labeling models that exit testing, and adding provisional tags to some scores.

This exposes a subtle commercial conflict.

Arena both grades model companies and charges them fees.

When examiners and examinees have a commercial relationship, concerns about fair testing arise.

Then there's the issue of client data.

In 2025, after Meta invested in Scale AI, Google, OpenAI, and other clients reportedly planned to reduce or stop cooperation, fearing their data and research directions might be exposed to competitors.

The same risk exists in the AI evaluation industry.

A company entrusting a third party to test its unreleased new model is essentially handing over its product capabilities, technical weaknesses, and research priorities to an external agency.

Who can access this information and how to ensure confidentiality will become key conditions for evaluation service providers to win major clients.

Finally, there's profitability.

An evaluation company that must recruit experts, design questions, and build environments for each new client will see labor costs rise rapidly alongside revenue growth.

Conversely, if it can standardize test tasks, expert experience, and automation tools into a product serving hundreds or thousands of enterprises, it has a better chance of achieving software-company profit margins.

Thus, while all may seem to be in the AI evaluation business, different companies' models can vary widely.

Some resemble software companies, others data service providers, and still others high-end talent outsourcing platforms.

For investors, valuation should not only focus on revenue growth rate, but also on how much profit these revenues can ultimately retain.

AI examiners have secured huge financing. Next, it's their turn to take the market's test.

This article does not constitute any investment advice.