Vals Raises $40 Million in a16z-Backed Funding to Challenge the Gold Standard of AI Evaluation

Overview

AI evaluation startup Vals recently announced the completion of a $40 million Series A funding round led by a16z (Andreessen Horowitz). The company is committed to becoming the "gold standard" in the field of AI benchmarking, aiming to address the pain point that existing evaluation systems lag behind technological development. Since its founding in 2024, Vals has risen rapidly in less than two years, having previously received seed funding from 8VC and Bloomberg Beta.

Reshaping AI Evaluation Standards

For a long time, benchmarking has been the primary means for AI companies to verify model capabilities and highlight competitive advantages. However, many legacy evaluation systems are outdated and cannot accurately measure the true capabilities of modern models, and some have even sparked controversies over "test cheating" due to the public release of questions.

Vals co-founder Rayan Krishnan pointed out that existing academic benchmarks often focus on abstract knowledge assessments, such as simulating bar exams, and fail to reflect the actual impact of models in real-world scenarios.

"We want to evaluate whether a model can produce work of equal quality to humans in specific domains, not just obtain the correct answer."

To this end, Vals adopts a dual strategy: first, it does not disclose specific test questions to prevent targeted model training; second, it delves into vertical industries such as law, finance, and programming to assess the actual ability of models to complete complex tasks. Additionally, the company focuses on the negative effects of models, analyzing potential risks they may pose when out of control.

Business Boom and Industry Layout

With the team expanding from 8 people at the beginning of the year to 25, Vals' business scale has seen explosive growth. It is reported that the company's current revenue is already 8 times that of the same period last year. Its business model is similar to students taking the SAT exam: enterprises pay Vals to conduct independent evaluations of models, using the results as a basis for technical troubleshooting and iterative optimization.

Currently, Vals' evaluation scope has extended to more challenging cutting-edge areas:

1. Recursive self-improvement: Evaluating model performance during the process of continuously optimizing itself.
2. High-risk domains: Covering mental health, cyberscurity, biosafety, and the law of armed conflict (such as the applicability of the Geneva Conventions).
3. Government services: Recently launched a model evaluation project targeting federal agencies.

Promoting Transparency in the AI Industry

Krishnan believes that as giants such as OpenAI and Anthropic prepare for their respective IPOs, high-quality third-party evaluations will become a key basis for AI companies to build public trust and submit public disclosure documents.

In an old brick building office on Folsom Street in San Francisco, Vals is attempting to redefine the "credential of capability" in the AI era. For investors, this is not just a fast-growing startup, but also a cornerstone enterprise with the potential to dominate future AI industry standards.