AI benchmarking and evaluation
Vals AI
Vals AI builds benchmarks that measure how well AI models, agents and products perform professional work. It was started to provide independent evaluation as models outgrew existing tests and benchmark data leaked into training sets. Its evaluations cover fields including finance, software engineering and law. Enterprises use Vals to choose models and compare their own AI products with frontier capabilities; model developers also cite its results in model cards.
Vals develops tasks with domain experts and separates public examples, licensed validation data and private test sets. Keeping the scored test data private helps protect results from training-data contamination, while public examples show what the evaluations ask systems to do. Its evaluation methodology distinguishes models tested through a controlled harness from agents and complete applications. Alongside correctness, it measures cost and completion time, making the resources needed to finish a task part of the comparison.
The Vals Index combines finance, coding, legal and tax benchmarks, weighting sectors by their share of U.S. GDP. Its tasks include building spreadsheet financial models, migrating code between languages and answering legal questions with citations. This connects model comparisons to concrete work rather than a single abstract capability score. For developers who need evaluations specific to their own software, Vals Smith, released publicly in August 2026, creates custom coding benchmarks from GitHub repositories.
1 talk
Newest firstLoading more talks and speakers…
1 speaker at AIE
Affiliations reflect their AIE event listings and recorded appearances, not necessarily current employment.
