← All speakers

Bio, Work & Ideas

Jeremy Silva

Conference affiliation: Freeplay · 2025

Jeremy Silva is an AI engineer at NVIDIA who previously led product at Freeplay, where he developed tools for agent evaluation, human review, and production monitoring. His work addresses a central challenge of enterprise AI: turning convincing prototypes into reliable products that improve through real-world feedback.

Silva began in data science, building natural-language-processing models for healthcare before GPT-3, then moved into machine learning engineering and infrastructure for scaling models. At Freeplay, he progressed from AI engineering in 2024 to product leadership by the 2025 AI Engineer World’s Fair. His professional profile connects that trajectory to NVIDIA.

His Freeplay work evolved from combining human review with model-graded evaluations into observability for complete agent workflows and automated analysis of production behavior. By 2026, he was describing an agent data flywheel that connects logs, reviewer feedback, evaluations, and experiments into continuous product improvement.

  • Evaluate the actual product. Silva favors product-specific evaluation criteria grounded in customer needs, representative examples, and recognizable failure modes. His guide to trustworthy evaluations treats prompts, models, cost, latency, and human judgment as parts of one testing process.
  • Turn human review into operational intelligence. Human expertise supplies context and domain knowledge that automated scoring misses. Silva’s work on Review Insights turns reviewer notes and labels into patterns that can inform new evaluations, datasets, and prompt experiments.
  • Make AI quality someone’s responsibility. He identifies an emerging AI quality lead who combines customer understanding with systems thinking to define evaluation criteria, investigate failures, label examples, and run experiments. The role can draw from product, operations, engineering, or domain expertise without requiring ownership of production code.
  • Build AI into the product strategy. Silva argues against isolated AI initiatives and features designed chiefly to demonstrate technical capability. His approach to integrated AI products starts with customer problems and develops incrementally from embedded assistance toward contextual, interconnected agent workflows.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

AI becomes useful when it improves the product customers already need. A support inbox and Workday’s employee services show how embedded capabilities can grow into dynamic products.

  • What does AI positioning actually differentiate?
    0:29 ↗
  • From testing LLMs to delegating work
    2:08 ↗
  • How a separate AI strategy produces separate features
    3:51 ↗
  • Make reliability part of product planning
    5:41 ↗
  • Crawl, walk, run in a shared support inbox
    7:27 ↗
  • Workday Help: build the content foundation
    10:18 ↗
  • From content authoring to contextual self-service
    13:31 ↗
  • Proactive assistance across the platform
    14:50 ↗
  • Beyond yesterday’s roadmap
    16:08 ↗

Key ideas

Scroll to read ↓

A promising AI product needs more than a launch: evaluations, human feedback, and clear quality ownership turn rapid development into reliable improvement.

  • What happens after the first version ships?
    0:00 ↗
  • Faster development increases operational demands
    1:12 ↗
  • Crossing the quality chasm
    2:09 ↗
  • Confident errors need human judgment
    3:34 ↗
  • Feedback needs reviewers
    5:09 ↗
  • Expanding QA beyond retrospective audits
    6:52 ↗
  • Give AI quality an owner
    9:09 ↗
  • Place review where it matters, then keep operating
    11:13 ↗

References