← All speakers

Bio, Work & Ideas

Dr Bryan Bischof

Conference affiliation: Hex · 2024

Dr. Bryan Bischof is Head of AI at Theory Ventures and previously led development of Hex Magic, an AI assistant for working with data. His work applies hard-won lessons from recommendation systems, analytics, and product engineering to a central challenge of enterprise AI: building assistants that people can actually trust.

Bischof earned a doctorate in pure mathematics at Kansas State University, studying representation theory and noncommutative algebraic geometry. As Blue Bottle Coffee’s first data hire, he built its data function across warehousing, demand forecasting, recommendations, and analytics. He subsequently worked on recommendation and data-science problems at Stitch Fix, then led data science at Weights & Biases, building teams encompassing machine learning and data engineering.

With Hector Yee, he coauthored Building Recommendation Systems in Python and JAX. He also teaches graduate-level data science and analytics at Rutgers.

At Hex, Bischof led the team building AI capabilities into analytics workflows, helping users write SQL and Python, explore data, and work inside a notebook environment. His move to Theory Ventures broadened that focus to enterprise agents, AI infrastructure, and the organizational conditions that determine whether emerging products become useful.

  • Retrieval as a recommendation problem. Bischof treats retrieval-augmented generation as a familiar recommendation-system challenge: identify relevant resources, rank them effectively, enforce practical constraints, and assemble useful context. His account of building Hex Magic shows why better retrieval and product architecture can matter more than retraining a model.
  • Failure-funnel evaluation. At Hex, Bischof and colleagues developed an evaluation framework that separates SQL-agent failures into stages: retrieving the necessary tables, selecting valid tables and columns, executing a query, and producing the correct result. He has credited his collaborators, while Hex’s engineering breakdown illustrates how aggregate scores conceal the precise failure that needs repair.
  • Trustworthy data agents. At Theory Ventures, Bischof helped create America’s Next Top Modeler, an agent competition grounded in a fictional retailer’s messy business data. His analysis of its results distinguishes coding-agent success on benchmark questions from the contextual judgment and curiosity required of a dependable analytical partner.
  • Product-minded AI teams. Bischof hires for data intuition, practical judgment, and domain expertise, using realistic take-home exercises instead of algorithm puzzles. He advocates involving designers and data specialists early, developing assistants alongside actual practitioners, centralizing shared AI infrastructure, and adding specialized machine-learning expertise when product maturity warrants it.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

Build an AI team around the work the product needs next: production engineering, data judgment, product discovery, and sustained access to domain experts.

  • What does an AI engineer actually do?
    0:00 ↗
  • The first hire must get capabilities into production
    1:54 ↗
  • Match the team to the product’s stage
    4:04 ↗
  • Hiring must follow the learning sequence
    5:58 ↗
  • Give every role a hiring thesis
    8:25 ↗
  • Look for judgment beyond completing the implementation
    11:10 ↗
  • Make the interview resemble the work
    12:52 ↗
  • Build with the people whose work the AI supports
    15:54 ↗
  • Inside the take-home and feedback interview
    18:20 ↗
  • Where security belongs
    22:11 ↗
  • Help existing teams through shared AI infrastructure
    24:35 ↗
  • What can you develop after hiring?
    27:48 ↗

Key ideas

Scroll to read ↓

Useful LLM products depend on domain expertise, deliberate hiring, and a feedback loop that turns real user interactions into evaluations, better decisions, and production guardrails.

  • Six practitioners encounter the same problems
    0:24 ↗
  • Build a product advantage that survives better models
    3:23 ↗
  • Evaluation belongs inside an improvement loop
    6:06 ↗
  • Get real interactions into the loop
    10:01 ↗
  • Prototype for economics that may become possible
    11:57 ↗
  • Buying another shovel does not tell you where to dig
    14:52 ↗
  • Premature model ownership creates the wrong work
    16:23 ↗
  • Evaluation is a core engineering skill
    18:24 ↗
  • Hire for the next stage of the product
    21:00 ↗
  • Turn broad quality goals into testable criteria
    23:54 ↗
  • Choose an evaluator you can align and maintain
    25:41 ↗
  • Inspect recognizable failures with their execution context
    28:34 ↗
  • Turn reference-free evaluations into guardrails
    30:43 ↗
  • The surrounding system still needs maintenance
    32:32 ↗
  • A compelling demo can precede a product by decades
    34:17 ↗

References