AI Engineer Summit 2025

Your LLM Ran Out of Knowledge — Now What?

12:51

Loading slides…

Read the talk

Your LLM Ran Out of Knowledge—Now What?

When professional judgment never makes it into training data, explicit domain rules can guide a model’s reasoning, from classifying a problem to generating scenarios for expert review.

Before you start: Familiarity with LLM prompts and basic application workflows is sufficient; no geopolitical modeling background is required.

Reasoning still needs domain knowledge

How can a model solve a problem in a domain whose knowledge was never available for training? Moving from chatbots to agents to reasoning models makes this question more pressing: greater reasoning capability does not automatically supply missing professional experience.

Minecraft library scene with empty shelves, a Risk Management sign, cobwebs, and the heading Research (LLM PoV).Open full source frame
Empty shelves labeled “Risk Management” in the “Research (LLM PoV)” sequence.

Coding, mathematics, law and physics have substantial bodies of structured material that can become training data. Other fields leave much less of their decision-making process on paper. Consider a corporate acquisition: a newspaper reports that company A wants to acquire company B, and a later headline reports the outcome. Those reports expose the starting point and the result, but not what happened in the negotiating room. MBA guidelines may describe how negotiations should work; they do not recover the actual discussions.

That missing middle contains practical judgment: knowledge held in people’s minds and never written down. Its absence can leave some professions with increasingly capable models while others struggle to apply them to everyday problems. Speaking from risk and security management, the presenter locates the bottleneck in practitioners’ documentation habits. If experience never becomes a record, model trainers have little material from which to teach it.

0:010:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:01 · section reference included

Give the model a practitioner’s rules of thumb

An intelligent graduate entering a profession faces a similar problem. They can reason, but they have not accumulated the experience needed to recognize every situation. Giving them guidelines and rules of thumb helps them apply their intelligence in a particular domain while they build that experience.

The proposed technique applies that pattern to an LLM: supply domain heuristics for the model to use during reasoning. Rather than waiting for a comprehensive training corpus, ask practitioners to express useful judgment as rules the model can apply to a new problem. The intended benefit is to compensate for gaps in practical wisdom; supplying rules does not itself establish equivalence to an experienced expert.

3:163:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:16 · section reference included

Match the problem and standardize the scenario

The demonstration begins with a long intelligence-estimate prompt entered into a frontend. Before attempting the analysis, the application determines what kind of problem it has received and which rules apply. It also reformats the question into a standard representation. These are preparation steps: the simulator has not yet started reasoning about the scenario.

The backend uses a dedicated LLM prompt to match the user’s input against a list of available heuristics. The presenter describes this as similar to tool selection, but uses a more explicit matching prompt because tool use had not worked as well as hoped in this implementation. That is a local design choice, not evidence that prompt-based routing universally outperforms tool use.

The preparation flow has two distinct responsibilities:

  1. Match the domain. Identify which available rule family fits the user’s problem.
  2. Parse the scenario. Return a consistent structure containing the material the simulator needs.

The frontend lets the user edit the parsed scenario before proceeding. This gives the later reasoning stage a predictable input shape without making the parser’s interpretation irreversible.

4:284:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:28 · section reference included

Assemble explicit, checkable domain rules

Once a match exists, the application retrieves the appropriate rules and behavioral guidance. The demonstration contains two rule families: corporate negotiations and geopolitical strategy. Their directives use a concrete must-do or must-not-do form rather than broad advice. One negotiation rule prioritizes agreements with the highest combined value. One geopolitical rule calls for at least three independent paths for critical resources. These are supplied heuristics, not validated universal requirements.

Code editor showing negotiation rule lists and the beginning of geopolitical strategy rules, with a small presenter inset.Open full source frame
Negotiation rules organized into “must_do” and “must_not_do” lists.

The purpose of that specificity is to make application of a rule easier to inspect. A collection of relatively simple directives can express a more sophisticated approach to negotiations or strategy. A higher-level behavioral prompt then incorporates the selected domain rules. The rule sets remain separate so that the prompt is easier to manage and the domain guidance easier to edit.

A compact Python representation of this separation is:

python

import json

rule_sets = {
    "negotiations": {
        "must_do": [
            "Prioritize agreements that create the highest combined value."
        ]
    },
    "geopolitics": {
        "must_do": [
            "Maintain at least three independent paths for critical resources."
        ]
    },
}


def assemble_instructions(domain: str, behavior: str) -> str:
    selected_rules = rule_sets[domain]
    return f"{behavior}\n\nDomain rules:\n{json.dumps(selected_rules, indent=2)}"

The important boundary is between general behavior and selected domain guidance. Adjusting a rule for a particular situation changes the instructions supplied to the model without requiring the entire application prompt to be rewritten.

6:366:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:36 · section reference included

Send the prepared scenario to the reasoning model

Back in the frontend, the original question has been classified as geopolitics and reformatted for the reasoning or planning model. The displayed semiconductor scenario separates its goal, constraints, conditions and additional parameters. The application has now identified the problem and packaged the user’s material into a consistent input for analysis.

Orac Reasoning Engine showing Type: geopolitics, a semiconductor scenario goal, constraints, conditions, and additional parameters.Open full source frame
Orac labels the scenario as geopolitics and displays its structured summary.

The demonstrated analysis goes to an Anthropic model using the presenter’s adaptation of WorldSim, a prompting-based simulation approach whose inspiration he credits to Nous Research and Karan. The exact Anthropic model is not specified. Here, the additional simulation instructions aim to give the geopolitical analysis a more realistic context.

That context includes physical geography and institutions such as the UN and WTO where relevant. These become additional parameters and constraints for the model to consider; they do not turn generated scenarios into a validated forecast. WorldSim is useful to this particular example, but the preparation pipeline does not depend on one model provider. The prepared scenario and rules can be sent to whichever reasoning model the practitioner chooses.

8:118:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:11 · section reference included

Inspect how the output applies the rules

The generated response first follows the requested presentation structure. More substantively, the output references two-times leverage and greater-than-50-percent control, both drawn from the supplied geopolitical rules. These numbers are rule criteria, not measurements of model performance. Their appearance shows that the response is incorporating the domain instructions rather than merely adopting the requested formatting.

The clearest example is scenario five: the output eliminates it because it would breach a rule. The rules therefore affect which proposed options survive, not just how those options are described. The scenario concerns US semiconductor development, but the presenter explicitly sets aside whether the resulting strategy is the best policy approach. Visible rule application is the demonstrated result; strategic optimality is not.

9:5110:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:51 · section reference included

Keep experts involved while exploring more options

The approach remains under development, with initial results the presenter describes as encouraging. In his implementation, the larger matching prompt provides more granularity than the tool-use approach he tried. Subject matter experts then supply tight rules and heuristics for the reasoning stage to apply. This makes expert participation part of constructing the system, before anyone reviews a generated answer.

Reasoning tools and simulation prompts can then explore alternatives, aiming to identify a preferred solution or return multiple scenarios. The presenter still wants a human subject matter expert in the discussion. He anticipates that experts could generate dozens of scenarios for review in a fraction of their usual time, but supplies no timed benchmark. The proposed benefit is a larger set of options for expert judgment, not the removal of that judgment.

Broader exploration could help with harder problems, decisions made under time pressure, or approaches that a practitioner might not otherwise consider. The reusable combination is a parsing engine, explicit domain heuristics and LLM problem solving. Where practical knowledge is scarce in the training record, this offers a way to bring some of that knowledge into the reasoning process—and extend model usefulness into professional domains that remain underexplored.

10:5311:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:53 · section reference included

Resources

Transcript download
Full transcript

Loading transcript…