Specifying behavior in a model call
What prompt engineering changes
On Nguyen’s slide, the shopper asks for a long dress in the style of The Great Gatsby. The candidate is a Maxi Godet Denim Skirt In Light Blue Wash. The length might fit, but the shopper asked for a dress and the item is a skirt. Claude’s reply on the last line, <answer> NO </answer>, passes two separate checks. The tags contain one of the two allowed labels, and NO fits this particular garment mismatch. Testing the recommendation rule requires checking its decisions on other items as well.

The prompt lists the allowed answers, but it gives Claude no example of when to choose either one. A demonstration pairs an earlier input with its desired answer. In their GPT-3 study, Brown and colleagues at OpenAI called a request with no demonstrations zero-shot, one with a single demonstration one-shot, and one with several few-shot. Nguyen’s request is zero-shot. The NO visible on the slide is the answer Claude generated. Language Models are Few-Shot Learners
| Approach | What the author supplies | What reaches the next call | How to check the effect |
|---|---|---|---|
| Nguyen’s handwritten instruction | A rule for judging relevance and an answer format | The rule, the query and the candidate item | Check format and relevance separately on new items. |
| GPT-3 in-context demonstrations | Inputs paired with desired completions | The pairs followed by a new, unanswered input | Compare predictions with an instruction-only baseline on the same cases. |
| DSPy prompt compilation | Task fields, examples and a scoring rule | Instructions or demonstrations chosen using development and validation results | Freeze the selected program and assess it on untouched cases. |
These choices combine. A DSPy compiler can select examples for a few-shot prompt that also contains a handwritten rule. In each case the intervention changes the input the model uses to predict its response. Deciding which documents, conversation history and tools also belong in each call is the broader work of context engineering.
Specify the task
An instruction tells the model what operation to perform. In Nguyen’s example, the query expresses a requirement and the item description supplies evidence about one candidate. Calling the operation recommendation still leaves decisions to the author. If a shopper mentions summer, should a winter coat qualify because it matches the requested color? Nguyen makes season and weather relevant when the shopper specifies them. A reviewer can apply that criterion to other garments. Calling the model a fashion expert leaves the priority between color and season open.
Missing evidence requires a decision too. If fabric matters and the description omits it, one catalog might exclude the item and another might send it for review. For the second catalog, the instruction could allow UNKNOWN when a required attribute cannot be determined. That would extend Nguyen’s binary rule and change what the caller must handle. Try giving the proposed instruction to two reviewers. A disagreement about which answer is allowed identifies a requirement worth settling before experimenting with phrasing.
A role can be useful when voice is part of the requirement. For Claude.ai’s conversation titles, Nguyen asked Claude to act as an editor and supplied previous titles so it could see the style to follow. Where an interface supports a system message, the application can use that channel for instructions. Its message boundaries depend on the model interface. Placement within a message is a separate variable. On long-document questions for Claude and Claude Instant, Nguyen reported better results when the question followed the document. Moving the same question around the same document is a way to test whether that result applies elsewhere. Special tokens and chat templates
Define acceptable responses
The caller needs to know where to find the result, what values it can take and what each value means. For the skirt response, a parser can locate the answer tags. A second check can verify that the enclosed text, after trimming spaces, is an allowed label. Now imagine replacing NO with YES. Both checks would still pass. Judging the value requires comparing the offered garment with the requested one. The same distinction applies to JSON. Constrained generation can restrict the model to continuations permitted by a schema, but choosing the right permitted value requires task evidence. Structured Outputs and Tool Calling
Input delimiters serve a different purpose. A tag such as <item> identifies material the model should inspect. A command appearing inside a product description can still influence the response. If an application lets a model request actions, the application must independently decide whether the action and its arguments are authorized.
Response requirements also affect what can fit. A token is a model-specific unit used to represent text and may be shorter than a word. The context window limits the sequence available to a call. Count the rendered instructions, message markers, examples and task data together. Where input and output share capacity, leave room for generation and respect any separate input and output limits. Counting the complete request and reserving room for generation
Asking for intermediate steps changes the requested output. Chain-of-thought prompting can also show those steps in worked demonstrations, which adds to the input as well. Assess the final answer independently and check any stated steps on their merits. In experiments with answer hints and biased answer positions, models sometimes justified a changed answer without mentioning the cue that influenced it. What a reasoning trace establishes
Learn from demonstrations
In-context learning occurs when a model uses information in the current request to adapt how it performs a task while its weights stay fixed. Examples can tell it which task to perform, how to format the answer, or which label goes with which input. To test whether the model uses a particular label mapping, keep the task and the new input fixed and change only the mapping in the examples. A fresh call needs the mapping supplied again. An explicit instruction can state it directly. To repeat the demonstration condition, supply the examples again. Fitting model parameters can carry the association into later calls through the weights.
| Same review text | Mapping A | Mapping B |
|---|---|---|
| Demonstration · I enjoyed every minute. | Foo | Bar |
| Demonstration · I hated it. | Bar | Foo |
| New input · What a delight. | Unanswered | Unanswered |
| Expected label for the new input | Foo | Bar |
Recognizing delight as positive helps in both columns. Only the demonstrations say which word to emit. If predictions track the reversal on fresh reviews, that is evidence the model used the supplied association. Reversing the familiar words positive and negative makes the test harder to interpret because their prior meanings compete with the examples. Models differed on these tests. Unrelated and reversed label experiments
A complementary experiment randomized the labels attached to examples. On several classification and multiple-choice tasks, much of the gain over zero-shot prompting survived. The model still saw representative inputs, the available labels and how an answer should look. A correct prediction can therefore have several explanations. For an application, verify the labels and test the intended association directly. Demonstration ablations
How prompting developed
From prediction interfaces to instructions
An early version of the scientific question used numbers in place of words. In 2001, Hochreiter, Younger and Conwell trained a recurrent network on families of functions. Given a sequence of inputs and previous target values, its internal state could carry information about the particular function it was currently predicting. This separated two timescales of learning. Training built the capacity to adapt, and examples in a sequence let the network use that capacity. Learning to Learn Using Gradient Descent
Tasks expressed as predictions
For language models, a practical question was how to express useful work as a prediction the model already knew how to make. In 2019, GPT-2 used a summary cue after an article and an unfinished sentence pair for translation. LAMA probed factual associations with cloze statements, sentences with a missing answer. Completing a birthplace statement let researchers test a specific fact using the model’s existing vocabulary. GPT-2 report and LAMA study
Text could specify the task during training too. T5, described in 2019, prefixed inputs with task identifiers so that translation, classification and other tasks could share one text-to-text interface. Pattern-Exploiting Training (PET), proposed in 2020, used blanks for classification. A verbalizer mapped predicted words to classes. PET fine-tuned models on patterns and used their predictions to label additional data. Its few-shot setting counted scarce labeled training cases. The GPT-3 experiments that year varied the number of examples in the input to a fixed model. Both used text to express a task, but reproducing the former required trained state and reproducing the latter required the examples in the request.
Training for direct requests
Following a direct instruction became a training objective in its own right. In 2021, FLAN fine-tuned a model on tasks expressed as instructions and evaluated it on task types withheld from that training. BigScience’s T0 varied the natural-language prompts used to train on supervised tasks. More formulations per dataset improved median held-out performance and reduced variation across prompts in its experiments. An engineer using the resulting model could benefit from that training and still write a task-specific instruction for each call. Instruction tuning
Searching for effective inputs
Choosing the input itself posed a search problem. In 2020, AutoPrompt used gradients from a model to propose replacement tokens for a template and tested the resulting strings. Soft-prompt tuning in 2021 instead trained continuous numerical values to prepend to an input. In 2022, Automatic Prompt Engineer used a model to propose instructions from examples. In 2023, Omar Khattab and colleagues introduced DSPy compilation for programs containing several model calls. These methods change who proposes an input and what form it takes. The developer still has to settle questions like Nguyen’s garment rule before a search can reward the behavior the application needs.
Choosing and testing demonstrations
Choose informative examples
Choose a demonstration by asking what the model could learn from it that the other examples leave undecided. Suppose positive reviews in a prompt all end in exclamation marks and negative reviews all end in periods. Both sentiment and punctuation predict every label. A positive review ending in a period and a negative review ending in an exclamation mark show both exceptions to the punctuation rule. Check that the sentiment labels are unambiguous before spending space on the pair. A mislabeled boundary case teaches a different distinction. Data Quality and Curation
A fixed set accompanies every request. A query-dependent set is selected for the current input, for example by finding similar cases in a library. Figure 1 of the coverage-based selection study illustrates a request to schedule a meeting with someone’s manager. Inspect the operation each example contributes. A second appointment paraphrase repeats scheduling. A manager-lookup example supplies the missing operation that identifies the attendee. Selecting for coverage helped particularly on tasks that required combining operations into a structured query. Meeting example, Figure 1 · Search and Retrieval
Similarity itself can help. In the KATE study, GPT-3 classified IMDB movie reviews with three demonstrations drawn from the SST-2 sentiment dataset per request. At temperature zero, random selection averaged 87.95% accuracy over five runs. Selecting similar reviews with an unchanged RoBERTa-large text encoder achieved 91.99%. The selection procedure also determined the order. Three reviews in either condition could occupy different numbers of tokens, so the result answers a question about fixed example count rather than equal input cost. KATE study, Table 3
Increasing count is a separate intervention. With the February 2024 Gemini 1.5 Pro, news-summary overlap with reference answers on XSum improved through roughly 50 examples, then declined. With the same source of demonstrations and a different target collection, XLSum, adding examples generally helped. Some summaries fabricated dates or times. Choose a set using the quality measure the application needs, its rendered length and its coverage. Inspect repeated labels and answer positions for accidental cues, and exclude assessment answers from the example pool. Many-Shot In-Context Learning
Test order and formatting
Prompt sensitivity is a change in behavior associated with a change to the input. Moving an intact demonstration changes its position. Swapping answer options changes where a choice appears inside a case. Changing separators changes how the same examples are presented. To measure any one of these, keep the others fixed and score the variants on the same cases.
| Comparison | Intervention and controls | What can be learned |
|---|---|---|
| Demonstration order | Researchers tried all 24 arrangements of four fixed, balanced SST-2 examples. | Accuracy varied. Rankings of the orders agreed weakly across the GPT-2 and GPT-3 models tested. |
| Prompt formatting | FormatSpread varied separators, capitalization and option numbering with demonstrations, order and evaluation cases fixed. | Ten sampled formats produced a median spread of 7.5 accuracy points across tested settings. The principal measure ranked answer options by model probability. |
| Execution variation | Run the identical request afresh several times with the same model and generation settings. | Differences estimate within-prompt variation. Replaying a cached answer supplies no new execution. |
If the product allows several equivalent phrasings, examine performance across those phrasings as well as the best score. A prompt that wins once and changes its answer on an innocuous reformat has a different deployment profile from one that works across the variants users actually send. Punctuation can change token boundaries. A difference between two formatted requests alone cannot isolate segmentation as its cause.
Protect independent assessment
A baseline is the prompt against which a change is judged. A metric specifies how to score the resulting behavior. For a relevance task, start with a clear instruction alone and compare it with the same instruction plus demonstrations. Give both versions the same new items and queries. Record whether the label can be extracted and whether it agrees with a reviewed decision. Pairing the results item by item reveals both newly correct decisions and correct decisions lost. Cases and trials · Paired comparisons
Separate the cases by the influence they are allowed to have. In Annabell Schäfer’s Langfuse classification experiment, a prompt assigned labels to research titles and abstracts. On fit cases, the optimizer could inspect mistakes and suggest an edit for a recurring confusion. On validation cases, the score determined whether to keep the edit. The test cases were withheld until search stopped. In the figure, follow the validation score to the frozen prompt. Seeing only a score still lets a developer or optimizer adapt to the cases behind it. Schäfer’s classification experiment
Which cases can influence the prompt?
A validation score has a route into the choice even when the optimizer cannot read the validation labels.
Read the diagram as text
- Fit groups. The author can inspect expected and actual answers.
- Candidate prompts. Include the original baseline.
- Validation groups.
- Compare and select. Use the same validation tasks for the candidates.
- Frozen prompt. Selected wording and ordered demonstrations.
- Untouched test groups. Withheld throughout proposal and selection.
- Final assessment.
- Fit groups → Candidate prompts: Errors inform edits.
- Candidate prompts → Compare and select: Prompts to score.
- Validation groups → Compare and select: Cases and reference answers.
- Compare and select → Frozen prompt: Scores determine choice.
- Frozen prompt → Final assessment: Selected prompt.
- Untouched test groups → Final assessment: Fresh tasks and references.
Split the data according to what will be new when the prompt is used. If one task was run several times, holding out a single run still lets the author inspect other attempts at the same task. To assess performance on new tasks, keep all runs of each task together before choosing demonstrations or drafting edits. If you suspect that repeated tasks inflated an earlier result, keep the validation cases fixed and compare prompts developed with and without related fit cases. Match the size and difficulty of the fit sets, the search budget and the scoring rule. Reserve whole unseen tasks for the final test. Assessment independence
Preserve the rendered prompt versions, the order of examples, model version, available facts, generation settings, answer extraction and per-case scores. Count labels consumed during development separately from demonstrations sent per call. Choosing a three-shot prompt can use hundreds of reviewed cases. If generation varies, repeat the matched runs. Repeats measure variability on the observed tasks, while independent task groups support conclusions about the next task. Sampling units and uncertainty
Improving reusable prompts
Revise through testable hypotheses
Read the actual request, reply and reference answer together before editing. A missing item description calls for an input repair. A mislabeled reference calls for review. If the facts and reference agree but the rule allows two interpretations, state which cases clearer wording should fix. In the Anthropic Prompt Doctor workshop, Jamie Neuwirth and Zack Witten collect cases in the evaluation interface as they revise a prompt. Keeping those inputs available makes it possible to see whether a revision changes the intended cases and whether it breaks others.
| Instruction | Without demonstrations | With the same contrasting pairs |
|---|---|---|
| Judge relevance. | Baseline | Can the examples convey the priority? |
| Judge relevance. Require the specified garment type. | Can the explicit rule settle the cases? | Do examples still help once the rule is explicit? |
Removing the examples and rerunning the task is an ablation. Read across the first row to see whether the examples help convey the garment rule. Read across the second to see whether they still help when the instruction states the rule explicitly. If two promising prompts differ in both wording and examples, test the two remaining combinations with the examples in the same positions. Score each combination on overall relevance and on any mandatory cases, since a gain on common items can hide failures on rare ones. A rule that resolves a shared ambiguity is easier to test than a growing list of exceptions that may conflict. Before opening the test set, freeze the chosen wording, ordered examples and settings, and save the results that justified the choice.
Search prompts automatically
Automatic prompt optimization searches for an instruction, an ordered set of demonstrations or a combination to reuse on future requests. Specify what the search may change and which model will execute the resulting prompts. In one search round, run the current instruction on fit cases and give the proposer its errors. Run the proposed replacement and the baseline on matched validation cases. The branch in the figure depends on those results. A plausible diagnosis earns a trial. A measured gain that meets the acceptance criteria earns a place in the next round. Generating several candidate answers for one user question instead spends computation on that instance.
The information available to the proposer determines what it can try. Automatic Prompt Engineer inferred instructions that could account for completed input–output pairs. OPRO showed the proposer earlier instructions alongside their measured scores. ProTeGi asked for a critique of errors on a small batch and generated edits addressing that critique, retaining several candidates for further search. Its textual gradient is feedback in words. AutoPrompt used numerical derivatives to suggest discrete token edits, while soft-prompt tuning fits numerical prompt values through training. For demonstration search, a simple baseline samples several sets from a reviewed pool and validates each set before selecting one.
One round of validation-guided search
The current instruction supplies both the failures that suggest an edit and the baseline against which to test it.
Read the diagram as text
- Current prompt.
- Inspect fit executions. Compare predictions with expected answers.
- Propose an edit. Address a recurring error.
- Compare on validation cases. Use the same task model and metric for both prompts.
- Retain edited prompt.
- Retain current prompt.
- Current prompt → Inspect fit executions: Instruction to execute.
- Inspect fit executions → Propose an edit: Mistakes and diagnostics.
- Propose an edit → Compare on validation cases: Edited instruction.
- Current prompt → Compare on validation cases: Baseline instruction.
- Compare on validation cases → Retain edited prompt: If validation improves.
- Compare on validation cases → Retain current prompt: If validation ties or declines.
A rising score can conceal two different failures. In Automatic Prompt Engineer’s TruthfulQA experiment, optimizing truthfulness could favor uninformative refusals. Fresh questions scored for truthfulness alone can reward the same shortcut. The remedy is to assess informativeness too. A sound metric can still yield an overfit prompt when repeated selection favors wording that happens to suit a finite set of questions. That failure calls for independent assessment. Automatic Prompt Engineer
Bound the search by a target or a budget such as a maximum number of evaluation calls. Check candidate length against the serving allowance before accepting a winner. Proposal and evaluation calls are development expenses. Every additional token in the selected instruction recurs across the workload. A run that exhausts its budget can return the baseline if no deployable candidate qualifies.
Inspect DSPy's prompt artifacts
In DSPy, a signature declares a task’s input fields, output fields and intent. A module makes that task callable. An adapter renders the declaration and actual inputs into messages and parses the reply into fields. The caller can therefore keep the same interface while changing the instructions used to implement it. Consider the subject-to-haiku signature in the DSPy 3.4.0 first program.
Illustrative pseudocode
Python-like pseudocodeIn the call history, look for the field descriptions, formatting instructions and subject value that the short declaration expands into. The caller reads the extracted text in reply.haiku. A four-line poem can occupy that field just as easily as a three-line one. If three lines are required, count them when scoring the unoptimized module. Field descriptions can communicate that requirement, and the metric determines whether a compiler gets credit for satisfying it. Kevin Madura’s call-history walkthrough
| Optimizer | Candidate material | Selection |
|---|---|---|
| BootstrapFewShot | Supplied labeled pairs and demonstrations generated by running the task | A metric checks generated predictions. Accepted examples populate the predictor up to configured limits. |
| MIPROv2 | Proposed instructions and bootstrapped demonstration sets | Bayesian optimization uses earlier scores to choose combinations to try. Trials can use small batches. Full-validation performance selects the returned program. |
Inspect the selected program’s state-only JSON to see its instructions, demonstrations and signature metadata. To reproduce that behavior in a fresh process, instantiate the program, load the saved state and configure the model client separately. The restored program and its target model are the combination to assess on untouched subjects. DSPy saving and loading
Use diagnostic feedback with GEPA
GEPA, named for genetic Pareto, is a prompt optimizer that uses feedback about individual executions to propose edits. In DSPy’s published haiku example, the revised instruction specifies line and syllable counts, imagery and poem-only output. To see why detailed feedback matters, suppose a line-count check fails. The score alone cannot distinguish a four-line poem from a three-line poem preceded by a title. Seeing the title suggests asking for the poem alone. GEPA calls this use of an execution record (a trace) and feedback to propose a revision reflection. It tests the revision on a small group of cases (a minibatch) before spending more calls on a promising candidate. Published haiku edit · Lakshya Agrawal on reflective search
| Candidate | Case 1 | Case 2 | Case 3 | Mean |
|---|---|---|---|---|
| A | Pass | Pass | Fail | 2/3 |
| B | Fail | Fail | Pass | 1/3 |
| C | Pass | Fail | Fail | 1/3 |
A dominates C because it matches or beats C on every case and improves on case 2. Only B handles case 3. GEPA’s Pareto selection keeps complementary case winners and samples parents according to their case-level strengths. B can contribute to another revision even though A leads on average. Merging candidates is optional and can introduce regressions. The original algorithm ultimately returns the highest mean on the selection set. If case 3 is mandatory, the deployment decision needs that additional check. Original GEPA algorithm, July 2025
The authors ran Qwen3-8B on held-out HotpotQA questions. Their exact-match scorer gives a case one point when the final answer matches a reference answer under DSPy’s matching rules and zero otherwise. The baseline passed 42.33% of cases, MIPROv2 passed 55.33%, and GEPA passed 62.33%. The two searches had matched caps on rollouts, meaning task executions. Actual usage differed, and GEPA received richer feedback and spent computation on reflection. Matching caps therefore leaves total cost open. DSPy 3.4.0 exposes a score-plus-text metric and an evaluation budget through its GEPA integration.
The quality of the diagnostic matters as much as its detail. In Mahmoud Mabrouk’s policy-judge experiment at Agenta, repeated runs of each task stayed on the same side of the development–validation split. That protected against repeated-task leakage. The reference judgments were still synthetic labels derived from assertions. Agreement with them measured agreement with that labeling procedure, leaving agreement with independent policy experts to be tested. Judge the Judge
Transfer and practical choices
Reassess transfer and sufficiency
Generalization concerns fresh cases. Transfer asks whether a result survives a change of condition, such as a different language, domain, input length or model version. A change in the kinds of cases encountered is a distribution shift. Replay the frozen prompt and its original baseline under the target condition before retuning. Their difference estimates the benefit that carried over. An improvement after editing on the new model answers a different question about how well the prompt can be adapted there. When deployment conditions shift
| Observed difficulty | Comparison | What it tests |
|---|---|---|
| The request lacks a policy fact. | Add the verified policy passage to the same question and instruction. | Whether sufficient evidence helps. A gain directs attention to request assembly. |
| A valid rare category disappears. | Inspect the saved instruction for exclusions. Repair the rule or demonstrated coverage and test fresh rare and common cases. | Whether restoring the distinction repairs the omission without sacrificing existing behavior. |
| The selected prompt exceeds the serving allowance. | Compare shorter sets of instructions or examples with room for the same required answer. | Which distinctions survive at a deployable length. |
| The task still fails with sufficient facts and a clear convention. | Check reviewer agreement. With reliable labeled data, compare a trained variant with the strongest deployable prompt on fresh cases. | Whether parameter adaptation adds enough to justify maintaining the trained model. |
Even within one domain, the better method can depend on the decision the application needs. In a March 2026 study, Smartling researchers compared expert-written zero-shot prompts, minimal DSPy signatures and GEPA-optimized signatures on linguistic work. Expert prompts retained advantages on some measures of detecting translation errors. Optimization helped on some measures of characterizing those errors. The methods had different access to expert effort and labeled data, and scoring was automatic. Choosing between them requires specifying which errors the workflow needs to find and what a missed error costs. Smartling comparison
Frequently asked questions
Is in-context learning training?
In ordinary in-context learning, the current input carries the examples and the model uses its existing weights. To repeat the demonstration condition in a fresh call, supply its examples again. An instruction can also state the same mapping. Fine-tuning requires loading the trained parameters. Soft-prompt tuning requires restoring the learned numerical prompt even when the base model is frozen.
How many demonstrations should I include?
Compare an instruction alone with a small set covering distinct decisions. Add a pair when it supplies a missing distinction, fits the rendered input budget and helps on fresh cases. Performance can level off or fall as the set grows, even on a task where the first examples helped.
Do roles, wording or example order reliably help?
A role can suggest a voice or perspective. A title such as expert supplies little guidance about a disputed decision. To test an accuracy claim, change the role, wording or ordering separately and compare on matched cases. An ordering selected on one model may rank poorly on another.
Does a written explanation verify the answer?
No. An explanation can be checked against evidence or an independent calculation. Its fluency alone establishes neither the conclusion nor what influenced the model. In controlled experiments, explanations sometimes omitted a cue that changed the prediction.
When is automatic optimization worth its cost?
For a recurring task, compare a bounded search with a handwritten baseline at the same required quality. Count expert and annotation time, proposal and evaluation calls, serving cost at expected volume, and the work of repeating the process after a model change. A higher validation score alone supplies no break-even estimate.
Does GEPA require DSPy?
No. GEPA can work with a standalone evaluator that executes a candidate and supplies diagnostic feedback. DSPy offers an integration for optimizing its programs and also supports other ways to select instructions or demonstrations.
When should I retest a saved prompt?
Retest after a change to the model, the rendered request or the workload. Run the saved prompt and its baseline on representative inputs under the new conditions. Preserve their results before making edits so that you can distinguish transfer from the benefit of retuning.
When should I supply context or try fine-tuning?
If a reviewer would need an absent fact, supply verified information and rerun the case. If the facts are available but the decision rule is unclear, clarify it or show contrasting examples. Fine-tuning is worth comparing when errors persist, reliable training pairs exist and independent cases can establish whether changing the model helps.
Open questions
Which examples earn their space at a fixed token budget? Similar reviews can supply useful local context or repeat a distinction already covered. Comparing fixed, similarity-selected and coverage-selected sets on the same queries at matched rendered lengths would show which gains survive the actual serving constraint.
When does automatic optimization repay its full cost relative to expert revision? Expert time and annotations enter at different stages, while a longer winning prompt incurs recurring expense. A comparison through one model or workload change, with both methods held to the same quality requirements, would give teams a basis for budgeting maintenance as well as launch.
How can search improve a rare mandatory behavior when only a few independent examples exist? Repeatedly selecting on those examples weakens their value as evidence, yet an average dominated by common cases can reward losing the behavior altogether. Progress would mean satisfying both requirements on fresh task groups with a single deployable prompt.























































