AI Engineer Summit 2023

Prompt Engineering Tactics

Loading slides…

Read the talk

Prompt Engineering Tactics

Small prompt changes can alter an AI product’s behavior. Task-specific personas, named sources, and emotional cues offer three ways to shape the response.

From a talk by Dan Cleary

Why not just ask for what you want?

Why engineer a prompt if you can simply tell a model what you want and usually get something useful back? That is the starting question for Dan Cleary, co-founder of PromptHub, a tool for teams to test, collaborate on, and deploy prompts. Getting a good answer once is useful; getting dependable behavior inside a product is a harder requirement.

Model outputs are nondeterministic, and small changes to a prompt can have outsized effects on the response. An answer that goes off the rails can damage trust in the product delivering it. Cleary illustrates the stakes with a slide pairing poor prompts with a headline about Replika chatbot behavior. Prompt quality becomes part of product quality when users encounter the output as a feature rather than an experiment.

Slide titled “Poor prompts can lead to disaster” with a Replika news headline and a presenter inset partly covering its lower-left corner.Open full source frame
“Poor prompts can lead to disaster,” illustrated by a headline about Replika chatbot behavior.

Access to ChatGPT has also raised expectations: users want crisp answers that match their request, arrive quickly, and do not hallucinate. Cleary offers three tactics for improving responses, whether the prompt lives in an everyday chat or an AI product feature.

0:150:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:15 · section reference included

Let task-specific personas collaborate

Multi-persona prompting asks the model to approach a task through several relevant roles. Cleary attributes the method to University of Illinois research; the matching work, Solo Performance Prompting, uses a single language model to simulate collaborating personas. The agents here are perspectives generated within the model’s response, not necessarily separate running systems.

For a book-writing task, Cleary suggests an author, a publicist, and the intended audience. The model leads a brainstorming exchange among them, allowing their different priorities to influence the result. The author can develop the material, the publicist can consider its appeal, and the audience can respond to what would make it worth reading. A prompt can make that sequence explicit:

Help me develop a book idea.

Bring together three personas: an author, a publicist,
and a representative of the book's intended audience.

Lead a brainstorming discussion. Have the personas propose
ideas, respond to one another, and revise their suggestions.
Show the collaboration, then present a final recommendation.

The important operation is the exchange: personas respond to one another before the model produces its final answer. Merely listing three roles would leave that collaboration unspecified.

The visible discussion lets the reader inspect how the proposed answer developed. Cleary recommends the technique for complex tasks or tasks requiring additional logic, and personally favors it for generative work. In the book example, its value is in developing and revising ideas through several perspectives.

1:512:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:51 · section reference included

Name the source you want the answer to follow

The according-to method changes a smaller part of the prompt. Start with Cleary’s digestive-system question, then append a source:

In what part of the digestive tube do you expect starch
to be digested? According to Wikipedia.

The suffix directs the model toward a particular source’s account. In the matching original according-to study, the starch example concerns initial digestion, and both the baseline and grounded answers identify the mouth. The example demonstrates more source-overlapping detail, rather than correcting an initially false answer.

Naming a source does not fetch it. Although Cleary describes the model going to the source to retrieve information, the studied mechanism relies on recall from training data, without internet access. A source cue can influence the answer, but it does not establish that the model consulted or verified a live page.

Cleary reports that according-to prompting can reduce hallucinations by up to 20%. The original study does not establish that general hallucination-rate reduction: it measures quotation overlap with Wikipedia and, separately, question-answering performance. More overlap with a source is not itself proof that an answer is correct.

Cleary recommends this tactic when a fine-tuned or general model should consistently draw answers from a particular source. He attributes the research to Johns Hopkins University and introduces it as recent work. The useful design choice is to make the desired source explicit instead of leaving the model to infer it.

2:533:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:53 · section reference included

Add an emotional cue

EmotionPrompt, which Cleary attributes to Microsoft and several universities, tests emotional stimuli appended to ordinary prompts. His analogy is a boss explaining that a project matters to your career or to a major client: the stakes may lead you to take the work more seriously. That human behavior motivates the prompting technique; the method tests whether similar language changes model responses.

The implementation is simply to preserve the task and add the cue at the end. The slide shows the addition “This is very important to my career.” For a writing request, that pattern could look like this:

Revise my cover letter for clarity and relevance to the job.
Preserve the facts in the draft.

This is very important to my career.

The task still specifies what to do. The emotional sentence supplies an additional signal about its importance.

Cleary says he has seen better outputs for tasks ranging from cover letters to change logs. Those are his practical observations. The September 2023 EmotionPrompt study reports relative benchmark improvements of 8% on Instruction Induction and 115% on BIG-Bench. Its expanded evaluation covers 24 Instruction Induction tasks and 21 BIG-Bench tasks across six models, with zero-shot BIG-Bench experiments. The study distinguishes original and APE prompt baselines, as well as average and best emotional-stimulus results; the headline percentages are not measured gains for the cover-letter or changelog examples. The slide pairs the prompt modification with an Original versus Ours results table across six models.

EmotionPrompt slide with original and modified prompt boxes, the addition “This is very important to my career,” and an Original versus Ours table for six models. A presenter inset overlaps the lower-left corner.Open full source frame
EmotionPrompt contrasts an original prompt with an emotional addition and shows results across six models.
3:373:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:37 · section reference included

Try the prompts with your team

These methods fit both direct use in ChatGPT and prompts embedded in product features. Cleary closes by offering all three as PromptHub templates: copy a prompt, run it through the playground, and share it with teammates or access it through a link. The public multi-persona collaboration template provides a starting point for the persona-selection and iterative-discussion pattern. The practical next step is to try a tactic on the task you actually need the model to perform.

4:334:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:33 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…