AI Engineer Summit 2023

No-code Fine-tuning: Mark Hennings

Loading slides…

Read the talk

No-code Fine-tuning

Mark Hennings shows how a team can turn examples into a specialized language model, evaluate it against a prompt, and build a press release workflow around facts.

From a talk by Mark Hennings

Before you start: Familiarity with prompting a language model is helpful; no programming or model-training experience is required.

What fine-tuning specializes

How do you write a product description, normalize messy text, or decide which support issue deserves attention first? These are recurring business tasks, but specifying the right behavior as rules can be difficult. Fine-tuning means training a foundation model for a specialized task. Mark Hennings opens with three overlapping groups:

  • Writing: emails, blog articles and product descriptions.
  • Transforming data: scrubbing fake emails from a list, extracting or normalizing information, translating, paraphrasing and rewriting.
  • Making judgments: qualifying sales leads, ranking support priorities, detecting fraud and flagging inappropriate content.

The common challenge is interpreting language with enough nuance to produce a useful result. Hennings argues that language models handle this more easily than traditional programming or rule-based approaches.

Slide lists eight specialized tasks, with a callout describing them as hard for rule-based systems or programming and easy for large language models.Open full source frame
Specialized tasks for language models, from writing copy to detecting fraud.

A prompt can already accomplish many of these tasks. The question is what changes when recurring instructions and examples become training data for a specialized model.

0:220:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:22 · section reference included

Speed, cost and shorter prompts

The proposed efficiency gain starts with matching the prompted model’s quality using a lighter model. If training teaches that smaller model the required behavior, inference can become faster and cheaper. Removing repeated instructions also shortens inputs, leaving more context space for completions. Training examples provide another place to express edge cases and let teammates contribute improvements.

Hennings also asserts that fine-tuning is naturally resistant to prompt injection, but he does not demonstrate an attack or a defense. Ordinary task fine-tuning should not be treated as a security guarantee.

For speed, Hennings compares GPT-4 with GPT-3.5 on the OpenAI API, then gives a cost example comparing GPT-4 with fine-tuned GPT-3.5. These are figures from the historical 2023 demonstration, not current model or pricing guidance.

ComparisonReported result
OpenAI API response time per tokenHennings reports approximately 196 ms for GPT-4 and 73 ms for GPT-3.5.
GPT-4 versus fine-tuned GPT-3.5 costHennings reports 88.6% savings in his example.

He describes the latency difference as roughly three times faster. The talk does not specify the model snapshots, workload or measurement procedure, and it does not supply the token counts or training-cost allocation needed to reproduce the savings calculation.

The prompt-compression example is a blog writer. Its engineered prompt specifies the topic, writing approach, tone, desired behavior and things to avoid. With fine-tuning, examples teach the recurring writing behavior; the request can concentrate on the topic that changes from one article to the next.

Hennings describes this particular blog-writing prompt as 90% shorter. The reduction depends on how much of the original prompt consists of reusable instructions. The useful separation is between behavior shared across requests and information unique to the current request.

1:231:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:23 · section reference included

Training data as a shared working surface

Imagine a GitHub repository whose entire codebase lives in one file. Hennings uses that as an analogy for a team maintaining one enormous prompt: everyone must work on the same artifact. Fine-tuning offers something closer to a repository with multiple files, where contributors can work on different pieces. Those pieces are training examples. Team members can add, edit and improve examples in a shared dataset, which then feeds the fine-tuned model.

Diagram contrasts four people pointing toward One Epic Prompt with four contributors above training data that feeds into a Fine-tuned Model.Open full source frame
Team contributions to one epic prompt versus training data feeding a fine-tuned model.

This makes the dataset a collaborative layer for specifying behavior. The reason to adopt it remains conditional: the resulting model needs to produce equal or better output. Hennings then turns to the practical obstacle—fine-tuning still looks like a developer’s job.

2:573:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:57 · section reference included

Lowering the technical and data barriers

A typical fine-tuning tutorial can involve provisioning GPU servers for training and inference, formatting data with ad hoc Python scripts, configuring parameters and making API calls. Hennings’s proposal is to automate that operational work behind a user interface. The person who knows what a good answer looks like should be able to contribute without owning every infrastructure step.

Hennings says 20 examples can be enough to start a fine-tuning experiment. They can even be handwritten, avoiding the initial burden of assembling the thousands of examples he associates with traditional machine learning. Starting an experiment is distinct from establishing production quality; that requires evaluation.

His intuition comes from few-shot learning. A prompt might contain five examples of the desired behavior. A training dataset can instead contain 20 or 100, without placing all those examples into each request. This extends the amount of example-based guidance available to the model. Hennings presents improved task fit from additional examples as an intuition, rather than a measured result.

3:423:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:42 · section reference included

Prototype, evaluate and learn from production

Prompt engineering remains the starting point in the proposed development lifecycle:

  1. Prototype with a prompt. Validate the concept and use the prompted model to help create an initial training dataset.
  2. Fine-tune and evaluate. Check whether the trained model improves on the prompted version, then test which models can achieve the required performance.
  3. Deploy after evaluation. Put the selected fine-tuned model into production.
  4. Feed experience back into training. Capture user feedback and log real examples, then incorporate useful cases into the dataset for subsequent improvement.

The loop connects the initial specification to the situations users actually encounter.

Developers can operate this process, but they need not be its only participants. People who understand the task can help with prompting, example creation and fine-tuning. Hennings introduces Entry Point, identifying himself as a co-founder, as tooling intended to make that participation practical.

4:565:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:56 · section reference included

Building the Press Release Writer dataset

The demonstration opens the Press Release Writer project in Entry Point, containing 20 examples. Hennings began with the desired outputs: he found 20 strong press releases through articles collecting examples of effective releases. That gave him writing to learn from, but not complete input–output pairs.

To supply the missing inputs, he used GPT-4 in ChatGPT to work backward from each release. The task was to extract the list of facts a professional writer would need to produce it. Each resulting pair therefore associates a factual brief with a finished press release.

Facts are the input; polished prose is the output. Hennings chooses that boundary because he does not want to rely on the model for factual knowledge. The intended result is a strong first draft built from supplied information. The examples teach how to turn that information into a release.

6:026:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:02 · section reference included

From structured fields to a training job

Entry Point exposes the training data as structured records. When a CSV is imported, each column becomes a field; this project has fields for the facts and the press release. Keeping them visible makes it easier to inspect what will enter the model’s training data.

Fields are inserted into input and output templates using Handlebars, much like inserting a contact’s first name into a personalized email. With illustrative field identifiers facts and press_release, the input template can be:

handlebars

{{facts}}

The corresponding output template can be:

handlebars

{{press_release}}

Applying those templates to each record supplies its factual brief as the input and its existing release as the target output. The templates define the format once; the records provide the changing content.

Fine-tuning does not require abandoning instructions. The demonstrated GPT-3.5 Turbo workflow also supports a system prompt. Hennings uses that to describe a hybrid: examples teach the task while a system instruction provides additional guidance, including when the dataset is small.

Once the dataset and templates are ready, the interface handles the training setup:

  1. Open Fine-tunes and select Add.
  2. Choose the model and platform; Entry Point supports multiple platforms.
  3. Review the token count and estimated cost.
  4. Press Start to launch the job.

Hennings quotes an estimated training cost of about $1 for the demonstrated job. The talk does not provide the token counts, selected model or training settings needed to reproduce that estimate. He then switches to an already-trained model in Entry Point Playground, rather than waiting for the new job to finish.

6:527:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:52 · section reference included

Iterating on facts instead of prompt boilerplate

In the Playground, Hennings supplies facts he wrote about the AI Engineer Summit. The trained model generates a draft with a title and recognizable press-release formatting. The demonstration shows the facts-to-draft workflow; it does not establish held-out quality or factual accuracy.

The generated draft becomes material for another pass:

  1. Write the list of facts.
  2. Generate the article.
  3. Read the draft and use it to identify ideas or refinements.
  4. Revise the facts and generate again.

This is the authoring loop Hennings finds useful. Training data carries recurring writing conventions, so each iteration can focus on what the release should say. The user works on the factual brief while the trained behavior supplies much of the writing boilerplate.

Hennings closes by mentioning Entry Point’s data-synthesis features and tools for comparing fine-tuned models, neither of which is demonstrated here, and directs viewers to the product. The completed walkthrough ends with a more focused writing process: inspect the draft, improve the facts, and let the examples carry the reusable style.

8:218:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:21 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…