AI Engineer World's Fair 2026
Training Krea 2 - What matters in generative model training.
Loading slides…
Read the talk
Training Krea 2 — What Matters in Generative Model Training
Stylistic diversity depends on more than sampling: Krea’s training pipeline connects data curation, caption quality, scalable filtering and staged optimization to the images a model can imagine.
From a talk by Sangwu Lee
Before you start: Basic familiarity with neural-network training and transformers is helpful; the article introduces the diffusion and data-curation mechanisms it uses.
What if you do not know which image you want?
What should an image generator optimize for when the user is still discovering what they want? Krea 2 makes stylistic diversity a central objective. Sangwu Lee introduces its released Medium variant through the research decisions behind data curation and training; a separate colleague’s talk covers training infrastructure. The release includes available weights and inference code, with distinct licenses: the repository identifies Apache-2.0 code and community-licensed weights.
Lee contrasts that objective with the reliable outputs of large production systems, including Nano Banana Pro and the model labeled ChatGPT 2 in his slides. Lee estimates that these systems can take one or two minutes to return an image, without specifying version, resolution or service conditions. Reliability can also favor familiar compositions: his example is an average-looking person placed in the center of the frame. He interprets this narrowing of outputs as mode collapse.
The prompt burning skull makes the distinction visible. The slide shows four varied Krea 2 images above four similarly composed orange skulls labeled ChatGPT 2. The latter outputs are individually acceptable; the missing property is variety across attempts. Krea’s goal is faster generation that lets someone iterate through different visual ideas.
That changes what counts as a useful product. A user requesting a specific poster or birthday card may value a dependable finished result. A creative studio exploring a visual direction may instead need variation, quick iteration and controls for steering the search. The target is not simply a good image, but a useful range of possible images.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn to denoise a compressed image
Diffusion training starts with an image, adds noise and asks a model to learn how to remove it. Once trained, the model can start from random noise and progressively turn it into an image. Training supplies examples of the denoising task; generation repeatedly applies the learned operation.
Krea performs this work in a spatially compressed representation. An autoencoder encodes pixels into a latent space, the diffusion model generates within that space, and a decoder converts the result back into pixels. This is the central computational motivation for latent diffusion: a diffusion transformer processes a shorter sequence instead of directly modeling every pixel.
Dense attention has a quadratic dependence on sequence length. For n image tokens, its pairwise attention work scales as:
Spatial compression reduces the sequence that incurs this cost. The autoencoder adds another component to the system, but makes the generative model’s training problem more economical.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Do not filter away the aesthetic
Once the architecture is fixed, much of the research work becomes deciding what to feed it. For a model intended to preserve stylistic diversity, conventional image-quality filtering can remove exactly the examples it needs. A low-resolution CRT image may score poorly as a clean photograph while being an excellent example of a desired aesthetic. Oversampling conventionally attractive images changes the model’s distribution, even if every individual filtering decision looks reasonable.
Undesirable data therefore needs a more specific definition:
- Redundancy: duplicate samples and overrepresented concepts need deduplication and clustering-based rebalancing.
- Missing supervision: a captioner may repeatedly omit an important visual property, leaving the model to absorb it as an unexplained default.
- Resolution mismatch: a densely text-filled image may be useful eventually but too difficult to learn from at a 256 × 256 training stage.
These are different problems and require different filters. A single aesthetic score cannot resolve them.
Krea also tries hard to exclude AI-generated images from pretraining. Lee describes synthetic aesthetics as unusually sticky: distillation can quickly produce an attractive model, but it can also import the recognizable appearance of its teacher. That shortcut conflicts with stylistic independence. The exclusion is specific to pretraining; it does not imply that synthetic examples have no role in later preference training. Lee also admits a less technical motivation: simply distilling another model is less satisfying research.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A good image can still be bad supervision
Caption construction is a pipeline rather than a single request to describe an image:
- Extract visible text with OCR. Accurate text supervision matters for learning to render text.
- Add available metadata. If the image depicts a known person, include that identity.
- Generate a detailed caption with a VLM. Use the image and the gathered information to capture relevant visual content.
- Rewrite the caption into the training format. Once the description is sufficiently complete, convert it into JSON prompts or another desired representation.
The order matters: reformatting a caption cannot recover information that never entered it.
The painting example exposes that failure. A captioner describes the painting’s subject but repeatedly fails to mention that it is framed, hanging on a wall and surrounded by a white background. The screenshot shows a colorful painting of a pink flowering tree against a pale wall. During training, the wall remains in the image even though it is absent from the description. A later request for a painting can consequently produce a painting hung on a white wall, rather than the artwork alone.
The image itself is usable. The problem is the pairing between pixels and language: important context is consistently unlabelled. After unsuccessful attempts to make the captioner capture it, Krea filters out or undersamples this kind of data. Data quality includes whether the supervision can explain the image, not just whether the image looks good.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Spend expensive judgments where they matter
Lee describes image-training collections ranging from two to ten billion images. At that scale, even a reasonable per-image operation becomes consequential. Krea begins deduplication with inexpensive hashes, including pHash and MD5, before applying embedding methods to a reduced collection.
The later stage uses SSCD and SigLIP representations to address near duplicates and semantic redundancy. These representations serve different purposes: SSCD is designed for image-copy detection, while SigLIP supplies language-image representations useful for semantic comparisons.
| Stage | Representation | Role |
|---|---|---|
| Initial pass | MD5, pHash | Cheap duplicate reduction |
| Reduced corpus | SSCD descriptors | Copy and near-copy detection |
| Reduced corpus | SigLIP embeddings | Semantic comparison |
The sequence reserves more expensive comparisons for the images that survive the inexpensive pass.
Classifier construction follows a related pattern. First, use a large vision-language model to establish a useful judgment—for example, whether an image appears AI-generated. Refine that behavior through a system prompt or fine-tuning, then distill its decisions into a small SigLIP classifier. The large model helps define the filter; the small model applies it across the corpus. Lee describes the resulting classifier as somewhat reliable, with an inference cost suitable for billions of images.
Lee connects this approach to Essential-Web v1.0: a large language model supplies taxonomy or annotation judgments that a compact model can reproduce at pretraining-corpus scale. He cites an approximately 500-million-parameter annotation model. The point is to distill the curation decision, avoiding the cost of running the largest model over every document or image.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Find unwanted features—and missing concepts
Sparse autoencoders provide another route to dataset tags. Lee’s earlier work on CLIP and other vision representations suggests a practical use: train an SAE on visual features, feed it an image and inspect the sparse features that activate. His example activates features corresponding to a horse, black-and-white appearance and blur. These become an unsupervised tagging system without requiring a manually specified label for every training image.
Once a useful feature exists, it can guide filtering or oversampling. Signatures, watermarks and border artifacts are examples of unwanted properties that an SAE feature might capture. The condition is essential: the method can filter on a property only if the learned representation exposes a suitable feature. It can also preserve or increase a desired style instead of automatically treating unusual visual properties as defects.
Coverage requires the opposite question: what should be present? Krea uses Wikipedia concepts to check its dataset’s world knowledge, although Lee says he does not know how much this procedure helped. Articles are ranked by PageRank, and selected concepts are checked against the image collection using plain-text or embedding search. The accompanying technical account clarifies the selection as the top 90% of English Wikipedia articles by PageRank, followed by representability filtering. Lee relates this to CLIP’s broad concept coverage; CLIP’s documented query construction used Wikipedia frequency and article search volume rather than this PageRank procedure.
This search also uncovered Barack Obama the horse. Lee does not establish that this particular horse entered the training set and thinks it probably did not. The example nevertheless shows why checking concept coverage can reveal more than familiar names. Across the overall curation process, Lee reports approximately 30–40 in-house classifiers, heuristics and filters. Together they determine which visual distribution the otherwise fixed architecture can learn.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn semantics first, then shape the distribution
With the data prepared, Krea follows a sequence familiar from language-model development: pretraining, mid-training, supervised fine-tuning, preference optimization and reinforcement learning. A separately trained prompt expander accompanies the image model. Resolution also follows a curriculum: learning what a horse looks like does not initially require the pixels needed to render its finest details.
Krea trains progressively from 256 to 1K resolution. Lower-resolution training establishes text-to-image semantics; higher-resolution stages develop structure and detail. The resulting pretrained model is a malleable base, much as an autocomplete language model is a base for later chat or tool-use training.
Mid-training and supervised fine-tuning then shape that base toward intended applications. Krea curates illustration, graphic design, photography and cinematic imagery, using larger mid-training collections and more focused SFT data to mold the output distribution. Pairwise preference judgments subsequently polish the result. These comparisons make the process more opinionated: choosing one image over another teaches the model which outputs the team wants it to favor.
Reinforcement learning adds feedback on generated images. Krea uses a GRPO-inspired method: the model generates images, sends them to reward servers and learns from the returned feedback. Text rendering and anatomical structure are examples of capabilities targeted by this loop. Lee describes the flow and its objectives, without supplying a reward formula or measured improvement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Translate user intent into training-distribution language
A production system must also bridge the gap between a short user request and the detailed descriptions used during training. Krea trains a small LLM to turn the user’s prompt into a longer, more detailed prompt. Lee describes this as an almost essential production step: descriptions closer to the image model’s training distribution tend to produce better images.
For example, a prompt-expansion training pair can preserve a simple subject while adding visual detail. The following Python record illustrates the input/output contract with a horse prompt:
python
import json
training_pair = {
"user_prompt": "a horse in a field",
"expanded_prompt": (
"A photograph of a horse standing in a grassy field, "
"with its full body visible, natural daylight, "
"and an open landscape in the background."
),
}
print(json.dumps(training_pair, ensure_ascii=False, indent=2))
The learned component produces the expanded description; the image model consumes that description as conditioning. This additional specificity helps explain the distribution match, but also makes prompt expansion a place where the system chooses details the user left open.
Beyond prompt expansion, Krea is working on multi-expert on-policy distillation. Separate experts specialize in capabilities such as photography or text rendering, then transfer those capabilities into a single student. Matching each specialist in its area is the goal of this ongoing work, rather than a completed result reported in the talk.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the next experiment cheap
The useful research levers are not limited to a particular optimization method. Infrastructure that permits fast iteration makes it easier to discover which data and training choices matter. Curated data has a longer useful life than a particular implementation: code and fashionable training methods change, while a good collection can remain valuable across those changes.
Lee favors simple, scalable methods with few hyperparameters to tune. Efficiency matters because it shortens the experimental cycle. Borrowing from LLM research also provides more than conceptual inspiration: it creates opportunities to reuse kernels, implementation work and an extensive body of literature. These choices reduce the cost of trying the next idea.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Simpler models, richer descriptions
Prompt expansion creates an architectural echo. In Lee’s comparison, an autoregressive decoder first expands the text, followed by an encoder-like diffusion transformer—a reversal of the familiar encoder–decoder ordering. It also resembles DALL-E 2 at the level of a model producing conditioning for another generative model. The analogy has a precise limit: DALL-E 2’s prior generates a CLIP image embedding, not an expanded text prompt, and its diffusion image generator is called a decoder.
That growing stack motivates a different aspiration: remove the VAE and separate text encoders, and train one clean transformer. Lee presents this as a direction he wants to pursue, not the architecture already used for Krea 2.
At the same time, stronger vision-language models can make the conditioning richer. The original latent-diffusion work already included bounding-box conditioning; Lee argues that generating useful boxes for training images is now much more practical. He names Ideogram and Reve as pursuing related directions. Boxes give the generator explicit spatial information that a conventional caption may leave implicit.
Scene graphs extend the idea from object locations to relationships among objects. Lee points to earlier work from Fei-Fei Li’s lab, while the displayed figure contrasts sentence and scene-graph conditioning leading to generated images. His broader proposition is that image-generation progress tracks progress in VLMs: better visual understanding can produce better descriptions to train and condition the generator.
The remaining question is which representations should carry that understanding. Bounding boxes and scene graphs are promising because they describe aspects of an image that prose may omit or express ambiguously. The technical ending returns to the same issue exposed by the painting on the wall: what the training system can describe determines what the image model can learn to control.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Official inference instructions and links to RAW and Turbo checkpoints, with guidance for sampling and fine-tuning.
A web-corpus curation system that uses a compact model to annotate documents with a structured taxonomy.
Language-supervised visual representation learning, including the construction of a broad concept-query list for collecting image-text pairs.
Pretrained image-copy descriptors and instructions for embedding extraction, similarity measurement and evaluation.
The original SigLIP method for training image-text representations with a pairwise sigmoid objective.
The foundational latent-diffusion paper, including compressed image representations and text or bounding-box conditioning.
A two-stage image generator combining a text-conditioned image-embedding prior with a diffusion decoder.
Further reading
- Krea 2 Technical ReportArticle
The team's detailed account of data curation, architecture, staged training, prompt expansion and infrastructure.
Johnson, Gupta and Li Fei-Fei's method for generating images from object relationships through predicted layouts.
Full transcript
Loading transcript…