AI Engineer World's Fair 2024
Running AI Application in Minutes: Quick Start with AI Templates
Loading slides…
Read the talk
Running AI Application in Minutes: Quick Start with AI Templates
Build and deploy a Python chat app, add retrieval over database rows or documents, and follow the engineering decisions that turn a working demo into a testable application.
From a talk by Pamela Fox, Harald Kirschner and Gabriela de Queiroz
Before you start: Familiarity with Python, environment variables, and a terminal will help; following the deployment steps also requires GitHub and Azure access.
Getting an application running before building everything yourself
How do you get an AI application running when you have an idea, limited resources, and little time to assemble the infrastructure? Microsoft’s Build with AI templates supply an application skeleton, while its startup program supplies credits, tools, and guidance. Gabriela de Queiroz introduces Founders Hub as accessible to people with an idea, not only established companies. She describes up to $150,000 in Azure credits, alongside third-party tools, GitHub, Microsoft 365, and LinkedIn Premium. Model options include OpenAI, Llama, Cohere, and Mistral offerings.
The support also includes volunteer one-on-one sessions about hiring and technical decisions. Build with AI provides the templates; programs such as Pegasus address later co-selling and go-to-market work, alongside partnerships with Y Combinator, Neo, and The Alchemist. These offerings address four practical constraints: time to market, resources, scalability, and access to guidance.
Pamela Fox takes the workshop through three application paths, beginning with the smallest one that can establish a working development and deployment loop.
| Template | Data it uses | Purpose |
|---|---|---|
| Basic chat | Conversation messages | Verify model access and application deployment |
| Postgres RAG | Database rows | Answer product questions, with SQL filter building |
| Document RAG | Blog posts or internal documents | Retrieve relevant passages before answering |
The two retrieval-augmented generation, or RAG, applications add data access after the basic chat app works.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Establish the account and model connection
The original workshop instructions open as a Word document in the browser. Following them requires a GitHub account and a Microsoft account. The workshop’s Azure Pass provided $50 of credit valid for seven days. These credits, temporary proxy access, account restrictions, and model choices belong to the recorded workshop setup; current repository instructions and service terms may differ. Before redeeming anything, check the Microsoft identity already signed into the browser—Fox has seen attendees accidentally use their children’s Minecraft accounts. A new personal account is an option if needed.
The redemption procedure keeps two identities distinct: GitHub identifies the attendee receiving a code, while the Microsoft account receives the Azure subscription.
- Follow the document’s check-in link and sign in with GitHub.
- Copy the individual promo code and choose Get on board with Azure.
- Confirm the Microsoft account before entering the code. Fox recommends a personal account because corporate accounts can restrict deployments.
- Redeem the code, then open the Azure portal’s Subscriptions page.
- Confirm that Azure Pass - Sponsorship appears, and select that subscription when deploying.
Fox’s redemption attempt fails because she has already completed it. The workshop’s assurance that attendees will not personally pay for deployment depends on using the sponsored subscription.
Next comes model access. The workshop uses a proxy to avoid waiting for the Azure OpenAI application-and-approval process Fox describes. Signing into the proxy with GitHub yields an API key and endpoint. This is a temporary exception to her preference for avoiding keys; ordinary application keys still need protection.
The proxy playground provides an immediate connection test. Fox changes the system message to request pirate jokes, enters the key, selects a model, and asks it to explain OpenAI. The response combines the informational request with pirate phrasing. Available choices include GPT-3.5 Turbo and GPT-4; she describes GPT-4 as better but slower in her experience. She discusses GPT-4o as her preferred vision option over GPT-4 Turbo with vision, while the hands-on applications use GPT-3.5 and embedding models. The playground also exposes the response-token limit, temperature, top_p, and input/output token usage. Temperature and top_p affect sampling; the workshop’s description of them as creativity controls is an informal intuition.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Open the template and understand its async backend
The AI App Templates Workshop supplies proxy-specific READMEs for all three applications. Start with the OpenAI Chat App guide, choose Open in GitHub Codespaces, and create a Codespace. Using these instructions matters because the proxy configuration differs from the repositories’ ordinary Azure OpenAI setup. Codespaces opens VS Code in the browser with a development environment for the repository.
Startup includes building a Docker-based environment, installing dependencies, and potentially synchronizing editor extensions. The build logs make that work visible. Harald Kirschner points out that Codespaces prebuilds can move preparation ahead of the interactive session; Fox has enabled them for the third repository, but not this first one. All three examples use Python backends. The initial chat app has vanilla JavaScript in a script tag, while the later applications use TypeScript, React, and Microsoft Fluent UI.
The source folder leads to a Quart application. Its Flask-like interface makes the async programming model familiar: an async function is a coroutine, and awaiting an I/O operation lets the server handle other work while the operation is pending. This matters when a model request takes several seconds. Async does not make that model generate faster; it prevents waiting for the response from monopolizing the request-processing path. The templates use Quart or FastAPI for this reason.
The essential Python pattern is an awaited request through an asynchronous client. A helper can accept the configured client and deployment name, leaving connection setup separate from message handling:
python
from openai import AsyncOpenAI
async def answer(
client: AsyncOpenAI,
deployment: str,
question: str,
) -> str:
response = await client.chat.completions.create(
model=deployment,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": question},
],
)
return response.choices[0].message.content or ""
The await is the point at which the coroutine can suspend while the model service responds.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Configure the connection and make the chat app your own
Create .env from .env.sample, then populate the proxy endpoint, API key, and deployment name. The workshop endpoint ends in /v1, and its deployment is gpt-35-turbo. If the terminal disappears during Codespaces setup, the terminal panel’s plus button opens another one.
An Azure deployment name is distinct from a model name. Azure can host multiple deployments of the same underlying model, each with its own name and capacity allocation. The application must address the deployment it intends to use. In this workshop, the deployment and model naming happen to align, but that is not a general assumption to build into connection code.
Run the guide’s Quart command in the Codespaces terminal, allowing clipboard access if the browser requests it. The server is local to the remote development container, not to your laptop, so copying its raw localhost address into a separate browser tab does not reach it. Instead, modifier-click the terminal link or open Ports and use the forwarded address or globe icon. Making the port public allows development sharing; the forwarded URL is still a preview, not the deployed application.
The first test asks about the weather in San Francisco. The assistant declines to supply information it does not know. Fox then replaces the generic helpful-assistant system message with instructions to make pasta jokes. Asking about the weather again produces a response about saucy weather and feeling like a soggy noodle. The edit demonstrates that the application is connected, that changes reach the running server, and that system instructions affect responses. It does not add a weather data source.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Deploy with a separate environment configuration
Once the local app behaves as intended, authenticate the Azure Developer CLI:
bash
azd auth login
Inside Codespaces, the demonstrated login uses a device code and browser flow. Select the same Microsoft account that redeemed the Azure Pass.
Codespaces is convenient for a workshop because it reduces differences between attendees’ machines. You can also run the repository in a local Python virtual environment after installing its requirements, or use VS Code’s Dev Containers extension for the Dockerized environment on your machine. The speakers settle on a stated 60-hour monthly Codespaces allowance during the session; treat that as the workshop’s account guidance, rather than a current entitlement for every account.
Next, create a named azd environment. Its variables live under .azure/<environment>/.env, separate from the root .env used by the local server.
| Configuration | Used by | Purpose |
|---|---|---|
.env | Local application | Development model connection |
.azure/<environment>/.env | azd | Deployment configuration |
Set the deployment variables from the workshop guide: disable creation of an Azure OpenAI resource, then supply gpt-35-turbo, the proxy key, and the proxy endpoint. Configuring the local server does not automatically configure the deployment. Inspect the environment file before proceeding.
Run azd up, choose the sponsored subscription, and select a region. The first template packages the code as a Docker image for Azure Container Apps. Provisioning then creates the container app, container registry, Container Apps environment, and Log Analytics workspace. Another template uses Azure App Service.
Infrastructure as code makes the deployment repeatable. Bicep declares the resources, their connections, and role assignments, so provisioning can recreate the intended arrangement. The same approach can connect a larger stack containing Postgres, Key Vault, Redis, and application hosting. Terraform is an alternative, but these examples use Bicep. Progress appears in the terminal and Azure portal.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recover from region and naming constraints
The live deployment encounters a constraint: Fox already used Central US for an earlier demonstration. She describes the workshop pass as allowing one container app per region. To recover, she creates another environment, copies the configuration, changes the region to West US, selects the new environment with azd env select, and runs azd up again. The working app she then opens was deployed previously; its Container Apps URL distinguishes it from the Codespaces preview.
The Bicep files are handwritten, with core modules copied from a shared repository. Fox describes a planned move to Azure Verified Modules, or AVM, to use maintained modules incorporating security practices. Bicep modules can come from a public registry, private registry, or local folder. For workshop troubleshooting, short environment names without symbols avoid some naming problems; azd env new starts a fresh environment when necessary. Naming rules, regional restrictions, and account constraints are the recurring failure categories.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Ground answers in retrieved database rows
The pasta assistant is entertaining but unreliable as a source of facts about Fox: asked what she likes to code with, it invents Spaghetti-Python. A blog-backed assistant first declines an ambiguous question, then answers a more specific question about her Python frameworks with citations. The difference is access to a relevant knowledge source.
The core RAG sequence is straightforward:
- Receive a user question.
- Search a database or search engine for relevant material.
- Place matching sources alongside the question in the model request.
- Instruct the model to answer from those sources.
The PostgreSQL RAG template applies this to product rows. Asked for the best shoe for hiking, it searches the database and produces an answer with citations that the user can inspect.
The interface’s Thought process view exposes the application’s retrieval stages, rows, and prompt. A separate model call first cleans up the user’s wording: Please tell me about the best shoes for hiking now becomes best shoes for hiking. Retrieval uses that shorter query. The final answer request then includes the original question, retrieved sources, and instructions for formatting citations. The two model calls have different jobs: one prepares the search, and the other synthesizes the answer.
This template can run against a local Postgres database before deployment. Its React frontend adds a richer interface, but the workflow remains familiar: configure the application, test locally, set deployment variables, and deploy to Azure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn documents into searchable, citable chunks
Documents require an additional preparation stage. The Azure Search OpenAI demo indexes material such as PDFs, Word documents, and spreadsheets. Fox describes a sample developed over more than a year and deployed by thousands of developers, including production users. Its features include speech, voice, vision, and user access control. She demonstrates versions backed by her blog and an internal company handbook, with clickable citations.
A 31-page handbook illustrates why the application retrieves passages instead of sending the entire document. Fox discusses context windows of 8K, 32K, and 128K tokens, but fitting the input is only part of the problem. Lost in the Middle examines how the position of relevant information affects long-context performance. Evidence can become harder to use when buried among other material. The retrieval goal is therefore to supply the most relevant chunks, not simply the largest possible prompt.
The ingestion pipeline has four responsibilities:
- Extract text. Azure Document Intelligence reads the source document.
- Split it. The sample uses chunks of roughly 500 tokens.
- Embed and index it. Store each chunk and its embedding in Azure AI Search.
- Preserve provenance. Retain the source file and page alongside the chunk.
At question time, search returns those records, and only the selected chunks enter the answer prompt. File and page metadata make the resulting citations navigable. Ingestion can run locally or in the cloud; its processing and additional infrastructure make these templates slower to deploy than the basic chat app.
Much of the sample’s ingestion code was written before the current range of libraries was established. Fox would now consider a LangChain splitter, and names LlamaIndex as another ingestion option. The crucial property is token-aware chunking, especially for multilingual documents: a character count does not reliably predict the token budget. A Chinese passage with a seemingly modest character count can consume much more context than expected. Anthony Shaw’s analysis of Chinese, Japanese, and Korean text adds a second requirement: preserve character boundaries while measuring token length, rather than cutting blindly through encoded token sequences.
After splitting, the OpenAI SDK can embed multiple chunks in a batch before they are stored in AI Search. Extraction remains format-specific. Besides cloud Document Intelligence, the sample has local options: pypdf for PDFs, a custom CSV parser, and Beautiful Soup for HTML. Fox uses Beautiful Soup to extract her blog’s text. A reusable splitter helps with chunk boundaries, but it does not eliminate the need to understand the source format.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Understand what changes when infrastructure changes
An audience question about replacing Bicep modules leads to the distinction between a module and a deployment. Bicep compiles into Azure Resource Manager, or ARM, JSON. An ARM deployment compares requested resource configuration with existing resources and applies the necessary changes. Moving to AVM is intended to reduce module maintenance, but it still requires reviewing the resulting resource configuration. Fox points to Azure CLI deployment what-if support for previewing changes; she does not establish an equivalent azd dry-run command during the session.
azd orchestrates the larger workflow. The repository’s azure.yaml maps application code to a host; infrastructure definitions describe the resources. The CLI combines packaging, infrastructure provisioning, and code deployment. AVM supplies reusable infrastructure modules within that process. This separation lets a similar development workflow target Container Apps, Functions, App Service, or Kubernetes, while the actual resource definitions and deployment host vary. Fox’s example catalog marks repositories that already support azd.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make quality observable and testable
Deployment opens the production questions: how will the application behave under load, and how will you know when it fails? The document-RAG repository includes a productionizing guide and integrates Application Insights through OpenTelemetry. Fox also names Langfuse as an observability alternative she likes.
Production preparation includes scaling, balancing OpenAI capacity, VNet deployment, user authentication, and load testing. Answer quality needs its own tests. A few successful demonstration questions—or a prompt edit that improves one answer—cannot establish overall improvement. Fox recommends roughly 200 evaluation samples as a minimum in her practice. That is a practical starting recommendation, not a demonstrated statistical threshold.
Kirschner describes Copilot Chat as several retrieval systems working together. Its @workspace path uses a local sparse index; another path uses a semantic index maintained by GitHub, with LLM-based ranking. Query keywords can be generated before retrieval. The evaluation cadence has two layers: a smaller regression-oriented set on pull requests and a larger daily set covering repositories across languages. Previously observed failures become regression cases. He identifies evaluation as an early major investment because prompt-crafting intuition is easy to mistake for dependable behavior. The local sparse implementation uses TF-IDF.
Fox’s blog evaluations combine model judges with deterministic checks. The judge compares a new answer with a synthetic reference answer, using an example-bearing prompt and a one-to-five rating scale. Groundedness and relevance use these GPT-based metrics. Her preferred simple check extracts citations and verifies that the generated answer includes the citations expected from the reference. The discussion also identifies passing code tests as another evaluation method for code tasks.
The experiment axes include text-only search, vector-only search, hybrid search, and hybrid search with a ranker; retrieving three, five, or ten results; and changing the prompt. Fox reports that her prompt tweaks had not produced enough aggregate improvement to justify changing the sample prompt. Retrieval parameters and switching from GPT-3.5 to GPT-4 made more meaningful differences in her experiments. This gives evaluation a concrete job: decide which changes deserve to become the new baseline.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Inspect the failures behind retrieval scores
Choosing a vector store or chunk size becomes easier when the evaluation material is familiar. Fox starts with aggregate results, then compares individual answers between the baseline and experimental runs. Filtering for changed citation matches helps identify cases worth reading. Domain knowledge is needed to determine whether the answer improved—and whether the evaluator itself made a sensible judgment.
The Azure AI Search team’s retrieval experiments motivate comparing hybrid retrieval with semantic reranking. Fox then tests retrieval choices on synthetic questions about her own blog, reported in Doing RAG? Vector search is not enough. These are generated-answer groundedness scores on a one-to-five scale, not retrieval accuracy:
| Retrieval mode | Groundedness |
|---|---|
| Vector only | 2.79 |
| Text only | 4.87 |
| Hybrid | 3.26 |
Text-only retrieval performs strongly in this application. Basic hybrid retrieval merges text and vector rankings using reciprocal rank fusion, but including vector results also introduces distracting material.
The failure examples are concrete. Fox has seen near-empty text embeddings match too broadly. In a customer’s vision-enabled application, a blank blue page embedded with an Azure Computer Vision model appeared repeatedly among results. Such candidates can occupy context that should contain useful evidence. Adding vector search therefore creates another retrieval path whose output must be inspected, not an automatic improvement.
Hybrid retrieval with the semantic ranker produces the best result in Fox’s comparison. Her published table gives groundedness of 4.89 for hybrid with ranker, versus 4.87 for text only. That is a small groundedness difference; it should not be read as a several-percentage-point gain on that scale. The complete comparison shown on screen includes groundedness, relevance, answer length, and citation match.
Fox describes the semantic ranker as a cross-encoder trained using human judgments of query/result relevance, also used in Bing. It evaluates the query together with candidate results and reorders them. Her preference for text search over vector-only search is specifically conditioned on Azure AI Search’s strong full-text retrieval. The brief uncertainty about SQLite is not a reason to rule it out: SQLite FTS5 provides full-text search, and this discussion establishes no measured SQLite-versus-Azure comparison.
The full-text discussion names Lucene, text analysis such as tokenization, and BM25 ranking, with Copilot’s TF-IDF index as a comparison point. These are related lexical-search techniques, not interchangeable algorithm names. Fox also explains the origin of the sample’s roughly 500-token chunks: the cited Search experiments used 512-token chunks. Treat that configuration as a useful starting point to evaluate on your documents, rather than a universal optimum.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Update embeddings deliberately, and retrieve code by structure
Vectors need updating when their source content changes. Changing the embedding model is broader: the corpus must be embedded again in the new model’s space. Fox proposes a separate AI Search index to evaluate the new text-embedding-3 family, with dimensionality as another experimental choice. She has not yet tested the improvement herself. The discussion mentions 256 and 512 dimensions without settling a final configuration. Kirschner recommends testing, including customer A/B tests where available; the reindexing work remains part of the migration.
The next demonstration makes code retrieval visible inside VS Code. Inline chat applies natural-language requests directly to code, while side-panel chat can discuss a selected region. Kirschner selects tests and asks about them; the response includes highlighted references and navigation to dependencies. Selecting relevant code gives the assistant an explicit context boundary.
For repository-wide questions—such as which tests exist or how benchmarks run—he invokes @workspace. In the demonstrated setup, the repository is primarily searched locally; he describes GitHub-maintained semantic indexes as available for Enterprise and some open-source repositories. The local sequence expands the query with repository-relevant words, applies stemming, retrieves through TF-IDF, and reranks the results. Kirschner reports that this approach often outperforms vector-only retrieval in their experience. He tentatively identifies the reranker as another GPT-3.5 call, while emphasizing ongoing experimentation.
The retrieved chunks are inspectable. For supported languages, custom semantic chunkers follow functions and code blocks rather than cutting solely by length. Kirschner identifies this structural chunking as a major source of improvement. The team’s language expertise helps it implement these boundaries; a preexisting online semantic index can also reduce retrieval work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Manage prompts and budget evaluation runs
A .prompty file in the editor prompts a discussion of Prompty, introduced at Build. It gives prompts an explicit artifact format: YAML metadata at the top and a Jinja template beneath it. Variables can be supplied to the template instead of scattering multiline prompt strings across application code. Fox names Promptflow and Azure AI Studio as users of the format and discusses LangChain integration as available or forthcoming. Her evaluation interface is a custom CLI and UI built on the Promptflow evaluation package, which also provides its own tooling.
The evaluation repository runs a smoke test against a target URL. Extending that to every application’s CI build requires reaching the correct deployed build and its Azure resources. Authentication and VNet isolation complicate access: the evaluator must work with the application’s real deployment topology. Fox has a working example, but describes a general solution across all repositories as unfinished.
Evaluation also has a different cost profile from ordinary unit tests. Each case can invoke the application’s model calls, followed by additional judge calls for groundedness and relevance. Those calls add latency and expense. Kirschner describes caching within the testing infrastructure: unchanged prompts and test inputs reuse previous responses, while changed inputs trigger another run. Fox notes that this is harder when an evaluation repository targets an application maintained elsewhere, because the remote application may change independently.
That test-response cache should be distinguished from both application answer caching and seeded generation. The OpenAI seed documentation describes best-effort reproducible sampling, not replaying a cached answer. Kirschner clarifies that his cache belongs to the testing infrastructure, not Copilot Chat itself. Repeated test inputs make reuse predictable; a user-facing RAG app may see fewer identical questions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate model compatibility from reliable output
An evaluator that calls a deployed application URL can remain independent of the underlying model provider. The starter templates themselves have a narrower integration contract: they are configured around OpenAI models and the OpenAI SDK. Fox considers the newly introduced Azure AI Inference SDK as a possible way to support additional hosted models, but has not tried it. The tradeoff is broader Azure model access versus the portability already available through the OpenAI SDK. Supporting hosted Mistral or Llama models would also require Bicep provisioning changes, not just swapping a client library. Requests in the issue tracker help prioritize that work.
Fox says the samples already work with Ollama, including local Phi-3 models. But being able to make a request is different from satisfying the application’s output contract. In her experiments with small local models, answers could synthesize information yet fail to follow instructions reliably. Kirschner responds that different models require different prompts. Fox’s local capacity limits her tests to models up to roughly 7B parameters; trying a 70B model is a future intention.
The visible failure is citation formatting. The frontend expects square-bracket citations so it can turn them into clickable source links. A fluent answer without that syntax breaks part of the product. Fox also reports more fabrication on off-topic questions in her local tests. Kirschner suggests narrower jobs for small models, such as reranking or judging, where they need not produce a full answer with strict citation formatting. Predictable output syntax and function-calling support still need to be assessed for the particular model and runtime.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use computation when the question spans the whole dataset
Retrieving a few passages is appropriate for answering from selected evidence. Counting or analyzing an entire database calls for a different operation. Fox describes a PyCon application in which a user uploads a CSV and asks about the top restaurants. The model generates pandas code, and the application executes it in a sandbox. SQL aggregate functions provide another route. The critical shift is that computation covers the dataset instead of asking the model to infer an aggregate from a few retrieved rows. Generated code must run within a safe execution boundary.
For a CSV where each row represents one restaurant observation and the restaurant name is in a restaurant column, the computation could be:
python
import pandas as pd
def top_restaurants(csv_path: str, limit: int = 10) -> pd.DataFrame:
observations = pd.read_csv(csv_path)
return (
observations["restaurant"]
.dropna()
.value_counts()
.head(limit)
.rename_axis("restaurant")
.reset_index(name="count")
)
This counts all applicable rows before selecting the top results. The surrounding application, rather than the generated pandas expression, must enforce sandboxing and permitted data access.
The final discussion returns to structured output. An attendee describes supplying a TypeScript definition for the desired answer and type-checking the response. Fox connects this approach to TypeChat. She describes experiments using TypeChat with local Phi-3 as an alternative to OpenAI function calling, but says those attempts struggled; further prompt work is proposed.
The exchange does not resolve Phi-3’s JSON response-format support. The attendee clarifies that the successful example of explicitly requesting JSON used GPT-3.5, not Phi-3. That leaves a concrete requirement for whichever model powers the application: verify that its output satisfies the format the consuming code expects. The workshop closes by inviting attendees to keep deploying with their remaining seven-day pass access—continuing from a working template into experiments with their own data, retrieval choices, and output constraints.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Proxy-specific workshop guides for a basic chat app, PostgreSQL RAG and document RAG.
A database-backed chat application with Azure deployment instructions.
Document RAG sample combining Azure AI Search retrieval with generated answers and citations.
The June 3 article explains hybrid retrieval and compares answer quality on questions about Pamela Fox’s blog.
Anthony Shaw explains multilingual tokenization and recursive splitting that respects token budgets and character boundaries.
Microsoft’s retrieval experiments comparing keyword search, vector search and semantic reranking.
Experiments on how the position of relevant information affects long-context question answering and retrieval.
Prompt asset format and tools for managing, debugging and evaluating prompts.
Further reading
- OpenAI seed parameter and reproducible outputsDocumentation
Archived guidance on seeded sampling, backend fingerprints and the limits of reproducibility.
Full transcript
Loading transcript…