AI Engineer World's Fair 2024
Running AI Application in Minutes: Quick Start with AI Templates
Loading slides…
Read the talk
Running AI Application in Minutes: Quick Start with AI Templates
Build and deploy a Python chat application, then extend the same workflow to product retrieval and document answers with citations, query rewriting and evaluated search.
From a talk by Gabriela de Queiroz, Aishwarya Srinivasan and Pamela Fox
Before you start: Basic familiarity with Python, environment variables and web applications is helpful; following the deployment steps also requires GitHub and Azure accounts.
How do you get an AI application running quickly?
How do you get from an AI application idea to something you can run and customize? This workshop starts with reusable application templates and the support around them. Gabriela de Queiroz introduces Microsoft for Startups Founders Hub as an entry point for founders with an idea, an incorporated company, or a business with or without funding. She describes signup as taking less than five minutes. Beyond benefits, the platform offers one-on-one guidance on technical questions, strategy and going to market. Its Build with AI area provides open-source material hosted on GitHub.
At the time of the workshop, Gabriela describes an offer of up to $150,000 in Azure credits, including Azure OpenAI usage. Azure OpenAI brings model access together with Azure’s security and compliance capabilities; other listed benefits include GitHub Enterprise, Microsoft 365 and LinkedIn Premium. Azure AI Studio also provides access to OpenAI and Llama models. These are program benefits subject to eligibility, rather than an automatic grant to every applicant.
Aishwarya Srinivasan emphasizes the people behind the platform. Startups can participate in private previews, try capabilities associated with models such as GPT-4 and GPT-3.5 Turbo, and send feedback to product teams. Expert matching covers infrastructure, Kubernetes, go-to-market questions and subscription pricing. The support is intended to follow a company from its initial idea through building and scaling.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Support beyond credits—and a shorter path to experimentation
Founders Hub also acts as a portal for tracking consumed credits, products and benefits, including third-party tools. Aishwarya names M12, Y Combinator, Neo, Alt Capital and Alchemist among Microsoft’s strategic VC partners. She describes a qualifying partner path that raises total credits to $250,000 and adds dedicated startup development and cloud architecture support. Pegasus adds go-to-market help and Azure Marketplace onboarding, while blogs, interviews and conference appearances help startups reach potential users. Gabriela’s concrete example is Nixtla and its Azure-integrated TimeGEN offering.
The engineering problem is the delay between wanting to experiment and having a complete application. Aishwarya describes startups waiting two weeks for a support ticket or spending weeks assembling their first end-to-end app from documentation. Templates aim to reduce that setup work, including for applications as involved as retrieval-augmented generation with Azure AI Search. The promise is a customizable starting point that can run within minutes, supported by GitHub issues and requests to the cloud advocate team. The hands-on portion begins with a shared setup document, reached through the slide’s short link or QR code, with the presenters available to help attendees through setup.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate deployment funding from model access
Pamela Fox’s setup starts with a signed-in GitHub account and two separate resources. The workshop Azure Pass provides up to $50 in credits and expires after seven days. It funds the Azure resources attendees deploy. The Azure OpenAI proxy provides access to the model service. At recording time, Pamela explains that direct Azure OpenAI access required an approval form; the workshop proxy avoids making attendees wait for that process. The pass and the proxy solve different prerequisites.
The first live interruption happens before provisioning: an attendee sees a blocked request when opening the setup document. Other attendees can open it, and the presenters suggest connectivity or an inaccurate short link as possible explanations without establishing a cause. Once the check-in page is open, redemption follows this sequence:
- Sign in with GitHub.
- Copy the Azure Pass promo code and choose Get on board with Azure.
- Confirm the Microsoft account that should receive the subscription.
- Enter the promo code and complete redemption.
Pamela recommends a personal Outlook account or a Microsoft account associated with a Gmail address because work accounts can impose restrictions. Her own redemption attempt returns an error because she has already redeemed the pass.
The subscription you select determines where charges go. Pamela’s portal contains both a paid subscription and a sponsorship subscription; the displayed $40 monthly spend belongs to her personal paid subscription, not to a measured cost of this template. Aishwarya warns attendees to use the sponsorship subscription ID in subsequent steps. A fresh Microsoft account can make that distinction easier by containing only the workshop sponsorship.
Account state creates several practical failures. One attendee is unexpectedly taken into an existing Minecraft-associated Microsoft account, prompting Pamela to recommend a separate browser to avoid inheriting the wrong login. Another encounters registration rejecting a number as an invalid business number; a helper later reports entering a phone number with permission. Address information is also required. University accounts are less clear-cut: one presenter has successfully redeemed a pass with one, but Pamela points out that tenant restrictions may still interfere with the templates.
To verify redemption, open the Azure portal and inspect Subscriptions → Azure Pass Sponsorship. An empty Resources view is normal before deployment: the subscription is the billing container, not an application. Next, sign in to the proxy with GitHub to obtain an endpoint and API key. Keep those settings and the sponsorship subscription ID available for configuration. Pamela treats keys as a temporary accommodation for the proxy; direct Azure OpenAI applications can use Managed Identity for keyless authentication.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with a shared development environment
The AI App Templates Workshop provides separate proxy-specific guides for a simple chat app, PostgreSQL RAG and Azure AI Search RAG. This article follows that historical proxy workflow: the matching workshop repository was archived in June 2026, and its presence does not mean the temporary proxy or credits remain available. Pamela introduces the document RAG template as the most established of the three and estimates that it has been deployed about 100,000 times.
Open the first repository in GitHub Codespaces. It supplies browser-based VS Code with the project loaded into a configurable development environment. Sharing that environment reduces differences in dependencies and local setup—the familiar problem of code working on one person’s machine but not another’s. The repository links are in the workshop document’s hands-on material; during the session, attendees are advised to skip a blank page or search for “hands on” to find them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run the chat app and ask for a haiku
The OpenAI Chat App Quickstart begins with local development. Here, “local” means the application server runs inside the Codespace; it still calls a remote model. Pamela’s sequence is to make that server work before deploying it to Azure. Create a .env file from the provided sample, then fill in the endpoint, key and deployment name. Codespaces reduces environment setup, but it also depends on connectivity; running the project on a local machine remains an option.
The proxy endpoint ends in /api/v1, and the OpenAI SDK sends requests there. The key authenticates those requests. The final setting is an Azure deployment name: in this workflow, Azure OpenAI uses a named deployment of a model, whereas a direct OpenAI request commonly specifies a model name such as GPT-4o. The workshop happens to name its deployment gpt-35-turbo, matching the underlying model designation. That coincidence does not make model names and deployment names interchangeable.
Start the Python backend using the repository’s instructions and open its forwarded port. Codespaces exposes the server through a generated browser URL. Pamela tests it with “Write a haiku about AI Engineer World Fair,” then requests a haiku about San Francisco. A short prompt with a short expected answer is useful here: it verifies the path from browser to backend to model and back without waiting through a lengthy generation. Both requests return responses in the demonstration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use async I/O and stream the response
The quickstart is a Quart application. Pamela introduces Quart through its Flask-like interface and support for asynchronous request handling; the Python samples use Quart or FastAPI. LLM requests spend time waiting on a remote service, so async and await let a backend handle other work while a request is in flight. The application then forwards the OpenAI client’s streamed output to the frontend as JSON Lines, also called newline-delimited JSON. Each line is a complete JSON value that the frontend can process incrementally.
The core Python operation can be expressed as an asynchronous generator that accepts a configured client and yields one serialized record per received chunk:
python
import json
async def stream_reply(client, deployment, messages):
stream = await client.chat.completions.create(
model=deployment,
messages=messages,
stream=True,
)
async for chunk in stream:
if not chunk.choices:
continue
text = chunk.choices[0].delta.content
if text:
yield json.dumps({"text": text}) + "\n"
Streaming does not remove the wait for the first token. It lets the interface begin displaying output once generation starts, instead of withholding the entire answer until completion. Pamela requests a long essay about San Francisco to make that behavior visible, but the response arrives too abruptly to establish token-by-token delivery. She suspects the proxy may affect it and cannot confirm the expected stream in the browser inspection.
A locally running application still calls Azure in this setup. Pamela also adds Ollama support to samples where possible, allowing local development against smaller models such as Phi-3 or Llama 2. That changes the model as well as its location, so the resulting answers can differ from the Azure-hosted model’s behavior.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Deploy through Azure Developer CLI and Bicep
Deployment begins with azd auth login. Inside Codespaces, Pamela uses the device-code flow: copy the code, complete browser sign-in, and authenticate as the account that holds the Azure Pass. Next, create a named Azure Developer CLI environment. These environments keep deployment configurations separate; Pamela describes maintaining roughly 20 while trying different sample configurations.
Local .env settings and AZD deployment variables serve different consumers. The local file configures the development server. AZD variables configure provisioning and the deployed application. Before running azd up:
- Disable creation of a new Azure OpenAI resource, because the app will use the workshop proxy.
- Set the model deployment name.
- Supply the proxy API key.
- Supply the proxy endpoint.
The repeated values are intentional: configuring the local server does not automatically configure the infrastructure deployment.
azd up works from infrastructure as code. The repository’s Bicep files describe resources and roles, including model-service resources where enabled, analytics and Container Apps. Pamela compares Bicep with Azure’s JSON-based ARM templates and names Terraform as another way to describe infrastructure. During deployment, select the sponsorship subscription and a hosting region.
The packaging step builds a Docker image. Its Dockerfile installs the application requirements and defines how to run the server. The first two templates deploy to Azure Container Apps; App Service, Functions and Kubernetes are other Azure hosting options. While provisioning runs, Pamela opens a previously deployed instance to demonstrate the shareable Container Apps URL. She then inspects the Azure resource group, which contains the container app and its registry. This separates two useful checks: the application responds at its public address, and the portal shows the infrastructure supporting it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retrieve product rows before generating an answer
A general chat model can produce plausible text about a domain without knowing the facts of a particular business. Retrieval-augmented generation, or RAG, gives it relevant source material at answer time:
- Receive the user’s question.
- Search a database or search index.
- Put the retrieved results and original question into the model prompt.
- Ask the model to answer from those sources.
The quality of the retrieved evidence matters because the model synthesizes what it receives. A fluent answer cannot repair missing or irrelevant source material.
The RAG on PostgreSQL template applies that flow to an existing product table. Think of a retailer whose customers ask which products meet their needs. The database returns matching rows, and those rows become the source material for the answer. Pamela demonstrates an outdoor footwear catalog; the application’s “thought process” view exposes the search query, retrieved rows and answer prompt. It is a diagnostic view of the application’s retrieval steps.
The retrieval step combines pgvector similarity search with PostgreSQL’s built-in full-text search. Vector retrieval searches through embeddings; full-text retrieval searches the words in the indexed content. Pamela recommends combining them and cites hundreds of evaluations behind her concern about vector-only retrieval. The final model call is straightforward: instruct it to answer from sources, then provide the question and matching product rows. The later Q&A returns to why the way those results are combined matters.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn conversational requests into search constraints
The advanced product flow first asks an LLM to rewrite the user’s question into a better search query. Conversational wording is not necessarily useful retrieval wording. In the first example, the rewrite returns the same query. The need becomes clearer in a follow-up such as “more options”: a search engine needs to know options for what, which requires context from the conversation. Pamela tries that follow-up, but the live PostgreSQL example does not establish the intended contextual rewriting behavior.
Query preparation can also produce structured constraints. For “I want climbing gear cheaper than thirty dollars,” the application uses OpenAI function calling to ask whether a price filter is appropriate. The model returns a less-than condition and a value of 30. The application constructs the SQL filter from that structure; it does not need to accept arbitrary SQL written by the model.
For the same climbing-gear request, a constrained Python filter builder can keep operators under application control and pass the price separately as a database parameter:
python
from decimal import Decimal
def build_price_filter(condition):
operators = {"less_than": "<"}
operator = operators[condition["operator"]]
price = Decimal(str(condition["value"]))
if not price.is_finite() or price < 0:
raise ValueError("Invalid price")
return f"price {operator} %s", (price,)
condition = {"operator": "less_than", "value": 30}
where_sql, parameters = build_price_filter(condition)
# where_sql: "price < %s"
# parameters: (Decimal("30"),)
This produces a filter ready to combine with the product search; it does not execute a query. The model identifies the user’s constraint, while the application owns the allowed SQL structure and parameter binding.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make documents searchable and answers inspectable
The Azure AI Search RAG sample extends the pattern to unstructured documents: PDFs, office documents, spreadsheets and HTML. Pamela asks an employee-handbook application, “What does a product manager do?” The answer includes citations. Clicking one opens the source PDF at the relevant page, so the reader can inspect the document behind the generated answer.
The query-time flow is still query rewriting, search and an answer prompt, all visible in the diagnostic view. The added complexity is data ingestion: a long document must become searchable pieces before a question arrives. Pamela describes chunks of about 500 tokens in the demonstrated ingestion pipeline.
| Stage | Operation | Service |
|---|---|---|
| Extract | Read document text | Azure Document Intelligence |
| Chunk | Split text into token-based pieces | Ingestion code |
| Embed | Vectorize each chunk | Azure OpenAI |
| Index | Store searchable chunks and vectors | Azure AI Search |
| Preserve | Store source documents | Azure Storage |
This preparation creates the units retrieval can return while retaining source documents that readers can inspect through citations.
The same architecture is not limited to PDFs. Pamela opens an instance built from her HTML blog and asks about good sleep strategies; the response draws on her posts. Replacing the supplied documents with your own content is the next customization step for this template.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Customize the app, then inspect what ingestion actually extracts
The presenters promise to add the slides to the same workshop document. Gabriela then points to three customization surfaces: the chat’s HTML, its system message and its source data. Her prompt example is a Nintendo-only assistant that responds “Whoa-ho” to other topics. Pamela’s blog application illustrates the data change: the interaction pattern stays familiar while the evidence corpus changes.
The Q&A turns from interface changes to ingestion. Pamela lists PDF, HTML, DOCX, PPTX, XLS and images among the formats handled through Document Intelligence in the demonstrated setup, with additional repository parsers for text. Two operations involving images need to be distinguished:
- OCR ingestion: Document Intelligence extracts text visible in an image so that text can enter the retrieval corpus.
- Vision question answering: A vision-capable model such as GPT-4o receives an image and answers questions about it. Pamela describes this as an optional, separate path supported by the repository.
Choosing between them depends on whether the useful information is the image’s text, its visual content, or both.
Extraction can also admit unwanted evidence. Asked about images inside PDFs, Pamela explains that Document Intelligence tries to extract as much text as possible. In a RAG application built from Python Playwright documentation, it extracted JavaScript from screenshots of Node Playwright examples. That text then interfered with the Python-focused answers. More extracted content can mean worse retrieval evidence when the added material belongs to the wrong language or context.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluate the retrieval combination, not just the vector index
Additional metadata enters text and vector retrieval differently. Mark an index field as searchable and full-text retrieval can search it; Pamela recalls the sample starting with three searchable fields. Vector retrieval only represents the content supplied to the embedding model. To include metadata in that representation, the application can combine it with the main text before embedding—a technique Pamela calls content stuffing or content expansion. Whether that helps requires evaluation.
Pamela opens Doing RAG? Vector search is not enough, her experiment using synthetic questions and GPT-4-generated ground truth about her blog. Groundedness was scored out of five:
| Retrieval configuration | Groundedness |
|---|---|
| Vector only | 2.79 |
| Text only | 4.87 |
| Hybrid | 3.26 |
| Hybrid with semantic ranker | 4.89 |
The 0.02 improvement over text-only search came from hybrid retrieval with the semantic ranker, not from simply adding vectors. The author flags possible lexical bias in the generated evaluation material, and the supplied account does not establish the full run configuration or statistical uncertainty. These conditions matter when interpreting the table.
Why can combining two retrieval methods make results worse? A nearest-neighbor vector search can return the closest available matches even when none is particularly relevant. Combining those candidates with text results through basic reciprocal rank fusion can introduce distracting sources into the answer prompt. Pamela warns that LLMs are susceptible to that distraction. Azure AI Search’s semantic ranker adds a learned reranking step against the query; in her experience, that step is what lets vector-inclusive retrieval match or improve on full-text search. The engineering task is to evaluate the complete retrieval pipeline, including how results are combined and reranked.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Confirm a working result and stop unused environments
The workshop does not end with every attendee verified as successful. Some still have key-related problems, and one reports that an application appeared to run without producing a working result. The presenters offer individual help, ask for feedback on the dense session and invite startup use cases that might fit other templates beyond the three shown. The practical checkpoint remains a working application, not merely a server process reporting that it started.
Before leaving the development environment, inspect your Codespaces list and stop or delete unused instances. Gabriela has noticed attendees with several open at once. The speakers describe a monthly allowance of 60 free hours; GitHub’s current Free-plan explanation specifies 120 core-hours, equivalent to 60 runtime hours on a two-core machine, so machine size matters. They also recommend automatic inactivity shutdown. Configure that through the documented Codespaces idle-timeout setting, whose current range is 5–240 minutes; the recording’s thirty-hour example is not a supported current setting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the document index current
Pamela says production adoption prompted the team to harden what began as a sample. The uses include public government websites that help citizens navigate documents, internal HR information, meeting transcripts that let employees ask what the CEO said, and sales training manuals. These applications introduce an ongoing requirement: answers must reflect new and changed documents.
The demonstrated ingestion process is manual. In response to a question about automatic updates, Pamela describes two alternatives:
- Scheduled indexing: Azure AI Search integrated vectorization can use an indexer pointed at blob storage. Pamela gives a five-minute refresh schedule as an example for updating the index from changing source content.
- Triggered ingestion: An Azure Function can run an ingestion workflow when a trigger fires. Pamela points to the Chat with Your Data Solution Accelerator as the example of this approach.
The choice depends on how documents arrive and how quickly changes need to become searchable. Once the application can answer from a corpus, keeping that corpus current becomes part of operating the application.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Simple chat application with Azure Container Apps deployment and managed identity authentication.
Product-table chat using PostgreSQL hybrid retrieval, FastAPI and structured query filters.
Deployable document-chat sample using Azure AI Search retrieval and Azure OpenAI generation.
Pamela Fox's retrieval experiment comparing text, vector, hybrid and semantically reranked search on synthetic questions about her blog.
Azure solution accelerator for deploying chat over organizational data.
Further reading
- AI App Templates WorkshopRepository
Archived proxy-based workshop guides covering simple chat, PostgreSQL RAG and Azure AI Search RAG.
The speaker's explanation of product-table retrieval, query rewriting and constrained SQL filters.
Historical announcement of startup AI templates, expert support and up to $150,000 in Azure credits.
Updates since the talk
Instructions for setting automatic shutdown after inactivity, with a supported range of 5–240 minutes.
Full transcript
Loading transcript…