AI Engineer World's Fair 2026
Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI
Slide chapters
25 slide and demo groups · Timestamps are approximate.
Read the talk
Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI
Jetashree Ravi explains how Fireworks combines workload-specific inference, several levels of training control and checkpoint deployment to help teams specialize open models—and keep updating them with production feedback.
From a talk by Jetashree Ravi
At a glance
Ideas worth remembering
Quality tuning and serving optimization address different problems: the Genspark example changes data and evaluation for RFT, while FireAttention targets workload-specific execution.
Choose the training interface by the control required: delegate setup to the agent, configure a managed run or define custom loops and losses through the training API. The agent does not support RFT in this presentation.
Hot loading addresses deployment downtime when replacing weights; numerical fidelity addresses quality changes caused by differences between training and serving numerics.
The demonstrations retain inspectable decisions: approve the agent’s proposed plan, examine completed-job settings and loss, clone a run with changes and choose a serving configuration for deployment.
A cheaper model still has to do the job
The opening question is familiar to anyone paying for model APIs: how quickly is the bill growing? Jetashree Ravi, who leads part of Fireworks AI’s applied machine learning team, starts with teams considering a move from closed models to open models for cost or performance reasons. The difficult part is keeping the quality users already expect. A cheaper response is useful only if it still completes the task.
Ravi builds the talk around three premises: open models are already production-ready for many workloads, customization can close remaining task-specific quality gaps, and deployed models increasingly need repeated training rather than a single release. These are the premises behind Fireworks’ combined training and serving product. The intended unit of work becomes a continuing sequence of checkpoints: train a model, serve it, learn from use and train it again.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Speed and quality are separate migration problems
Fireworks approaches this as a production infrastructure problem. Ravi reports serving about 30 trillion tokens, supporting more than 10,000 companies and processing about 200,000 requests per second. The token total has no clear reporting interval in the recording, so it is best read as a scale claim rather than a comparable throughput measure.
The customer examples distinguish what teams need to improve:
- Cursor: inference speed. Composer 2 was trained and hosted on Fireworks. Ravi attributes a reported 13× inference speedup to the FireAttention engine.
- Vercel: serving latency. Fireworks helped fine-tune and host its model, with a reported 40× latency improvement.
- Genspark: task quality. Quality was a bottleneck when moving from closed to open models for a product that lets users design systems through prompts. Fireworks cleaned the dataset, helped define evaluator functions and ran reinforcement fine-tuning, reporting a 12% quality improvement.
These are Ravi’s reported customer outcomes, with no comparison baselines, workload specifications or quality-scoring protocol supplied. In particular, the 12% figure does not establish whether the change is relative or measured in percentage points. The useful technical distinction is the work behind each result: serving optimization addresses execution speed, while Genspark’s intervention changes the training data and the evaluator used to guide learning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Optimize the inference stack for the traffic it will receive
FireAttention is Fireworks’ inference engine. Ravi explains its role through the path between a model and the GPUs executing it: CUDA kernels implement the GPU work, and Fireworks builds its own kernels as part of the engine. That gives the team a place to optimize execution for a customer’s workload, alongside changes elsewhere in the serving stack.
The input to this optimization is more concrete than “make the model fast.” A customer supplies input-token lengths, output-token lengths, a prompt-cache hit rate and a latency target—Ravi’s shorthand is X, Y and Z. Those properties describe the traffic the system must handle. Fireworks then tunes the stack, including kernels, toward the requested latency and a lower serving price. The optimization is tied to that traffic profile; the talk does not present a universal configuration that delivers the same gain for every workload.
The engine also uses speculative decoding, disaggregation and quantization. The concrete extension described here is training a custom draft model for speculative decoding: customers upload a dataset, much as they would for fine-tuning, and train the draft model on the platform. Ravi links a well-trained draft model to faster main-model inference, better throughput and meeting latency requirements. The recording does not develop the decoding algorithm or quantify the draft model’s contribution separately.
There is a practical ordering to these decisions. Customers generally reach this stage once they are satisfied with model quality. If an open model still gives inadequate answers, faster execution leaves the main migration problem unresolved. That moves the discussion from serving optimization to training.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three interfaces divide the work differently
Fireworks offers three ways to train, distinguished by how much of the training process the user wants to specify. Choosing among them means deciding whether to delegate configuration, supply it directly or write the learning procedure.
- Training agent: delegate setup. Through the UI, a user points the agent to an uploaded dataset and describes the goal. The agent can help select a model, clean data and choose hyperparameters. Ravi positions it for product managers and builders who are newer to training.
- Managed training: supply the configuration. Through an API or UI, the user chooses a model, uploads data and sets hyperparameters, then starts a job. The service handles execution. Ravi describes this as the option most customers use.
- Training API: define the learning procedure. ML engineers can write their own training loop and loss functions when the managed service does not provide the control their task requires.
Managed training also makes the running job inspectable. Its dashboard shows progress across epochs and the training loss, so users can examine how learning proceeds after launch. Teams already using Weights & Biases can supply an API key and export some observability metrics there.
The training ecosystem includes supervised fine-tuning, reinforcement fine-tuning and DPO, but the interfaces do not all support the same methods. The important exception is the training agent: it does not support RFT in this presentation. Its automated dataset cleanup and hyperparameter assistance therefore do not extend to RFT. For that work, Ravi describes direct help with evaluator design, dataset cleaning and hyperparameter selection—the same pieces involved in the Genspark example.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the checkpoint moving from training into production
The continuing train-to-deploy process begins with a trained checkpoint that can be deployed from the same platform. Once it is serving traffic, monitoring exposes token counts, cache hit rates and the behavior of a trained speculator. Those metrics and user feedback can inform another training run, continuing from the existing checkpoint to a new one. Retraining remains a choice; the description does not imply that every observed metric automatically updates the weights.
Compute availability supports both sides of this process. Fireworks partners with cloud and neo-cloud providers to obtain GPUs for training and hosting at scale. Two other capabilities address the handoff between checkpoints: hot loading changes the weights in a live deployment, while numerical fidelity aims to preserve the behavior obtained during training.
Hot loading was initially implemented for Cursor and then offered more broadly. When a new checkpoint is ready, the user can swap the deployed weights for the new weights without downtime, as Ravi describes it. This makes an existing deployment the destination for successive training results. The talk explains the user-visible operation, without detailing the internal weight-loading or request-handling mechanism.
Numerical fidelity addresses a different failure: a model trained on platform A can lose quality when served on platform B because the numerical behavior differs. A deployment can therefore succeed operationally while failing to preserve the trained model’s quality. Fireworks’ stated promise is to align training and inference numerics on its platform so this handoff introduces no numerical differences. No numerical tolerance or validation procedure is given, so the promise should not be read as a demonstrated guarantee across arbitrary external stacks.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The training agent proposes a plan before running it
The first demonstration starts with two inputs: the location or name of the training dataset inside Fireworks, and a separate evaluation dataset. The user asks the agent to fine-tune a model and choose both the base model and hyperparameters. This delegates the setup decisions while still giving the run distinct data for training and evaluation.
What can the user inspect before the job starts? The agent returns an SFT plan, explains its base-model selection and describes the proposed hyperparameters and high-level training process. The user approves that proposal before execution. The consequential transition is from a requested goal to a concrete training plan, followed by an explicit decision to run it.
After approval, the agent reports progress through the steps. At completion, it summarizes the configuration, base model and hyperparameters, then returns evaluation metrics such as accuracy and F1 score, along with recommended next steps. These are the described outputs of the demonstration; the recording supplies no metric values from which to judge this particular run’s quality gain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Managed training exposes the configuration and the completed run
The managed-training demonstration makes the user responsible for the choices the agent previously proposed. After uploading a dataset and choosing a model, the user selects an evaluation dataset in the evaluator settings. Optional settings allow finer control over hyperparameters. Clicking Create starts the training job.
What remains available to inspect after training? Opening the completed job reveals the dataset, hyperparameters and loss during training. A run can also be cloned with different settings. That connects a recorded configuration to the next experiment: retain the earlier setup, change the desired settings and launch another run rather than reconstructing it from memory.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Deployment still requires a serving configuration
How does the trained model become a serving deployment? In the Deployment section, the user selects the newly trained custom model, chooses deployment settings and clicks Deploy. Available templates include throughput, fast and minimal. Ravi describes them as configurations optimized for generic workloads using throughput, latency, time to first token and related metrics. The choice matters because training produces the model, while deployment settings determine how it is served.
The ending leaves the most customizable route open. There is no complete training-API demonstration because its setup varies by customer; Ravi points engineers toward cookbooks for custom loops and loss functions. The demonstrated path is narrower and concrete: configure or delegate a training run, inspect its results, select the resulting custom model and deploy it with an appropriate serving configuration. The earlier hot-loading capability then provides a way to bring subsequent checkpoints into the live deployment.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Related talks
- Beating RL With Reflection: GEPA and Optimize Anything
Explores a complementary route to specialization: using execution traces and evaluator feedback to improve prompts, agent programs and skills, with an ending that considers combining those changes with weight training.
Read the complete timestamped transcript
6 slide groups · 19 demo milestones. Extracted from the recording; expand builds to follow changes. Timestamps are approximate.
- 0:12
Hi, everyone. Uh, before we get started, I actually have a quick question for all of you. How many of you actually today used a closed source model like ChatGPT or Anthropic? Okay. How many of you feel like it's getting really expensive really quickly?
- 0:31
Okay. That's, that's a good percentage of you all. And, uh, just like all of you might be thinking of switching from closed source model to open source models for cost reasons, performance reasons, there are, uh, other customers and users that are going through the same journey. And today, I'm going to talk about where these open source models shine and how to make the performance or quality of these open source models match the closed source models
- 1:01
itself. And before I get into that, quick introduction of myself. Uh, I work at Fireworks. I lead part of the applied machine learning team. My name is Jetashree, but feel free to call me Jat if that's easier. Um, let's talk about the whole train-to-deploy loop that, that the open source models are taking today when users are switching from closed source to open source models. But even before we get into that, let's talk about some fundamental truths that are true today
Stop Renting Intelligence
Fireworks AI introduces the train-to-deploy loop for users moving from closed-source to open-source models, with a focus on matching closed-source performance or quality.
Text in image
Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI Fireworks AI
- 1:31
in the AI ecosystem. So the th-three fundamental truths that are true, uh, are one, open source models, for the most part, are production ready. If any of you have used GLM 5 point-- 5.2, you know that that is true, especially for the coding use cases. Two, if open source models don't, like, beat or match the quality of closed source models, the differentiating factor, uh, comes from how you customize, train, and improve these open source models to
- 2:00
match the performance and quality of closed source models. And three, the AI stack itself for most of our customers and users is shifting from, "Let's just sell a model once it-- once and that's done" to a more continuous loop that is improving from training to inferencing, uh, the same open source model itself. So that's where Fireworks kind of comes in. It makes the whole process of you having to go through the continuous
From Foundational Models to Owned Intelligence
Fireworks argues that production-ready open models shift differentiation toward customization and continuous improvement, with inference and training brought together in one system.
Text in image
From Foundational Models to Owned Intelligence 3 Fundamental Truths in AI Today - Open models are now production-ready - Differentiation comes from how you customize and improve them - The AI stack is shifting from serving models → continuously improving them Fireworks AI World-class Inference and Training. In one system.
- 2:30
journey of training these open source models, as well as hosting these open source models at scale much more easier and also much more cost effective as well. And I'll get into how we actually do the training itself and different flavors of training that we offer towards the end of the presentation. But I did also wanna quickly talk about inference before we go to the training portion of it, which has become really popular among our customers and user base. And before we also talk about inference and training, I just wanted to give you a quick overview
Production-Grade Inference and Training
Fireworks reports 30T tokens served per day and customer results including Cursor’s 13× faster inference, Vercel’s 40× faster latency, and Genspark’s 12% quality lift versus closed models.
Text in image
Production-Grade Inference and Training Purpose-built for open-source models; Turns Your Data Into Your Edge Scale in Production 30T tokens served / day 10,000+ companies globally across industries 286K+ requests / second Production Impact CURSOR 13× Faster Inference Powers Composer 2. Vercel 40× Faster Latency 93% error-free. Genspark 12% Quality lift vs closed models Trusted platform Built by founding team from PyTorch (Meta) + Vertex AI (Google Cloud) $327M Raised Series C, Oct 2025 BACKED BY SEQUOIA BENCHMARK Index Ventures Lightspeed
- 3:00
of how, uh, inference looks like, especially at scales, and what are the kind of production impact that we have seen among some of our customers and user base itself. So today at-- in production at scale, uh, we serve about thirty trillion tokens, and it's increasing. Um, this number was actually pre-GLM 5.2 release. I'm sure this has pretty much kind of increased by about at least five trillion tokens a day now with the GLM 5.2 release.
- 3:30
We support more than ten thousand companies globally and process about two hundred K requests a particular second itself. And the kind of production impact that we are seeing within our customer and user base is very significant. So Cursor, uh, if-- for those of you who already use Cursor and Cursor models, uh, the Composer 2 model for Cursor was trained and also hosted on our platform today. And, um, it is able to achieve
- 4:00
thirteen X faster inference because of the FireAttention engine that we have. And I'll speak a little bit about that in the next slide as well. Uh, Vercel is also another big customers, uh, of ours. We help them actually fine-tune and also host the model on our platform, and they see a forty X latency improvement. And another u-big use case study is actually Genspark. Um, Genspark lovables are in very similar field where they actually let you design systems with just
- 4:30
prompts itself. And, uh, quality is a big bottleneck or was a big bottleneck for them when they were trying to switch from closed source to open source models, and we helped them fine-tune. We did, uh, RFT for them. We helped them clean the dataset, define evaluator functions, and at the end of the RFT training job itself, we saw that there was twelve percent quality improvement for the Genspark use case. Um, also, we are a, a trusted platform among a-all our other
- 5:00
customers and user base as well, and we have raised, uh, around three twenty-seven million CDC, uh, a while ago, and we are about two hundred people working at Fireworks. That brings me to the next portion of it. I know in the previous slide I did mention that we made latency faster for a couple of our customers, such as Vercel. And how we do it is quite simple a-at a, at a higher level. Um, at Fireworks, we have our own inference engine called FireAttention.
Fireworks Optimizer
FireOptimizer uses optimization goals and a production workload profile to configure inference across 84,000+ possible combinations, using techniques including speculative decoding, caching, quantization, and a custom CUDA kernel.
Text in image
Fireworks Optimizer Transforms generic models into tailored enterprise solutions with blazing fast speed personalized to your workload The optimization space for LLM inference is complex Chart axes: quality, speed, cost Chart tick labels: [illegible] 84,000+ possible combinations 3 speculative decoding modes 5 quantization modes 7 hardware SKUs 4 parallel execution modes 5 cross-host setups 10 kernel options 4 quality tuning approaches FireOptimizer turns an intractable manual optimization problem into a workload-specific solution. Optimization goals Production Workload and Profile Speculative Decoding Adaptive caching Quantization FireAttention: custom CUDA kernel Personalized configuration for faster,higher-quality Inference
- 5:30
So you have a model, you have GPUs, and the way the model sits on top of the GPUs is through a translation layer called CUDA kernels. And as part of FireAttention, we build our own CUDA kernel. So when a customer comes to us, uh, uh, with a workload and a performance target in mind, then we can optimize this particular inference engine and the whole AI stack, including that CUDA kernel for that specific use case. So most of the time, customer like Vercel in the previous slide, come to us and tell us, uh, they have X
- 5:59
input tokens, Y output tokens, uh, they have Z prompt to-- uh, prompt cache hit rate, and they wanna meet so and so latency targets. And we help them optimize this whole inference stack, including that CUDA kernel, so that we can serve this model, uh, at a cheaper price, but also at a-- uh, that can meet their performance target and for their specific workload itself. Uh, as part of FireAttention and inference optimization, we do also implement the very well-known
- 6:29
techniques in the AI field, which is things like speculative decoding, disaggregation, quantization. In terms of speculative decoding itself, we also allow customers to train their draft models on our platform. Uh, so they can even upload the dataset, very similar to how they do for fine-tuning, for, uh, training a draft model, uh, which helps them, uh, make the model, the main model, uh, perform much faster, uh, meet latency requirements, also gives you
- 6:59
better throughput if the draft model is trained, uh, very well. And, um, so once people actually-- people actually get to this stage mainly once they're happy with model quality. But when customers are switching from closed source to open source models, the biggest bottleneck or the gap is quality. And the way we, we try to solve for it is through our training product. And our training product itself comes in three different flavors for different
Three tiers of training
Fireworks offers three training tiers: an agent that handles data prep, managed training with a chosen method, and an API for custom training loops and loss functions. Control increases across the tiers.
Text in image
Leverage your data. Own your AI Three tiers of training on the same platform you serve from. Start anywhere, scale as you grow. Training Agent FOR Product managers & builders YOU BRING Data + a description METHODS SFT · Classification · DPO Zero ML expertise required. The agent handles data prep, evals, training, and deployment. Managed Training FOR ML engineers YOU BRING Formatted data + chosen method METHODS SFT · DPO · RFT Standard fine-tuning. Reliable, scalable, one-click deploy. Out of the box smart defaults Training API FOR Advanced platform teams YOU BRING Your training loop + loss METHODS SFT · DPO · GRPO · custom Full control. LoRA and Full Param, 265K+ context. Custom loss functions. Recipes or raw primitives. LOW CONTROL → HIGH CONTROL
- 7:29
segments, uh, within our customer and user base, one being the training agent. This is very popular among our product managers and builders that get in-- that got into this AI inferencing and training field very recently. So the agent can be accessed through our UI, and, uh, you can-- you, you just need to link it the dataset that you have uploaded, uh, on the Fireworks platform, as well as description on what you're trying to do as part of
- 7:59
this training run itself. Uh, it will even let you choose the model. It will clean the dataset for you. It will also help you do the hyperparameter, uh, fine-tuning itself, 'cause it's not always easy to get the right, uh, hyperparameters itself, um, when you are doing training for your specific workload. The next flavor, al-also one of our most popular flavors among our customer and user base, is our managed training, and this is what most of our customers use today.
- 8:29
They can use our managed training platform, one, through API and second through the UI itself. It is as simple as you inputting the hyperparameters that you think will be good for your model, choosing a model, uploading the dataset itself, and then clicking a button. That's how simple we make it on, uh, Fireworks for the managed training itself. And, uh, I know one challenge a lot of people have is observability once the training run itself kicks in. So we
- 8:59
also have a good UI dashboard that shows you how the training, uh, is happening, how well your model is, uh, learning at each epoch, what does the loss function look like. Additionally, you can also plug in, uh, your Weights & Biases API key into the managed training service itself, uh, and export some of the observability metrics to platforms like Weights & Biases if you're already using that. And the third and the kind of the most advanced flavor of training that we offer is the
- 9:29
training API itself. And the training API is very popular among some, uh, of the ML engineers within, uh, different organizations or within our customer base. And in this, in this particular-- this particular offering of the product lets, uh, ML engineers define their own training loop and loss functions. And this becomes important when you want to customize your training job according to your needs that is-- that goes well, uh,
- 9:59
above, beyond the managed training service that we offer itself. And these, these different, uh, flavors or the products within the training ecosystem that we offered are offered for, uh, supervised fine-tuning, reinforcement fine-tuning, and even DPO as well. Um, nowadays, I think RFT has become extremely, extremely popular. And if you ever need help in helping define a good evaluator function, then we have worked with customers like
- 10:29
Genspark, that I was talking to you about in the previous slide, to help them define an evaluator function, clean the dataset, um, and also choose the right hyperparameters, especially for RFT. Unfortunately, the training agent itself does not support RFT, so it does not, uh, help you clean the dataset or even do the hyperparameter tweaking. But we are more than happy to help you do it if you have a big use case in mind. So now that we know all about the different, uh, training
- 10:59
offerings that we have, I also wanted to show you a quick demo of each of these, the training agent, managed training, and the training API itself. Uh, but before we get into that, I just wanted to close off by saying how the data flywheel looks like today on our platform. I know in the first slide, I was talking to you about the realities of AI today, where most people prefer doing a continuous loop, where they train, host, train, host as they get new and new data and feedback from their users. And that is
The Data Flywheel
Fireworks connects training, deployment, monitoring, feedback collection and retraining in a loop. Hot loading lets a new checkpoint replace deployed weights without downtime.
Text in image
The Data Flywheel The Unified Platform Advantage: Train → Deploy → Improve 1 TRAIN SFT / DPO / GRPO 2 DEPLOY One API call to auto-scaling endpoint 3 MONITOR Track latency, accuracy, cost 4 COLLECT Gather feedback, failures, edge cases 5 RETRAIN Trigger new run with fresh data Each iteration produces a smarter model On-demand GPUs We provide the compute. No GPU scavenging, no queuing, no idle cluster costs. Spin up and down in minutes. Hot loading in seconds New checkpoint → live deployment in seconds via WeightSyncer. Test 10 model versions in the time it takes to provision a cluster DIY. Numerical fidelity CUDA-level correctness guarantees. We validate that performance optimizations don't corrupt your model. What you train is what you deploy. No vendor sprawl Training and inference on the same platform. No data export, no model migration, no format conversion.
- 11:29
what we are trying to make more seamless on our platform, and that's also why we have different training products, different offerings, as well as making it easier, uh, for you to host and make it also cheaper and faster for you to host those models at scale itself. And the way this flywheel works is kind of very simple. Once these, uh, models are trained, I will show you, uh, a demo in the next slide where the models get trained. It's, it's-- you just are one click away from deploying these models into
- 11:59
production. And once you've deployed, you can, uh, monitor. We have, uh- Monitoring metrics for the number of tokens, cache hit rate, how your speculator is working if you've trained a speculator. Uh, we help you collect gathered, uh, feedback from all the, all the metrics that you have collected, and if you choose to also retrain on the same platform from the end-- from the checkpoint that you were using to a new checkpoint itself. So some of... some capabilities on the platform
- 12:29
that makes these data flywheel possible is, one, definitely having a host of, uh, GPUs. We do have-- We do partner with different cloud and neo-cloud providers, which makes it easier for us to provide compute and also host these models at scale. Another thing i- we offer is hot loading, and this has become really popular, especially when we were training models for Cursor, and this was actually a feature we implemented specifically for Cursor, and then we thought would be useful for general public while they were training
- 12:58
models as well. Uh, what hot loading essentially means is when you have a new checkpoint, uh, you, you can directly go to live deployment without any downtime itself. So if you have a deployment and you've gone through this continuous loop and trained a new checkpoint, then you can just click a button and swap out the weights itself that you were using, uh, uh, previously with the new endpoints. Another thing that makes it, uh, easier to train on our platform is the numerical fidelity. So a lot
- 13:28
of the time customers use platform A for training and host on platform B. But what happens when you do that is you'll have issues in quality of the model, and the reason you have those issues is numerics is not always compatible when you train on a different platform and host on a different platform. So that's why we offer training and inferencing on the same platform and ensure that, uh, you won't find any numerical differences after you finish training and once you decide to host this particular model on the platform
- 13:58
itself. And, um, one other thing that, that makes it more seamless is because training and inference is on the same platform, you're just one click away from, uh, deploying this model at production scale. Now that we've learned all about, uh, the inference and training, I just wanted to show you a quick demo of different training products and how they work on our platform. The first portion of the demo is showing you how you can use the training agent itself. Over here, you
Training agent interface
The training agent opens with suggested tasks and a dataset attachment. The speaker introduces giving it training and evaluation datasets and asking it to choose a model and hyperparameters.
Text in image
What can I do for you? BETA Try one of these Classification fine-tuning Supervised fine-tuning DPO Fine-tuning Build custom evaluators + Dataset Interface text: Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Pause 7 CC
14:19Demo milestone
Training agent fine-tuning demo
The demo asks the training agent to inspect separate training and evaluation datasets, select a model, run hyperparameter sweeps, and deploy it. The agent checks the datasets and begins building a plan.
Text in image
What can I do for you? BETA Try one of these Classification fine-tuning Supervised fine-tuning DPO Fine-tuning Build custom evaluators First, inspect the training dataset accounts/pyroworks/datasets/pilotdemo-lb-sslainddef-tr-20260330-220026 and the eval dataset accounts/pyroworks/datasets/pilotdemo-lb-sslainddef-ev-20260330-220026. Then, fine-tune a suitable open model on this data using the training dataset for training and the eval dataset for evaluation. Select the best open model based on the data characteristics, run hyperparameter sweeps, and deploy the final fine-tuned model. + Dataset Interface text: Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
14:24Demo milestone
- 14:28
can, uh, input where your training dataset lives within the Fireworks, what it's called. Uh, you can also, uh, give it the eval dataset itself. Uh, I know people have the train and eval or a test dataset separately, so you give the mo-- you give the agent both the datasets, and then you ask it, uh, to fine-tune and even select the hyperparameters and the model for you. The, the agent then goes on and tells you the step that it will,
Training agent fine-tuning demo
The demo asks the training agent to inspect separate training and evaluation datasets, select a model, run hyperparameter sweeps, and deploy it. The agent checks the datasets and begins building a plan.
Text in image
First, inspect the training dataset accounts/pyroworks/datasets/pilotdemo-lb-sslainddef-tr-20260330-220026 and the eval dataset accounts/pyroworks/datasets/pilotdemo-lb-sslainddef-ev-20260330-220026. Then, fine-tune a suitable open model on this data using the training dataset for training and the eval dataset for evaluation. Select the best open model based on the data characteristics, run hyperparameter sweeps, and deploy the final fine-tuned model. Hello! I'll start by routing this to the fine-tuning workflow, which will inspect your datasets, determine the best approach, and create a detailed plan. Tool execution I'll start by verifying the session root, checking the datasets, and reading the necessary reference files. Tool executions (11) Good. Both datasets are READY. Let me now download and inspect the actual data to understand the task type, format, and token characteristics. Tool executions (4) Now let me inspect the data in detail: Tool executions (2) Excellent! Now I have a clear picture. Let me read the remaining reference files needed for planning: Tool executions (16) Now I have a thorough understanding of the data and all references. Let me analyze the findings and build the plan. Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. Interface text: Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
14:44Demo milestone
Dataset findings
The agent finds 508 training examples for legal defendant-name extraction and recommends qwen3-8b, judging 4B potentially too weak for legal text and 32B excessive for short JSON outputs.
Text in image
Now I have a thorough understanding of the data and all references. Let me analyze the findings and build the plan. Key findings from dataset inspection: • Task: Legal NER / structured extraction — extracting individual defendant names from securities class action complaint excerpts, returning strict JSON {"individual_defendants": [...]} • Format: CHAT, 3-turn (system → user → assistant), already clean • Training set: 508 samples, ~316,900 tokens (API), ~294,640 tokens (local estimate) • Eval set: 507 samples, ~316,400 tokens (API), ~294,475 tokens (local estimate) • Avg tokens per example: ~580 total, ~550 input (system+user), ~31 output (assistant) • Output: Structured JSON with short responses — objectively checkable • No conversion needed: Data is already in proper CHAT format Model selection reasoning: This is a structured extraction task with short outputs. The data is moderate-sized (508 train). The outputs are JSON with named entities. Given: • The task is relatively straightforward extraction (not complex reasoning) • Outputs are short (~31 tokens avg) • 508 training samples is modest • A smaller model can handle this well and is more cost-efficient I'll recommend qwen3-8b as the primary base model — it offers a good balance of capability and cost for this structured extraction task. The 4B models might underperform on legal text comprehension, while 32B would be overkill for short JSON extraction. Evaluation path: Since outputs are structured JSON that can be objectively checked (exact match on extracted names), this is a strong Path C candidate — agent-generated evaluator. Epochs: min(5, ceil(2500/508)) = min(5, 5) = 5 for HP tuning and final run. Now let me build the plan and cost inputs: Tool executions (5) Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Credits: $0.00 Impersonate pyroworks V Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 714:52Demo milestone
- 14:59
it will take to accomplish the task that you have given it. So it, it details the SFT plan. It tells you why it selected the base model. In this case, it selected Qwen3-8B. And it'll also give you some information on the hyperparameters and what the whole training process can look like at a high level. And once you're done, uh, once you are, uh, you feel okay about the output that the agent has given, you can approve whatever the agent has, uh, the a- the agent has outlined. And
Model and evaluation plan
The agent selects Qwen3-8B for legal name extraction, proposes an evaluator that matches names against ground truth, and lays out a three-configuration hyperparameter sweep.
Text in image
• Outputs are short, structured JSON — objectively checkable • 502 unique assistant responses across 508 training examples Base Model Selection Selected: qwen3-8b (accounts/fireworks/models/qwen3-8b) Rationale: This is a structured extraction task over legal text with short JSON outputs. The 8B model provides strong text comprehension for legal domain language while remaining cost-efficient. The 4B models risk underperforming on nuanced legal entity recognition, while the 32B model would be overkill given the short, structured output format and moderate dataset size. Evaluation Path: Path C (Agent-Generated Evaluator) Since outputs are structured JSON with objectively verifiable content (extracted defendant names), an agent-generated evaluator will: 1. Parse the JSON output 2. Compare extracted names against ground truth using normalized exact match 3. Compute per-sample and aggregate accuracy The evaluator will be shown to you for review before it is used for model selection. Hyperparameter Sweep (3 configs × 1 epoch each) Config | LoRA Rank | Learning Rate | Epochs 1 | 8 | 1.5e-4 | 1 2 | 16 | 1.0e-4 | 1 3 | [illegible] | [illegible] | [illegible] Interface text: Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
15:02Demo milestone
Selection and final training
The plan evaluates three candidates on 200 eval samples: the best evaluator score wins, with lowest eval loss as a fallback if rollout is incomplete. After approval, it retrains the winner on all 508 training examples.
Text in image
[illegible] 2 16 1.0e-4 1 3 32 5.0e-5 1 All 3 HP tuning jobs run in a single batch (≤6 active job limit). Selection Process 1. Pre-rollout checkpoint: Parse remote training metrics for all 3 candidates, write training_job_eval_loss_ranking.md, present it in chat 2. Rollout comparison: Run all 3 candidates through the evaluator on a 200-sample subset of the eval set (one candidate at a time, ≤16 in-flight requests) 3. Winner selection: Best evaluator score wins; fallback to lowest eval loss if rollout is incomplete 4. Promotion gate: Present winner and ask for approval before proceeding to full training Full-Data Final Training After promotion approval: • Retrain the winning config on the full 508-sample training set • Epochs: min(5, ceil(2500/508)) = 5 • Same base model, LoRA rank, and learning rate as the winning HP config • Eval dataset used for validation Post-Final-Training Evaluation Interface text: Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
15:09Demo milestone
Plan cost and approval
The agent estimates $1.40 for inference and training, then receives approval to execute the plan. It promises progress reports in chat and a final report at the end.
Text in image
Report Delivery • HP tuning ranking → shown in chat immediately • Rollout/selection report → shown in chat immediately • Post-final-training update → shown in chat before final evaluation • Final report → shown in chat as the final message Estimated Cost Breakdown Category | Estimated Cost Inference | $0.13 Training | $1.27 Total | $1.40 Do you approve this plan and cost estimate? (yes / request changes / cancel) Approved, proceed. Tool executions (4) The plan is approved. Let me save all artifacts now. Tool executions (4) The plan has been created and approved. Let me now execute it. Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Credits: $0.00 Impersonate pyroworks V Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
15:17Demo milestone
- 15:29
once you approve it, it will actually go through different steps and keep you posted on if it finishes step two, what does, uh, it look like? If it successfully finishes step three, what does that look like now? And at the end of the whole training run, it will also give you a summary of, uh, the config, the base model, um, and, uh, different hyperparameters and how that impacts it as well.
Agent-guided fine-tuning
The agent ranks three tuning configurations by evaluation loss, then reports 37.28% exact-set accuracy and 0.6694 average F1 on 507 samples. It attributes some exact-match failures to name normalization.
Text in image
The metrics file format might be different. Let me inspect the raw metrics: Tool executions (6) Now I understand the format. Let me fix the parser: Tool executions (2) ✅ Step 3: Pre-Rollout Checkpoint Complete HP Tuning Results — Ranked by Eval Loss Rank | Config | LoRA Rank | Learning Rate | Final Train Loss | Final Eval Loss | Job ID 1 | HP1 | 8 | 1.5e-4 | 0.7041 | 0.6706 | [illegible] 2 | HP2 | 16 | 1.0e-4 | 0.7034 | 0.6710 | [illegible] 3 | HP3 | 32 | 5.0e-5 | 0.7175 | 0.6814 | [illegible] All three configs are very close in eval loss. HP1 (LoRA 8, LR 1.5e-4) leads by a slim margin. Now proceeding to rollout comparison. Step 4: Rollout Comparison First, prepare the 200-sample eval subset for inference: Tool executions (2) Now running inference for all 3 candidates. I'll run them sequentially as each needs its own deployment: Tool executions (3) Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Credits: $0.00 Impersonate pyroworks V Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
15:35Demo milestone
Show 2 builds
- 15:35
A table ranks three tuning configurations by evaluation loss before rollout comparison.
Text in image
The metrics file format might be different. Let me inspect the raw metrics: Tool executions (6) Now I understand the format. Let me fix the parser: Tool executions (2) ✅ Step 3: Pre-Rollout Checkpoint Complete HP Tuning Results — Ranked by Eval Loss Rank | Config | LoRA Rank | Learning Rate | Final Train Loss | Final Eval Loss | Job ID 1 | HP1 | 8 | 1.5e-4 | 0.7041 | 0.6706 | [illegible] 2 | HP2 | 16 | 1.0e-4 | 0.7034 | 0.6710 | [illegible] 3 | HP3 | 32 | 5.0e-5 | 0.7175 | 0.6814 | [illegible] All three configs are very close in eval loss. HP1 (LoRA 8, LR 1.5e-4) leads by a slim margin. Now proceeding to rollout comparison. Step 4: Rollout Comparison First, prepare the 200-sample eval subset for inference: Tool executions (2) Now running inference for all 3 candidates. I'll run them sequentially as each needs its own deployment: Tool executions (3) Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Credits: $0.00 Impersonate pyroworks V Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
- 15:53
Agent reports final model accuracy and F1, followed by deployment and report steps.
Text in image
at epoch 5 has higher eval loss (0.7338). This is expected behavior — the task evaluation (accuracy/F1) will determine actual performance. Step 7: Direct Final-Model Evaluation Now running the evaluator on the full 507-sample eval set using the final model: Tool executions (5) ✅ Step 7: Direct Final-Model Evaluation Complete Final Model Performance (507 samples): • Accuracy: 37.28% (189/507 exact set matches) • Average F1: 0.6694 This is a meaningful improvement over the 1-epoch HP candidate (31.5% → 37.3% accuracy). The model extracts many correct names (F1 ≈ 0.67) but struggles with exact set matching due to name normalization issues (e.g., "Nicholas D Woodman" vs "Nicholas Woodman", accented characters). Step 8: Deployment The final model is already deployed from Step 7 inference (deployment inference-4d9e2015). The model accounts/pyroworks/models/qwen3-8b-ft-lr1e4-lora16-ep5-1777414879 is available for production inference on Fireworks. Step 9: Final Report Tool executions (5) Now let me generate the final report and save actual costs: Tool executions (2) Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Credits: $0.00 Impersonate pyroworks V Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
- 15:35
- 15:59
And once it's-- once the whole model training is done, it actually gives you metrics such as accuracy and F1 score, uh, and it also summarizes, kind of re-summarizes and gives you an overview of everything that, uh, that it did. And one cool feature is it also gives you recommended next step as part of, uh, the training run itself. Now, the next portion of, uh, this demo actually talks about how we can use our, uh, managed training platform. I did mention that you just
Training report and next steps
The agent reports 37.28% exact set match and 0.67 average F1 on 507 evaluation samples. It flags overfitting and name-formatting mismatches, and recommends early stopping, normalization and fuzzy evaluation.
Text in image
3. Hyperparameter Sweep: 3 configurations were tested (varying LoRA rank and learning rate). The winner was LoRA rank 16 + learning rate 1e-4. 4. Full Training: 5 epochs on all 508 samples with the best hyperparameters. 5. Evaluation: The final model achieved 37.28% exact set match accuracy and 0.67 average F1 on the 507-sample eval set. Final Model • Model ID: accounts/pyroworks/models/qwen3-8b-ft-lr1e4-lora16-ep5-1777414879 • Total Cost: ~$1.30 Key Observations • The model shows some overfitting after epoch 2 (best validation loss was at epoch 2) • Many "errors" are actually name normalization issues (middle initials, accented characters) rather than true extraction failures • The F1 of 0.67 indicates the model finds most correct names but struggles with exact matching due to formatting differences Recommended Next Steps • Consider early stopping at epoch 2-3 for better generalization • Improve name normalization handling (middle initials, accents, suffixes) • Consider fuzzy matching evaluation for a more realistic accuracy picture • The model is ready for production use at the model ID above Interface text: Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support Memory usage: 369 MB [illegible] Reply... + Dataset Fireworks Agent can make mistakes. Please verify responses. 7
16:09Demo milestone
Select a base model
The managed training demo opens the supervised model library to choose a base model or LoRA adapter. The presenter says dataset and hyperparameter choices follow.
Text in image
Supervised | Reinforcement | Preference 1 Model Choose a base model or a LoRA adapter to start fine-tuning. Select a model Search Model Library | Custom Models Kimi K2.6 (LoRA) Moonshot AI GLM 5.1 (LoRA) Z.ai Kimi K2.5 (LoRA) Moonshot AI Gemma 4 26B A4B IT (LoRA) Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
16:26Demo milestone
- 16:29
have to select the model dataset and the hyperparameters, and then you're just a button click away from actually training a model. Say you've uploaded, uh, the dataset, chosen a model, uh, all you'll have to do is then, uh, go to the, go to the, uh, evaluator, uh, settings, choose the evaluator dataset, and also you get to choose the hyperparameter as part of optional settings if you decide to have a more fine-grained hyperparameters for your, uh, training run itself.
Fine-tuning setup
The demo shows a supervised setup using a selected dataset without validation, followed by reinforcement settings for rollout and training parameters such as epochs, batch size, and LoRA rank.
Text in image
Supervised Reinforcement Preference Model Kimi K2.6 2 Dataset Learn more Select dataset demo-gsm8k-math-dataset-1000 OR Upload or Drag & Drop Select evaluation option Do not use a validation dataset Back Next 3 Optional Settings Learn more Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
16:30Demo milestone
Show 2 builds
- 16:30
Fine-tuning form showing a selected dataset and validation option.
Text in image
Supervised Reinforcement Preference Model Kimi K2.6 2 Dataset Learn more Select dataset demo-gsm8k-math-dataset-1000 OR Upload or Drag & Drop Select evaluation option Do not use a validation dataset Back Next 3 Optional Settings Learn more Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
- 16:39
Reinforcement fine-tuning form with optional settings expanded.
Text in image
Supervised Reinforcement Preference Model Kimi K2.6 Evaluator & Dataset demo-gsm8k-math-dataset-1000 Rollout Max Output Tokens: 2048 Temperature: 1 Top P: 1 Response Candidates Count: 4 Top K: 0 4 Optional Settings Learn more Model Output Name Default: ft-<timestamp>-<random-string> Job ID Default: auto-generated Display Name An optional name for your job Epochs 1 Batch Size 65536 LoRA Rank 8 Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
- 16:30
Select evaluator and dataset
The reinforcement fine-tuning form pairs an evaluator with a dataset. The demo selects gsm1k-evaluator and a GSM8K math dataset, with an upload option available before proceeding to rollout.
Text in image
Supervised Reinforcement Preference Model Kimi K2.6 2 Evaluator & Dataset Learn more Create new Select evaluator See all evaluators gsm1k-evaluator Select dataset demo-gsm8k-math-dataset-1000 OR Upload or Drag & Drop Back Next 3 Rollout Learn more Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
16:42Demo milestone
Configure rollout
The rollout form sets a 2,048-token output limit, temperature and top P of 1, and four response candidates. Optional settings follow for finer control of the training run.
Text in image
Supervised Reinforcement Preference Model Kimi K2.6 Evaluator & Dataset gsm1k-evaluator • demo-gsm8k-math-dataset-1000 3 Rollout Learn more Max Output Tokens: 2048 Temperature: 1 Top P: 1 Response Candidates Count: 4 Top K: 0 Extra Rollout Parameters Back Next 4 Optional Settings Learn more Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
16:51Demo milestone
Configure training parameters
Optional settings let the user adjust training hyperparameters, including learning rate, context length, and loss method. Once the configuration is ready, the presenter says to click Create to start the training job.
Text in image
4 Optional Settings Learn more Model Output Name Default: ft-<timestamp>-<random-string> Job ID Default: auto-generated Display Name An optional name for your job Epochs: 3 Batch Size: 65536 LoRA Rank: 8 Learning Rate: 0.0002 Max Context Length: 65536 Chunk Size: 200 Gradient Accumulation Steps: 1 Learning Rate Warmup Steps: 0 Loss Method: GRPO Enable Weights & Biases Back Create Interface text: Fine-Tuning / Create Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
16:59Demo milestone
- 17:00
And once you're happy with all the configurations, including the model and the dataset itself, you just need to create, uh, you just need to click on the button that says Create. Um, and as soon as you click on the button that says Create, a, a training job then ki-kicks out, and you can see that, uh, the UI looks, uh, similar to this when a training job actually gets completed.
- 17:26
And if you click on that particular training job itself, it'll give you information on the dataset it used, the hyperparameters it used, what the model loss itself looks like when the model was training, um, and then you can even clone these training runs and reproduce it with different settings and configurations itself.
Completed training job
The completed job shows model loss dropping sharply in the first few steps, then declining more gradually. Its details include the dataset, training settings and estimated cost.
Text in image
qwen3-4b ORPO smoke 202605... accounts/pyroworks/dpoJobs/nbnkwm6631pfuk3k View Model Completed Actions Deploy Model Loss None Show Step Y-axis: Model Loss (loss) Y-axis ticks: 0.0, 0.5, 1.0, 1.5, 2.0, 2.5 X-axis ticks: 1, 5, 10, 15, 20, 25 Output Model: qwen3-4b-orpo-smoke-20... Base Model: qwen3-4b Type: Conversation State: Completed Created On: Wednesday, May 27, 2026 Estimated Cost: $0.74 Configuration Dataset: data-compliance-zh-dpo... Epochs: 1 LoRA Rank: 16 Learning Rate: 0.0001 Learning Rate Warmup Steps: 0 Batch Size: 32768 Gradient Accumulation Steps: 1 Deployments No current deployments. Interface text: Fine-Tuning / Preference / nbnkwm6631pfuk3k Total Spend: $0.00 Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
17:32Demo milestone
Deployments
From Deployments, the presenter says to select the trained custom model, choose its deployment settings, and deploy it. The list shows existing replicas, GPU counts, and statuses.
Text in image
Deployments Manage your on-demand deployments and serverless deployed models. Learn more Create Deployment Search by name, model, or created by On-demand LoRA Addons Name | Base model | # Replicas [Min,Max] | GPUs | Status yftxy54p | accounts/fireworks/models/qwen3-8b | 0 [1,1] | 1 | Deleting ID: yftxy54p | ID: qwen3-8b cd-rft-q4b-[illegible]-a-0214-205340-refresh-71ac256a-1771306506 | accounts/fireworks/models/qwen3-4b-instruct-2507 | 0 [0,1] | 1 | Scaled to 0 ID: cd-rft-q4b-[illegible]-a-0214-205340-refresh-71ac256a-1771306506 | ID: qwen3-4b-instruct-2507 f9d9qqwv | accounts/fireworks/models/gemma-3-27b-it | 0 [0,1] | 2 | Deleting ID: f9d9qqwv | ID: gemma-3-27b-it deepmath-qwen3-32b-1773352493 | accounts/fireworks/models/qwen3-32b | 0 [0,2] | 2 | Scaled to 0 ID: deepmath-qwen3-32b-1773352493 | ID: qwen3-32b deepmath-qwen3-32b-1773352719 | accounts/fireworks/models/qwen3-32b | 1 [0,0] | 2 | Ready ID: deepmath-qwen3-32b-1773352719 | ID: qwen3-32b deepmath-qwen3-32b-1773353003 | accounts/fireworks/models/qwen3-32b | 1 [0,0] | 2 | Ready ID: deepmath-qwen3-32b-1773353003 | ID: qwen3-32b deepmath-qwen3-32b-1773353198 | accounts/fireworks/models/qwen3-32b | 1 [0,0] | 2 | Ready ID: deepmath-qwen3-32b-1773353198 | ID: qwen3-32b Interface text: Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 7
17:39Demo milestone
- 17:47
And once you're happy with the fine-tuned model, you can go to the Deployment section, choose the custom model that just got, uh, trained on our platform, and then click on Deploy once you're happy with the deployment configurations, or you can choose a set of deployment configurations that we have, such as throughput, fast, minimal, uh, and this is, uh, kind of a template that, that works well for generic workloads that we have optimized based on throughput, latency,
Deployment shapes
The demo compares deployment shapes for a custom model, with Full Precision selected. The presenter describes the presets as templates optimized for generic workloads and metrics such as throughput and latency.
Text in image
Deployments / Create Base Model reasoning-gsm8k-tianyi-qwen-r1-fcs Region GLOBAL Deployment Shapes Custom Learn more Throughput NVIDIA H100 80GB $7.00 1 GPUs Precision: FP8 Fast NVIDIA H100 80GB $7.00 1 GPUs Precision: FP8 Minimal NVIDIA H100 80GB $7.00 1 GPUs Precision: FP8 Fast NVIDIA H200 141GB $7.00 1 GPUs Precision: FP8 MM V2 Full Precision NVIDIA H200 141GB $7.00 1 GPUs Precision: BF16 Learn more Deployment Scaling Autoscaling · Replicas: 0-1 · Scale to zero: 60 minutes Hot Load Disabled Interface text: Credits: $0.00 Impersonate pyroworks Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support 3:51 / 4:53
18:08Demo milestone
Deployment scaling
The deployment form enables autoscaling from 0 to 1 replicas at 80% usage. The open menu also offers concurrency, request rate, and token-rate metrics.
Text in image
Fast NVIDIA H200 141GB 1 GPUs Precision: FP8 MM V2 $7.00 Full Precision NVIDIA H200 141GB 1 GPUs Precision: BF16 $7.00 Learn more Deployment Scaling Enable Auto Scaling Min Replicas: 0 Max Replicas: 1 Advanced Options Autoscaling Metric Usage % Concurrency Requests per second Generated tokens per second Prompt tokens per second Autoscaling Threshold: 80% [illegible] Disabled Projected hourly cost $0 - $7 Deploy Interface text: Deployments / Create Credits: $0.00 Impersonate pyroworks V Home Agent Analytics Settings CREATE Serverless NEW Deployments Fine Tuning Batch API EXPLORE Fire Pass NEW Models Playground MANAGE Custom Models Datasets Evaluations Docs Discord Support
18:14Demo milestone
- 18:18
uh, time to first token, and other such metrics.
- 18:23
And, uh, I unfortunately don't have an entire demo for our training API because the setup itself differs from customer to customer. But we do have a lot of cookbooks on our websites that you can see and follow, especially if you're interested in customizing your own training loops and loss functions itself. Uh, and that's all I have for you for today. I just wanted to give you a brief overview of how the whole training to inferencing loop looks like. Thank you very much, and if you
Training API documentation
The Training API puts GPU infrastructure on Fireworks and training logic in Python. Cookbook recipes help users customize loops and loss functions; the docs mark the API as private preview.
Text in image
Who does what Fireworks handles | Cookbook recipes handle | Direct Training SDK users implement GPU provisioning and cluster management | Training loop structure for supported recipes | Training loop logic (forward_backward_custom + optim_step) Service-mode trainer lifecycle (create, health-check, reconnect, delete) | Resource setup, health checks, reconnect, and cleanup | Manager/client wiring when working below recipe utilities Distributed forward pass, backward pass, optimizer execution | Common losses and reward/evaluation plumbing | Loss function and batch construction Checkpoint storage and export | Checkpoint save, resume, promotion, and weight sync helpers | Checkpoint calls (save_weights_for_sampler_ext, DCP snapshots) Inference deployments and hotload | Deployment sampling and serving-integrated evaluation for RL recipes | Custom rollout, sampling, and evaluation logic Preemption recovery and job resume | Resume logic for supported recipe checkpoints | Resume policy and state restoration calls Distributed training (multi-node, sharding, FSDP) | Config surfaces for learning rate, grad accumulation, context length, W&B | Hyperparameter schedules, data pipeline, and experiment tracking Interface text: Documentation API & SDK Reference CLI Reference Demos Changelog Resources Community Status Dashboard Search... Kimi K2 family Advanced Training Training Overview Fireworks Agent Managed Fine-Tuning Bring Your Own Trainer Training API (Tinker compatible) Using the Training API Introduction Quickstart Training and Sampling Loss Functions Saving and Loading Training Shapes Price comparison vs Tinker Vision Inputs API Reference The Cookbook On this page What is the Training API? Who does what System architecture How service-mode training works Minimal training step lifecycle Datums Logprobs and forward_backward_custom Futures Checkpointing and weight sync Key APIs Renderers Comparing Training API pricing vs DIY bare metal What to compare Useful formulas Keep assumptions explicit FAQ Why is my training run “doing nothing” even though code executed? What’s the difference between base and delta checkpoints, and when [illegible] Ask a question... 7
18:26Demo milestone
Show 2 builds
- 18:26
Documentation table comparing responsibilities across the platform, cookbook recipes, and direct SDK users.
Text in image
Who does what Fireworks handles | Cookbook recipes handle | Direct Training SDK users implement GPU provisioning and cluster management | Training loop structure for supported recipes | Training loop logic (forward_backward_custom + optim_step) Service-mode trainer lifecycle (create, health-check, reconnect, delete) | Resource setup, health checks, reconnect, and cleanup | Manager/client wiring when working below recipe utilities Distributed forward pass, backward pass, optimizer execution | Common losses and reward/evaluation plumbing | Loss function and batch construction Checkpoint storage and export | Checkpoint save, resume, promotion, and weight sync helpers | Checkpoint calls (save_weights_for_sampler_ext, DCP snapshots) Inference deployments and hotload | Deployment sampling and serving-integrated evaluation for RL recipes | Custom rollout, sampling, and evaluation logic Preemption recovery and job resume | Resume logic for supported recipe checkpoints | Resume policy and state restoration calls Distributed training (multi-node, sharding, FSDP) | Config surfaces for learning rate, grad accumulation, context length, W&B | Hyperparameter schedules, data pipeline, and experiment tracking Interface text: Documentation API & SDK Reference CLI Reference Demos Changelog Resources Community Status Dashboard Search... Kimi K2 family Advanced Training Training Overview Fireworks Agent Managed Fine-Tuning Bring Your Own Trainer Training API (Tinker compatible) Using the Training API Introduction Quickstart Training and Sampling Loss Functions Saving and Loading Training Shapes Price comparison vs Tinker Vision Inputs API Reference The Cookbook On this page What is the Training API? Who does what System architecture How service-mode training works Minimal training step lifecycle Datums Logprobs and forward_backward_custom Futures Checkpointing and weight sync Key APIs Renderers Comparing Training API pricing vs DIY bare metal What to compare Useful formulas Keep assumptions explicit FAQ Why is my training run “doing nothing” even though code executed? What’s the difference between base and delta checkpoints, and when [illegible] Ask a question... 7
- 18:40
Fireworks documentation introduces the Training API and links to cookbook guidance.
Text in image
Using the Training API Introduction Fireworks Training API — custom training loops with full Python control over objectives, while Fireworks handles distributed GPU infrastructure. The Training API is currently in private preview. Request early access to get started. Using a code agent? Clone fw-ai/cookbook. The cookbook includes the _skills/dev/ skill, which gives agents repo-specific guidance for setup, debugging, hotload, RL recipe internals, and checkpoint promotion. What is the Training API? Fireworks Training API lets you write training logic in plain Python on your local machine while model computation runs on remote GPUs managed by Fireworks. Most users should start from cookbook recipes, the recommended entry point [illegible] Interface text: Documentation API & SDK Reference CLI Reference Demos Changelog Resources Community Status Dashboard Search... Kimi K2 family Advanced Training Training Overview Fireworks Agent Managed Fine-Tuning Bring Your Own Trainer Training API (Tinker compatible) Using the Training API Introduction Quickstart Training and Sampling Loss Functions Saving and Loading Training Shapes Price comparison vs Tinker Vision Inputs API Reference The Cookbook Copy page On this page What is the Training API? Who does what System architecture How service-mode training works Minimal training step lifecycle Datums Logprobs and forward_backward_custom Futures Checkpointing and weight sync Key APIs Renderers Comparing Training API pricing vs DIY bare metal What to compare Useful formulas Keep assumptions explicit FAQ Why is my training run “doing nothing” even though code executed? What’s the difference between base and delta checkpoints, and when [illegible] Ask a question... https://docs.fireworks.ai/fine-tuning/training-api/cookbook/overview 7
- 18:26
Training cookbook recipes
The cookbook lists training-loop recipes. The speaker points to cookbooks for customizing training loops and loss functions, noting that setup differs by customer.
Text in image
cookbook / training / recipes Name | Last commit message | Last commit date .. async_rl_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday dpo_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday igpo_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday orpo_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday rl_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday sft_loop.py | docs: update cookbook from Fireworks staging (#478) | yesterday Interface text: fw-ai / cookbook Code Issues 3 Pull requests 77 Actions Projects Wiki Security and quality 659 Insights Settings Type / to search Files main Go to file .github archived integrations multimedia skills training examples recipes renderer tests utils README.md __init__.py pyproject.toml .gitignore Add file 5 people docs: update cookbook from Fireworks staging (#478) yesterday History
18:46Demo milestone
- 18:53
have any questions, I'm happy to take them in the next minute or so that we have.