AI Engineer Summit 2025
Building Trust in Enterprise AI: Evaluating Domain-Specific LLMs for Real-World Financial Scenarios
Loading slides…
Read the talk
Building Trust in Enterprise AI: Evaluating Domain-Specific LLMs for Real-World Financial Scenarios
FailSafe tests whether financial language models can handle damaged queries while recognizing when the supplied documents cannot support an answer.
From a talk by Waseem Alshikh
Before you start: Basic familiarity with language models and retrieval-augmented generation is helpful; no finance expertise is required.
Does better general accuracy make specialization unnecessary?
If general language models keep improving, is it still worth building models specifically for finance or medicine? For Waseem Alshikh, Writer’s cofounder and CTO, this is a practical investment question. He traces Writer’s model development back to its founding in 2020 and early encoder and decoder models. At the time of the talk, he estimates that the company has published roughly 16 models, with another 20 on the way. Its portfolio includes general models—Palmyra X, Palmyra 3 and Palmyra 4—alongside models for creative work, financial services and medicine, including Palmyra Medical. Palmyra 5 is still forthcoming in this account.
Alshikh describes good general models as reaching roughly 80–90% accuracy, without identifying the benchmarks behind that range. The improvement he has observed since early 2024 raises several alternatives to continued specialization: concentrate on general models, fine-tune them, or invest in reasoning models. If those approaches already deliver the needed performance, maintaining separate financial and medical models may be unnecessary. The decision requires evidence about what happens beyond ordinary benchmark questions.
Financial services supplies the test case. The same evaluation question applies to medical and customer-support models, but those domains need their own evidence. Alshikh mentions similar results from medical evaluation without presenting that evaluation here.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Break the query without changing the underlying task
Writer develops FailSafe, documented in Expect the Unexpected: FailSafe Long Context QA for Finance, to test whether strong benchmark performance survives realistic financial requests. Its design separates failures in the user’s query from failures in the context supplied to answer it. That separation matters: recovering a user’s intended question and recognizing insufficient evidence require different behavior.
The first query perturbation is misspelling. Instead of assuming clean input, the evaluation introduces spelling mistakes, punctuation problems and common typing errors. The displayed example makes these defects concrete, highlighting split errors, segment errors, real-word errors and common typos in a financial query. A useful model should recover the intended question when the wording remains interpretable.
Two further query variants test how much the model depends on polished financial language:
- Incomplete queries: Remove keywords or other information, leaving parts of the request unclear.
- Out-of-domain queries: Express a financial question without expert terminology. Alshikh describes nonexpert wording and pasted general material; in the published benchmark, this perturbation uses everyday-language rephrasing.
Together, these variants test robustness to how people ask, rather than assuming every user knows the domain’s vocabulary.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What if the uploaded document cannot answer the question?
The second family of tests changes the evidence available to the model. Missing context asks for information that the prompt does not provide; the published benchmark implements this by removing the document entirely. OCR errors preserve a document but damage its text during the simulated conversion from a physical page: characters become corrupted, spacing changes and words merge. These tests distinguish absent evidence from evidence that is present but difficult to read.
Irrelevant context introduces a particularly revealing failure: the user asks about one financial document but uploads a completely different document. Does the model identify the mismatch, or answer anyway? A fluent answer is not sufficient here. The model must determine whether the supplied evidence supports the requested claim.
A small Python fixture illustrates that distinction. Suppose the question asks for revenue in an annual report. The clean and OCR-damaged versions retain the relevant fact; the missing and irrelevant versions do not. The values below are a teaching example of the perturbation pattern:
python
question = "What revenue does the annual report report?"
cases = [
{
"id": "baseline",
"question": question,
"context": "Annual report: Revenue was $12 million.",
"expected": "answer_from_context",
},
{
"id": "ocr_error",
"question": question,
"context": "Annual report: Revenuewas $12 million.",
"expected": "answer_from_context",
},
{
"id": "missing_context",
"question": question,
"context": "",
"expected": "identify_insufficient_context",
},
{
"id": "irrelevant_context",
"question": question,
"context": "Office policy: Visitors must sign in at reception.",
"expected": "identify_insufficient_context",
},
]
The expected field specifies the desired behavior, not an observed model result. Keeping the question fixed makes the change in available evidence explicit.
The financial evaluation also needs diversity across its examples, rather than many variations of one narrow request. Alshikh shows a data and token inventory, then says that the white paper, data, evaluation set and leaderboard are openly available through GitHub and Hugging Face. The FailSafeQA dataset provides a public starting point for examining the cases.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Score recovery and grounding separately
The evaluation asks two questions: can the model produce the correct answer, and does it follow the supplied context? Robustness and context grounding are separate requirements. A model can recover from a typo while still failing to recognize that a document contains no answer.
The original paper’s methods give these dimensions more precise meanings:
| Dimension | What is evaluated | Aggregation |
|---|---|---|
| Robustness | Baseline and answerable perturbations | Average of each example’s minimum compliance |
| Context grounding | Missing or irrelevant context | Average compliance under those conditions |
Taking the minimum across answerable variants makes a fragile variant matter: success on the clean question does not erase failure when the same task arrives with a defect. Grounding instead tests appropriate behavior when the evidence is unavailable or irrelevant. Neither measure is unrestricted factual accuracy.
Alshikh presents a calculation overview and two lists of models, separating chat models from thinking models, before moving through the result charts. This comparison tests whether reasoning capability also brings better adherence to evidence, rather than treating those capabilities as interchangeable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Willingness to answer can hide a failure
The thinking models initially appear helpful because they keep answering. Under faulty context, that behavior becomes a liability. Alshikh reports that models continue producing answers even when given wrong data or a completely different grounding document, and associates that behavior with increased hallucination. The failure is not an inability to generate an answer; it is an inability to respect what the evidence permits.
On the answer-focused results, Alshikh reports that general and domain-specific models are close, with thinking models scoring slightly higher. The grounding results expose a different pattern, with text generation and question answering standing out as weak areas. Looking only at answer performance would conceal the cases where the model should recognize that the context is inadequate.
Moving from charts to the underlying numbers, he calls out o1, o3 and Palmyra Fin. Misspelled, incomplete and out-of-domain queries receive strong results: models can often handle poor grammar or imperfect wording and still recover an answer. That success establishes one useful capability, but the next part of the table asks whether they also stay within the attached evidence.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
More reasoning does not guarantee better grounding
Alshikh pauses on the grounding table and asks what looks different. A brief response—“Smaller is better”—gets his agreement. He describes larger thinking models as roughly 50–60% worse on grounding, but does not specify the comparison denominator. The useful conclusion is narrower than a general rule about model size: in these financial tests, some smaller models follow the supplied context better than larger thinking models.
The mechanism is visible in the behavior he describes: a context is attached, a question is asked, and the answer draws on information entirely outside that context. Irrelevant-context tests show the same problem. A model’s ability to supply information from elsewhere can therefore work against a task whose requirement is to answer from the attached document. The highlighted table marks selected context-grounding question-answering scores and points to Palmyra Fin.
This leads Alshikh to question whether the observed chain of thought deserves to be called thinking. His criticism concerns these domain-specific tasks: additional reasoning has not prevented unsupported answers. Robustness to damaged input, correctness of an answer and avoidance of hallucination can diverge, so progress on one does not settle the others.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reliability requires more than choosing a model
Even the strongest result leaves substantial room for improvement. Alshikh describes the best score across robustness and grounding as no higher than 81%. He translates that into roughly 20 completely wrong responses per 100 requests, but the published measure requires a more careful interpretation: the original paper reports Palmyra-Fin compliance of 0.81, combining robustness of 0.83 and grounding of 0.80. It is a composite score, not a measured production error rate.
That remaining gap motivates a full system for reliability. Alshikh calls for retrieval-augmented generation, grounding and guardrails around the model. Retrieval supplies evidence; grounding keeps answers tied to that evidence; guardrails provide constraints around the resulting behavior. The context-failure tests explain why merely attaching a document cannot be the whole solution: the attachment can be absent, damaged or wrong, and the system still has to respond appropriately.
The talk returns to the original investment question: should Writer continue building domain-specific models? Alshikh’s answer is yes, based on the implementations and financial benchmark results presented. General accuracy is improving, but reliable adherence to supplied context still lags. Specialized model development remains part of the response, alongside the retrieval, grounding and guardrails needed to make those models useful in real financial workflows.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Original benchmark paper with perturbation definitions, scoring equations, evaluation prompts and model results.
Public dataset containing 220 financial examples with reference answers, citations, query variations and OCR-corrupted contexts.
Full transcript
Loading transcript…