AI Engineer Summit 2025

How To Build an AI Strategy That Fails

Hamel Husain· Founder, Parlance LabsGreg Ceccarelli· Co-Founder, SpecStory17:03

Loading slides…

Read the talk

How to Build an AI Strategy That Fails

AI projects become expensive failures when strategy, language, staffing, and evaluation separate the technology from the people who understand the work.

From a talk by Hamel Husain and Greg Ceccarelli

What would guarantee an AI strategy fails?

Suppose the goal were to waste an AI budget, derail the company, and alienate everyone involved. What would you do? Hamel Husain and Greg Ceccarelli open with that deliberately inverted assignment. Following Grace’s presentation on best practices, they promise a guide to spectacular failure: not merely a disappointing project, but the kind of organizational damage that threatens companies and careers. The recommendations that follow are intentionally bad advice.

Two presenters stand beneath the talk’s title slide, which pairs the title and subtitle with a robot image.Open full source frame
How to Build an AI Strategy That Fails: A Guide to Wasting Time, Money & Resources.

Their perspectives span the executive office and implementation work. Greg introduces himself as an AI startup co-founder, formerly chief product officer at Pluralsight; Hamel is a machine learning engineer and independent consultant who has worked with many companies adopting AI. They frame the exercise with a maxim they attribute to Charlie Munger: “Invert. Always invert.” Instead of describing an ideal strategy, identify the decisions that reliably undermine one.

0:180:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:18 · section reference included

Divide the company, then disconnect spending from value

Start by dividing your own organization. Set unreasonable goals that disconnect customer willingness to pay, price, and cost. Attend every AI conference, but never bring the learning back to your team. The image of Moses parting the Red Sea supplies the organizational metaphor: create impenetrable silos and reward secrecy. Knowledge accumulates in individuals without becoming a shared basis for decisions.

Next, invert the value stick. The resulting anti-value stick replaces economic reasoning with promises, purchases, and complexity:

LabelInverted meaningDecision it encourages
WTPWishful thinking promisesPromise that AI can do everything.
PRICEParticularly ridiculous infrastructure costs everywhereBuy expensive GPUs without cost-benefit analysis.
CostCascade of spectacular technical debtBuild tightly intertwined systems.
WTSWhy this system?Answer every objection with “because AI.”

The promises escalate from writing emails to walking dogs, solving climate change, and achieving world peace. Infrastructure spending follows without a calculation of what it buys. Technical debt then becomes a perverse form of job security: make the system so convoluted that only its creator can repair it. Finally, the question of why the system should exist disappears behind AI itself as the justification. The joke ends with a product resembling magic, only more expensive and less reliable.

Slide titled “Adhere to the Anti-Value Stick,” showing WTP, Price, Cost, and WTS beside four explanatory arrows and a cartoon dog in a burning room.Open full source frame
The anti-value stick pairs promises, infrastructure costs, and technical debt with a “This is fine” cartoon.
2:192:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:19 · section reference included

Replace strategy with declarations and a permanent backlog

With the organization divided, strategy can become a sequence of declarations that never requires understanding the work:

  1. Fake the diagnosis. Take last year’s annual report or operating plan, highlight passages you barely understand, and declare that AI must fix them. Do not consult the people doing the work.
  2. Make the guiding policy unbounded. Aim to become the global AI leader in everything, leaving everything undefined and someone else’s responsibility.
  3. Announce disconnected actions. Commission an AI SEO tool that guarantees top Google rankings for garden gnomes, a generative art plugin that makes NFTs of the CEO’s cat, and an AI drone lunch delivery service. Present the collection at an all-hands in a shiny suit, repeatedly invoking disruption.
  4. Remove the finish line. Replace timelines with perpetual beta. Put the highlighted financial reports into a massive GitHub backlog.

Each step preserves the appearance of strategy while removing a constraint that would make it actionable: a grounded problem, a bounded direction, coherent work, or a completion date.

Communication can protect that ambiguity. Post a deliberately absurd 4,000-page strategy document in every Slack channel until people lose the will to engage. Then explain it with a sentence that piles multimodal agentic transformers, few-shot learning, chain-of-thought reasoning, synergy, and hyperparameter space on top of one another. Confidence substitutes for meaning, including for the person speaking. The rejected alternative is a clear business-on-a-page approach: something concise enough for colleagues to understand and question.

5:005:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:00 · section reference included

Use technical language to exclude the people who know the work

Jargon does more than make documents unpleasant. The name assigned to a task changes who believes they can participate. Hamel describes a mental health client where writing a prompt was described as building agents. The mental health experts were consequently absent from the room and unsure how to contribute. An activity that needed their judgment had been presented as belonging to someone else’s technical specialty.

The same substitution appears in other parts of an AI system:

Technical labelWork people need to understand
RAGGive the AI the right context.
Prompt injectionUsers tricking the AI into unwanted behavior.
Prompt writingSpecify how the AI should respond to customers.

The failure comes from using the labels to conceal the work and its owners. If engineers alone write prompts, the people who best understand customers may never shape the instructions. The displayed jargon guide extends the joke: translate ordinary tasks into language that makes even prompt writing seem inaccessible.

Slide instructing engineers, not domain experts, to write prompts, alongside a table translating RAG, prompting, hallucination, hyperparameters, and prompt injection into everyday language.Open full source frame
An examples slide pairs excluding domain experts from prompt writing with translations of common AI terms.
7:317:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:31 · section reference included

Mobilize the wrong expertise and ship before testing

The giant backlog now needs people. Greg introduces a parody of Geoffrey Moore’s Zone to Win: zoning to lose. Begin by assigning AI tasks without regard to relevant experience. The concrete example is outsourcing data review to an offshore QA team with little context about the business. The missing business context is the consequential part: reviewers are being asked to judge outputs without the knowledge needed to recognize whether those outputs serve the task.

Next, turn incubation directly into customer exposure. Launch untested, bug-ridden chatbots from the incubation zone, skipping beta testing and quality assurance. What should have been a place to develop new ideas becomes a route to a public failure. Finally, pull the best engineers away from revenue-producing products to support the effort. The South Park-style sequence—remove essential capacity, wait, expect profit—ends here in collapse. The existing business pays for the experiment before the experiment has established that it works.

8:599:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:59 · section reference included

Treat every failure as a purchasing decision

Once the organization is in disarray, focus on tools instead of processes. Do not analyze the failures already created; respond to each with another technical purchase or intervention:

  • Wrong retrieval results: buy a more expensive vector database instead of investigating why the RAG system retrieves the wrong documents.
  • Unclear progress: adopt every available evaluation metric without adapting it to business needs, then trust the numbers even when they make little sense.
  • Failing agents: switch frameworks and vendors, or fine-tune without measuring whether the change improves anything.

Fine-tuning without evaluation becomes, in Greg’s phrase, “alchemy with a lot more electricity.” The intervention may be substantial, but there is no measurement connecting it to improvement.

This approach also transfers responsibility for evaluation to the vendor: plug in a tool and expect it to define and solve the problem. The Whac-A-Mole demonstration captures the resulting loop. A problem appears, so hit it with a tool. Another appears, so hit that one. The original returns, so choose a different tool. Activity continues, but the organization never develops an explanation for the recurring failure.

10:2810:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:28 · section reference included

Collect enough metrics to find a success story

Treating evaluations as a vendor problem leads naturally to the dashboard of everything. Assume that a single evaluation solution can fit every business, collect every off-the-shelf metric, and ignore whether any of them track customer outcomes or actual failure modes. Hamel’s illustrative scores of 3.5 and 4.5 expose the problem: if nobody knows what the difference means, the apparent precision does not help anyone decide what to change. Keep collecting metrics until one trends upward, then claim success.

Evaluation frameworks offer plenty of numbers to populate that dashboard. Greg names cosine similarity, BLEU, and ROUGE as targets one could optimize while ignoring user experience. The missing step is cross-checking those measures with domain experts and users. Accepting an LLM’s judgment that an answer is accurate without challenge simply replaces that check with another automated output. A metric needs a defensible relationship to success in the actual task. Otherwise, optimization can produce a better-looking dashboard without a better product.

12:0212:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:02 · section reference included

Avoid the data, then make sure nobody else can inspect it

The most potent failure technique is to stop looking at data altogether. Hamel recommends keeping a blindfold nearby for accidental encounters with evidence. Greg extends the joke into complete trust in AI outputs nobody has inspected. Leaders can then designate data review an engineering problem and return to meetings about meetings, assuming that developers possess more domain expertise than the business teams.

Quality assurance shifts to customers, who might eventually complain if something goes wrong. Meanwhile, leaders substitute intuition for evidence, even in million-dollar decisions. If the facts threaten that confidence, the blindfold returns. The sequence removes both ways the organization might discover a mistake before it becomes expensive: deliberate inspection and informed challenge.

The final escalation is to make data inaccessible to everyone else. Treat engineers as coding wizards who can handle everything, even if they have not spoken to a customer in years. Forget that a spreadsheet can support annotation and direct inspection. The executive’s refrain becomes “Remember, this is beyond me.” Then put the data in systems only engineers can access, ensuring that domain experts cannot review it either.

The practical comparison is a spreadsheet or Airtable versus a custom analysis platform so elaborate that it supposedly takes a team of PhDs to operate. The joke awards bonus points if the platform takes six months to load and errors incessantly. Complexity now obstructs the basic activity the organization needs: letting knowledgeable people examine examples and say what is wrong. A simpler review surface preserves their ability to participate.

13:3513:49
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:35 · section reference included

The advice is inverted; the experience is real

Following this sequence would waste time and resources while alienating the people needed to make AI useful. That is where the satire gives way to the speakers’ direct invitation: visit AI-Execs for practical advice. At the time of the talk, they also announce the forthcoming O’Reilly publication AI Essentials for Tech Executives for February 27. Their recommendations have been inverted, Greg emphasizes, but the lived experience behind them has not. They close by inviting questions at the speaker booth after the presentation.

15:5516:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:55 · section reference included

Resources

From the talk

  • Husain and Ceccarelli's free guide to AI strategy, communication, hiring and direct data review.

  • Geoffrey Moore's framework separates enterprise work into performance, productivity, incubation and transformation zones.

Transcript download
Full transcript

Loading transcript…