AI Engineer Summit 2025
How To Build an AI Strategy That Fails
Loading slides…
Read the talk
How to Build an AI Strategy That Fails
AI projects become expensive failures when strategy, language, staffing, and evaluation separate the technology from the people who understand the work.
From a talk by Hamel Husain and Greg Ceccarelli
What would guarantee an AI strategy fails?
Suppose the goal were to waste an AI budget, derail the company, and alienate everyone involved. What would you do? Hamel Husain and Greg Ceccarelli open with that deliberately inverted assignment. Following Grace’s presentation on best practices, they promise a guide to spectacular failure: not merely a disappointing project, but the kind of organizational damage that threatens companies and careers. The recommendations that follow are intentionally bad advice.
Their perspectives span the executive office and implementation work. Greg introduces himself as an AI startup co-founder, formerly chief product officer at Pluralsight; Hamel is a machine learning engineer and independent consultant who has worked with many companies adopting AI. They frame the exercise with a maxim they attribute to Charlie Munger: “Invert. Always invert.” Instead of describing an ideal strategy, identify the decisions that reliably undermine one.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Divide the company, then disconnect spending from value
Start by dividing your own organization. Set unreasonable goals that disconnect customer willingness to pay, price, and cost. Attend every AI conference, but never bring the learning back to your team. The image of Moses parting the Red Sea supplies the organizational metaphor: create impenetrable silos and reward secrecy. Knowledge accumulates in individuals without becoming a shared basis for decisions.
Next, invert the value stick. The resulting anti-value stick replaces economic reasoning with promises, purchases, and complexity:
| Label | Inverted meaning | Decision it encourages |
|---|---|---|
| WTP | Wishful thinking promises | Promise that AI can do everything. |
| PRICE | Particularly ridiculous infrastructure costs everywhere | Buy expensive GPUs without cost-benefit analysis. |
| Cost | Cascade of spectacular technical debt | Build tightly intertwined systems. |
| WTS | Why this system? | Answer every objection with “because AI.” |
The promises escalate from writing emails to walking dogs, solving climate change, and achieving world peace. Infrastructure spending follows without a calculation of what it buys. Technical debt then becomes a perverse form of job security: make the system so convoluted that only its creator can repair it. Finally, the question of why the system should exist disappears behind AI itself as the justification. The joke ends with a product resembling magic, only more expensive and less reliable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Replace strategy with declarations and a permanent backlog
With the organization divided, strategy can become a sequence of declarations that never requires understanding the work:
- Fake the diagnosis. Take last year’s annual report or operating plan, highlight passages you barely understand, and declare that AI must fix them. Do not consult the people doing the work.
- Make the guiding policy unbounded. Aim to become the global AI leader in everything, leaving everything undefined and someone else’s responsibility.
- Announce disconnected actions. Commission an AI SEO tool that guarantees top Google rankings for garden gnomes, a generative art plugin that makes NFTs of the CEO’s cat, and an AI drone lunch delivery service. Present the collection at an all-hands in a shiny suit, repeatedly invoking disruption.
- Remove the finish line. Replace timelines with perpetual beta. Put the highlighted financial reports into a massive GitHub backlog.
Each step preserves the appearance of strategy while removing a constraint that would make it actionable: a grounded problem, a bounded direction, coherent work, or a completion date.
Communication can protect that ambiguity. Post a deliberately absurd 4,000-page strategy document in every Slack channel until people lose the will to engage. Then explain it with a sentence that piles multimodal agentic transformers, few-shot learning, chain-of-thought reasoning, synergy, and hyperparameter space on top of one another. Confidence substitutes for meaning, including for the person speaking. The rejected alternative is a clear business-on-a-page approach: something concise enough for colleagues to understand and question.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use technical language to exclude the people who know the work
Jargon does more than make documents unpleasant. The name assigned to a task changes who believes they can participate. Hamel describes a mental health client where writing a prompt was described as building agents. The mental health experts were consequently absent from the room and unsure how to contribute. An activity that needed their judgment had been presented as belonging to someone else’s technical specialty.
The same substitution appears in other parts of an AI system:
| Technical label | Work people need to understand |
|---|---|
| RAG | Give the AI the right context. |
| Prompt injection | Users tricking the AI into unwanted behavior. |
| Prompt writing | Specify how the AI should respond to customers. |
The failure comes from using the labels to conceal the work and its owners. If engineers alone write prompts, the people who best understand customers may never shape the instructions. The displayed jargon guide extends the joke: translate ordinary tasks into language that makes even prompt writing seem inaccessible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Mobilize the wrong expertise and ship before testing
The giant backlog now needs people. Greg introduces a parody of Geoffrey Moore’s Zone to Win: zoning to lose. Begin by assigning AI tasks without regard to relevant experience. The concrete example is outsourcing data review to an offshore QA team with little context about the business. The missing business context is the consequential part: reviewers are being asked to judge outputs without the knowledge needed to recognize whether those outputs serve the task.
Next, turn incubation directly into customer exposure. Launch untested, bug-ridden chatbots from the incubation zone, skipping beta testing and quality assurance. What should have been a place to develop new ideas becomes a route to a public failure. Finally, pull the best engineers away from revenue-producing products to support the effort. The South Park-style sequence—remove essential capacity, wait, expect profit—ends here in collapse. The existing business pays for the experiment before the experiment has established that it works.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Treat every failure as a purchasing decision
Once the organization is in disarray, focus on tools instead of processes. Do not analyze the failures already created; respond to each with another technical purchase or intervention:
- Wrong retrieval results: buy a more expensive vector database instead of investigating why the RAG system retrieves the wrong documents.
- Unclear progress: adopt every available evaluation metric without adapting it to business needs, then trust the numbers even when they make little sense.
- Failing agents: switch frameworks and vendors, or fine-tune without measuring whether the change improves anything.
Fine-tuning without evaluation becomes, in Greg’s phrase, “alchemy with a lot more electricity.” The intervention may be substantial, but there is no measurement connecting it to improvement.
This approach also transfers responsibility for evaluation to the vendor: plug in a tool and expect it to define and solve the problem. The Whac-A-Mole demonstration captures the resulting loop. A problem appears, so hit it with a tool. Another appears, so hit that one. The original returns, so choose a different tool. Activity continues, but the organization never develops an explanation for the recurring failure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Collect enough metrics to find a success story
Treating evaluations as a vendor problem leads naturally to the dashboard of everything. Assume that a single evaluation solution can fit every business, collect every off-the-shelf metric, and ignore whether any of them track customer outcomes or actual failure modes. Hamel’s illustrative scores of 3.5 and 4.5 expose the problem: if nobody knows what the difference means, the apparent precision does not help anyone decide what to change. Keep collecting metrics until one trends upward, then claim success.
Evaluation frameworks offer plenty of numbers to populate that dashboard. Greg names cosine similarity, BLEU, and ROUGE as targets one could optimize while ignoring user experience. The missing step is cross-checking those measures with domain experts and users. Accepting an LLM’s judgment that an answer is accurate without challenge simply replaces that check with another automated output. A metric needs a defensible relationship to success in the actual task. Otherwise, optimization can produce a better-looking dashboard without a better product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Avoid the data, then make sure nobody else can inspect it
The most potent failure technique is to stop looking at data altogether. Hamel recommends keeping a blindfold nearby for accidental encounters with evidence. Greg extends the joke into complete trust in AI outputs nobody has inspected. Leaders can then designate data review an engineering problem and return to meetings about meetings, assuming that developers possess more domain expertise than the business teams.
Quality assurance shifts to customers, who might eventually complain if something goes wrong. Meanwhile, leaders substitute intuition for evidence, even in million-dollar decisions. If the facts threaten that confidence, the blindfold returns. The sequence removes both ways the organization might discover a mistake before it becomes expensive: deliberate inspection and informed challenge.
The final escalation is to make data inaccessible to everyone else. Treat engineers as coding wizards who can handle everything, even if they have not spoken to a customer in years. Forget that a spreadsheet can support annotation and direct inspection. The executive’s refrain becomes “Remember, this is beyond me.” Then put the data in systems only engineers can access, ensuring that domain experts cannot review it either.
The practical comparison is a spreadsheet or Airtable versus a custom analysis platform so elaborate that it supposedly takes a team of PhDs to operate. The joke awards bonus points if the platform takes six months to load and errors incessantly. Complexity now obstructs the basic activity the organization needs: letting knowledgeable people examine examples and say what is wrong. A simpler review surface preserves their ability to participate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The advice is inverted; the experience is real
Following this sequence would waste time and resources while alienating the people needed to make AI useful. That is where the satire gives way to the speakers’ direct invitation: visit AI-Execs for practical advice. At the time of the talk, they also announce the forthcoming O’Reilly publication AI Essentials for Tech Executives for February 27. Their recommendations have been inverted, Greg emphasizes, but the lived experience behind them has not. They close by inviting questions at the speaker booth after the presentation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Husain and Ceccarelli's free guide to AI strategy, communication, hiring and direct data review.
Geoffrey Moore's framework separates enterprise work into performance, productivity, incubation and transformation zones.
Further reading
The authors explain process-driven improvement and provide a plain-language AI communication cheat sheet.
- The AdvantageBook
Patrick Lencioni's approach to organizational health through cohesive leadership, clarity and communication.
Felix Oberholzer-Gee explains willingness to pay and willingness to sell, and how businesses create and distribute value.
Full transcript
Loading transcript…