The $304 Million Lesson in How Not to Delegate to AI

In 2021, Zillow gave its AI algorithm full authority to buy homes. No graduated ramp-up. No human checkpoints on edge cases. Just: here’s the budget, go buy houses.

By Q3 2021, the algorithm had overpaid on thousands of properties. Zillow took a $304 million write-down, shut down Zillow Offers entirely, and laid off 2,000 people — 25% of their workforce.

The postmortem wasn’t complicated. The AI was competent at estimating home values in normal markets. But nobody had defined what “normal” meant, nobody was checking the work when conditions shifted, and nobody had a mechanism to dial back autonomy when the model’s confidence exceeded its accuracy.

Zillow didn’t have an AI problem. They had a delegation problem.

“Using AI” vs. “Working with AI”

There’s a distinction I keep coming back to in conversations with enterprise teams:

Using AI is a tool relationship. You pick it up, you get a result, you put it down. The AI is passive. The human drives everything. ChatGPT drafts an email. Copilot suggests code. Midjourney generates an image. You use the output or you don’t.

Working with AI is a collaboration relationship. There’s delegation, division of labor, and — critically — a feedback loop. The AI handles what it’s good at. Humans handle what they’re good at. The boundary between the two isn’t fixed — it shifts as capability and trust evolve.

This isn’t semantic. It determines whether you get incremental productivity gains or actual transformation.

The data backs this up. When Harvard and BCG studied 758 consultants using GPT-4 on real consulting tasks, they found something surprising: on tasks inside the AI’s capability frontier, consultants using AI were 25.1% faster and produced 40% higher quality work. But on tasks outside that frontier — where the AI gave plausible but wrong answers — consultants using AI performed 23% worse than those without AI.

The researchers called this the “jagged technological frontier” — AI capability isn’t a smooth line. It’s brilliant at some things and confidently wrong at others. The consultants who got the best results weren’t the ones who used AI the most. They were the ones who understood where to delegate and where to stay involved.

That’s the difference between using AI and working with it.

Why This Matters Now

Most organizations are stuck in “using” mode. McKinsey’s 2023 State of AI survey found that while adoption is accelerating, only 33% of organizations were using generative AI regularly in even one business function. The other 67%? Experimenting, piloting, dabbling.

And the MIT Sloan Management Review and BCG Henderson Institute found something even more telling: only 10% of companies achieved significant financial benefits from AI. The differentiator wasn’t technology sophistication. It was how they integrated AI with human workflows. Companies that redesigned workflows around human-AI collaboration were 5x more likely to see significant financial results.

Five times more likely. Not because of better models or more data. Because of better collaboration design.

The companies treating AI as a tool they “use” are getting tool-level returns. The companies working with AI — defining roles, setting boundaries, evolving the relationship — are getting transformation-level returns.

  “Using AI” “Working with AI”
Relationship Human drives everything Division of labor
AI role Passive tool Active collaborator
Interaction Fixed, one-directional Evolving, feedback loop
Capability Incremental Transformational
Sounds like “Draft this email” “Handle customer inquiries under $500. Flag anything unusual. I’ll review weekly.”

What “Working With” Actually Looks Like

When Wilson and Daugherty studied 1,500 companies for their research on human-AI collaboration (published in Harvard Business Review), they found that firms achieved the most significant performance improvements when they designed what they called “reimagined processes” — workflows where humans complement AI strengths and vice versa.

Not AI replacing humans. Not humans supervising AI reluctantly. Genuine collaboration where each side does what it’s best at.

Here’s what that looks like in practice:

JPMorgan’s COiN platform reviews 12,000 commercial credit agreements — work that previously consumed 360,000 hours of lawyer time annually. But JPMorgan didn’t hand the AI full authority to approve loans. The AI extracts clauses, flags anomalies, and surfaces risks. Lawyers review the flags and make decisions. Over time, as accuracy improved in specific clause types, more routine extractions were automated. The delegation expanded gradually based on demonstrated performance.

Stitch Fix uses AI to select candidate clothing items for their subscription boxes, but human stylists make the final curation decisions. The AI handles pattern matching across millions of preference data points — something humans can’t do at scale. The stylists handle taste, context, and the “this doesn’t feel right” judgment — something AI can’t do reliably. CEO Katrina Lake and Chief Algorithms Officer Eric Colson have been transparent about this model: neither the AI nor the humans could deliver the same results alone.

Both examples share a pattern: clear roles, defined boundaries, evolving scope. That’s working with AI, not just using it.

The Cost of Getting This Wrong

The failure mode isn’t just Zillow. It shows up everywhere:

Knight Capital (2012): An automated trading system deployed without adequate human oversight checkpoints. In 45 minutes, a software error accumulated $440 million in losses. The firm was acquired shortly after. The system worked fine in testing — the failure was in the delegation model, not the technology.

Microsoft Tay (2016): A chatbot launched with full autonomy to learn from and respond to Twitter users. No graduated ramp-up, no human review gate for a new system in an adversarial environment. Shut down within 16 hours.

Amazon’s recruiting AI (2018): An AI system was given decision-making authority over resume screening without adequate human review of its criteria. It had learned from 10 years of predominantly male hiring data and was penalizing resumes containing the word “women’s.” The system was competent at pattern matching — but nobody was checking which patterns it matched.

Every one of these is a delegation failure, not a technology failure.

The pattern: organization gives AI authority without defining scope, checkpoints, or escalation paths. AI does what it’s optimized to do — confidently, at scale. Nobody catches the drift until the damage is done.

The Real Question

When I talk to enterprises about AI, the conversation usually starts with technology: which model, what platform, how to integrate.

But the question that actually determines success is simpler: How will you work with this?

  • What decisions will AI make autonomously?
  • What decisions require human review?
  • How will you know when the AI is operating outside its competence?
  • How does the relationship evolve over time?

These aren’t governance questions in the compliance sense. They’re collaboration questions. The same questions you’d ask before delegating a critical process to a new team member.

And that reframing — from “how do we implement AI” to “how do we work with AI” — is what separates the 10% of companies getting real results from the 90% still experimenting.

This is Part 1 of a 4-part series on Working with AI. Next: The Delegation Model — What Working with AI Looks Like in Practice.

References:

  1. Dell’Acqua, F., McFowland, E., Mollick, E., et al. “Navigating the Jagged Technological Frontier.” Harvard Business School Working Paper 24-013, September 2023.
  2. McKinsey & Company. “The State of AI in 2023: Generative AI’s Breakout Year.” August 2023.
  3. Ransbotham, S., et al. “Winning With AI.” MIT Sloan Management Review and BCG Henderson Institute, October 2019.
  4. Wilson, H.J. and Daugherty, P. “Collaborative Intelligence: Humans and AI Are Joining Forces.” Harvard Business Review, July-August 2018.
  5. Zillow Group Q3 2021 Earnings Report. November 2021.
  6. SEC. “Report on Knight Capital Group LLC’s August 1, 2012 Trading Event.” October 2013.
  7. Reuters. “Amazon scraps secret AI recruiting tool that showed bias against women.” October 2018.