Cheap vs Correct AI Integration: Cost of Hype

A cheap AI integration demos in five minutes. A correct AI integration changes whether users come back.

The difference is not model price. It is whether the AI sits inside a trustworthy job.

We have not published a separate post-mortem of a failed AI feature in a client product, so this page sticks to the framework we actually use in reviews — not an invented story.

AI DECISION — DOES IT EARN RETENTION?

We ship AI only when it improves completion or removes friction we can measure.

✕ Hype trap

AI demo that adds steps. Looks clever, completion drops.

Defer

✓ Earns retention

AI removes a real step or error. Measured completion ↑.

Ship

— No change

AI that doesn’t change the job. Adds cost, not value.

Skip

⚠ Risky

AI helps some, hurts others. Needs instrumented test.

Test first

high friction ↑low retention → high retention

We instrument completion rate before/after. If it doesn’t move, the AI doesn’t ship.

Cheap AI: what it looks like

Cheap AI is a generation step added on top of a fragile flow.

• Prompt stitched into a screen without typed boundaries — number for money, string for status.
• No clear empty, failure, and permission-denied states for the AI output itself.
• No measurement of whether the job is finished faster or more reliably.
• The demo is a generated paragraph; the production question is “what do we retry when the model times out?”

It feels fast. It generates support load where the generation is wrong in a way the user cannot spot quickly. The cost shows up after launch.

Correct AI: what we aim for

Correct AI is a material inside a well-defined job, as described in our AI retention guide.

• Typed boundaries around AI output — the same discipline we use for money in Ankik, where balanceInCents is a bigint with explicit reconciled | pending | disputed.
• Optimistic UI where relevant — show the result now, verify in the background, explain clearly on failure, as shown in good engineering by design.
• Instrumented before/after: completion rate, time-to-complete, and retry rate measured on the core job.
• Five states designed for the AI path as well: what empty looks like, what measurable “success” is, what failed safely preserves, and who can fix a denied case.

The engineering is not about the model vendor. It is about where the generation lives in the product.

Hype vs correct — comparison we use in reviews

Dimension	Cheap (hype)	Correct (earn retention)
Placement	On top of a fragile flow	Inside a typed job with clear boundaries
States	Happy path only; errors bubble as 500s	Empty, loading, success, failure, denied — all designed
Measurement	None, or vanity “tokens used”	Completion rate and friction before/after
Cost view	Model call price only	Model + support + rework + trust cost
Retention impact	Demo liked, return rate flat or down	Return rate moves because the job is genuinely easier
When we ship	Because a deck needed “AI”	Because a specific friction was measured and removed

We review each proposed AI feature against this table before we estimate it. The same table determines where it falls in the MVP cost breakdown — a cheap call that creates a support queue is not cheap.

The cost of hype

Hype cost is not the invoice. It is:

• Users who pause to verify every generation, adding a second job (checking) to the first.
• Support threads about “is this number right?” because money was a float and status was a string.
• A second build to re-wire the job after users have already learned to distrust it.

We prefer the checklist from our MVP pillar: if the AI path cannot answer the five states, it does not ship — no matter how good the demo looks in a review.

For scope, use our pruning template. For timeline, see how we ship in 28 days.

FAQ — cheap vs correct AI integration

What is cheap vs correct AI integration? Cheap ships a prompt on a fragile flow and optimizes for the demo. Correct places generation inside a typed job with empty, failure, and denied states and measures completion rate before and after.

Why does cheap AI cost more after launch? The invoice hides support threads, verification pauses, and a second rebuild once users distrust the output. Correct AI counts model plus support, rework, and trust cost.

How do you know if AI earns retention? Instrument the same core job twice — with and without the AI variant — and keep it only if more strangers finish faster and with fewer retries.

Have an AI idea you want to pressure-test? Map Your AI Idea to Hype vs Correct — we put each proposal in the table with you and instrument the one that earns retention.