Your AI feature works in the demo. Is it ready for real users?
A fixed-price, 10-business-day review of everything around the model: evaluation, guardrails, cost and operations. It ends with a written report and a prioritized plan your team can execute.
Most AI features stall between demo and production. It's almost never the model.
The model is usually good enough. What's missing is everything that makes it safe to put in front of customers and keep it there.
No evaluation set
You can't prove it's good, and you won't notice when it gets worse.
No guardrails
Nothing stops it when it's wrong.
No cost ceiling
Nobody knows what the bill looks like at 10x usage.
No operations plan
Who gets paged when it breaks? How do you turn it off?
Five areas, the same ones in the checklist.
Each area gets a score, the risks it carries and what to fix first.
Problem and success criteria
What the feature is supposed to do, for whom, and how you'll know it's working.
Evaluation
Whether you can measure quality, compare changes and catch regressions before users do.
Guardrails and control
Deterministic checks on outputs, human approval for high-impact actions, prompt injection and PII handling.
Cost and performance
Cost per request and per user, a 10x usage projection, limits, p95 latency, and whether a smaller model would do the job.
Operations and ownership
Versioning of prompts and models, logging, alerts, provider portability and a runbook.
Ten business days, three steps.
Kickoff and access
A kickoff call, then access to the repo, prompts, sample traffic or logs, and cost data.
Review
Architecture, prompts, evaluation and costs, plus short interviews with 2–3 of your engineers.
Report and walkthrough
A written report with risks ranked by impact, a prioritized 30/60/90-day plan, and a 60-minute walkthrough call.
A report you can act on, not a slide deck.
- Written readiness report with a scored assessment across the 5 areas
- Top risks ranked by business impact
- A starter evaluation set design, built from your real cases
- Cost model with a 10x usage projection
- Prioritized plan your team can execute
10 business days. No retainer required.
Book a 20-min callIf you want help executing the plan afterwards, I can stay on as fractional CTO or bring senior engineers from my team.
David Levy
Fractional CTO and AI Engineering Lead. Founder of Together Devs.
Leading software teams since 2005, as developer, development manager, solution architect, founder and CTO.
AI layer for an investment platform
Built the AI layer for an investment platform with 100+ investors and over USD 100M on it. Financial advisors reviewed every suggestion first, deterministic checks wrap every output, and every action needs human approval. Nothing reaches the market without it. Automation came last.
4 million accounts, zero downtime
Led the migration of 4 million user accounts for Channel 4 (UK) from Akamai to Auth0 with zero downtime, running both systems in sync until the new one earned the switch.
Built for one situation.
A good fit
Teams with an AI feature in prototype or early production that leadership wants in front of customers.
Not a fit
Teams still choosing a use case from scratch, or companies looking for a slide deck on "AI strategy".
Frequently asked
Do you need access to our code?
Read-only access to the repo, prompts and logs. An NDA is available.
What if we're not on OpenAI?
The review is vendor-neutral. It works with any provider or with open models.
Will you tell us to rebuild?
The plan prioritizes the smallest changes that de-risk the launch.
What happens after the 10 days?
You own the report. If you want, I can stay on as fractional CTO or provide engineering support to execute the plan.
Is the work remote?
Yes, with overlap with US working hours.
Find out what stands between your demo and real users.
A 20-minute call is enough to tell whether the audit fits where your feature is today.
david@togetherdevs.net