Start Hacking Now: What a €XXM API Migration Taught Me About AI in Production

The Problem ## The 6 Key Learnings ## 1. 🔨 Hack First, Polish Never ## 2. ⚡ AI Generates Code > AI Runs Tests ## 3. 🗄️ MCP: Query JSON Like a Database ## 4. 🎲 Accept Being Surprised ## 5. 💾 Offline AI > Online AI (Sometimes) ## 6. 🎓 Knowledge Transfer > Expert Does Everything ## Practical Takeaways ## The Messy Middle Is the Point ## Demo Code ## Discussion Questions This week at API Days Paris, I watched something rare: a client and consultant presenting together about an AI project that actually shipped to production. Cyrille Martraire (CTO at Arolla) and Thomas Nansot (Director of Engineering at a major European mobility platform) walked through their journey using AI to validate a critical API migration—one handling hundreds of millions of tickets annually. What made it different? They shared the dead ends, the failed approaches, and how their final solution was nothing like the original plan. Thomas's company needed to migrate from a legacy API to a new architecture. The stakes were high—any regression could affect millions of transactions. Traditional testing would be prohibitively expensive and slow, especially since every contract change meant redoing the work. They needed a way to guarantee non-regression between APIs with completely different structures. When Thomas reached out, Cyrille didn't ask for requirements docs. He immediately built a hacky prototype with fake systems to prove the concept could work. The lesson: In AI projects, velocity of learning beats polish. You can't plan your way through uncertainty—you prototype your way through it. Their first production attempt was elegant: AI agent does everything end-to-end. The breakthrough? Use AI to generate the test code, not run the tests. The pattern: AI works "offline" to create tools, then those tools do the actual work. The API schemas were massive. Cramming everything into the LLM context caused attention issues—even when it technically fit, quality degraded. Solution: Model Context Protocol (MCP) They specifically recommended the "JSON-to-MCP" tool. Why it matters: We're discovering design patterns for LLMs at scale. MCP is like moving from "here's a phone book" to "here's a search interface." Thomas's quote hit hard: "As a manager, I expected to have a clear vision of what would work. I had to admit the solution was completely different from what I imagined—and it was better." The winning solution wasn't in the original plan. They needed short feedback cycles and willingness to pivot. The mindset: If you bring too much certainty to AI projects, you limit yourself. Let the technology surprise you. Key insight: "AI is sometimes better offline." Examples of offline AI: After proving the technical concept, Cyrille's role shifted from "maker" to "coach." Impact: Even AI-skeptical engineers got excited about these techniques for their own work. Real value = Solving the problem + Building internal capability If you're considering AI for production: What I appreciated most was their honesty. Too many AI talks show polished end results and skip the dead ends. But the dead ends are the story. That's where the learning happens. They didn't have a perfect plan. They had a hypothesis, willingness to iterate, and courage to be surprised. That's probably the most valuable lesson of all. I created a working demo of these patterns: [GitHub link] Cyrille Martraire - CTO at Arolla, author of "Living Documentation," deeply embedded in software craft community Thomas Nansot - Director of Engineering managing ~150 engineers on a mobility distribution platform serving millions across Europe If you attended API Days Paris or have experience with AI in production, I'd love to hear your takeaways in the comments! Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse COMMAND_BLOCK: # Quick prototype approach legacy_response = fake_legacy_api() new_response = fake_new_api() ai_compares(legacy_response, new_response) # Does it work? Kind of? Good enough to continue! Enter fullscreen mode Exit fullscreen mode COMMAND_BLOCK: # Quick prototype approach legacy_response = fake_legacy_api() new_response = fake_new_api() ai_compares(legacy_response, new_response) # Does it work? Kind of? Good enough to continue! COMMAND_BLOCK: # Quick prototype approach legacy_response = fake_legacy_api() new_response = fake_new_api() ai_compares(legacy_response, new_response) # Does it work? Kind of? Good enough to continue! COMMAND_BLOCK: # ❌ Approach 1: Live AI (expensive, slow) for each test: result = ai.compare(legacy_api(), new_api()) # $$$ # ✅ Approach 2: Generated code (cheap, fast) test_code = ai.generate_comparison_code() # Once for each test: result = test_code.run() # $0, deterministic Enter fullscreen mode Exit fullscreen mode COMMAND_BLOCK: # ❌ Approach 1: Live AI (expensive, slow) for each test: result = ai.compare(legacy_api(), new_api()) # $$$ # ✅ Approach 2: Generated code (cheap, fast) test_code = ai.generate_comparison_code() # Once for each test: result = test_code.run() # $0, deterministic COMMAND_BLOCK: # ❌ Approach 1: Live AI (expensive, slow) for each test: result = ai.compare(legacy_api(), new_api()) # $$$ # ✅ Approach 2: Generated code (cheap, fast) test_code = ai.generate_comparison_code() # Once for each test: result = test_code.run() # $0, deterministic CODE_BLOCK: prompt = f"Analyze this entire JSON: {10mb_schema}" Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: prompt = f"Analyze this entire JSON: {10mb_schema}" CODE_BLOCK: prompt = f"Analyze this entire JSON: {10mb_schema}" CODE_BLOCK: mcp = JSONMCPServer(huge_schema) email = mcp.query_path("passenger.email") keys = mcp.list_keys("journey.segments") Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: mcp = JSONMCPServer(huge_schema) email = mcp.query_path("passenger.email") keys = mcp.list_keys("journey.segments") CODE_BLOCK: mcp = JSONMCPServer(huge_schema) email = mcp.query_path("passenger.email") keys = mcp.list_keys("journey.segments") - Slow: 2+ minutes per test - Expensive: ~$1 per test run - Unreliable: Random failures - Live AI: $1 × 1000 tests = $1000 - Generated: $2 to generate + $0 × 1000 = $2 - Full AI approach → Too slow & expensive - Slice & compare → Too complex - Generated code + MCP → Success! - ✅ AI generates test suites → run 1000x - ✅ AI writes Terraform modules → apply repeatedly - ✅ AI creates validation rules → check all data - ✅ AI generates docs templates → reuse forever - External expert builds solution - Proves it works, gets buy-in - Expert teaches internal team hands-on - Team runs it independently - Learnings apply to other projects - Start with hacky prototypes - Consider generated artifacts over live decisions - Use MCP-style patterns for large data structures - Plan for short feedback cycles - Build internal capability, not just solutions - Wait for perfect requirements - Assume "full AI" is always the answer - Fight context window limits—work around them - Plan everything upfront - Keep expertise external - MCP server for JSON querying - AI code generation examples - Fake APIs for quick prototyping - Generated vs. live AI comparison - Have you tried the "AI generates code" pattern in your projects? How did it compare to live AI? - What's your biggest challenge with LLM context windows? - How do you balance exploration vs. planning in AI projects?