FUT Evolution: building an AI coach for Ultimate Team
FUT Evolution is a companion app for EA SPORTS FC Ultimate Team that I design, build, and run solo: iOS, Android, and web. It started as one more stats app in a crowded market. It became a bet that the next companion app is not a database but a coach: an AI agent that knows your squad, a scanner that imports that squad from a single screenshot, and a computer-vision layer that watches your actual matches. This page is the story of that bet, act by act, with the real numbers.
A market of lookup tables
Ultimate Team is the flagship mode of EA SPORTS FC, one of the most played sports games in the world. You collect player cards, assemble a squad under formation and chemistry constraints, and trade on a live transfer market with its own economy, price cycles, and daily meta shifts. Around that economy grew an entire ecosystem of companion tools: price trackers, tier lists, squad builders, stats databases.
They are good at what they do. And what they do is lookup.
| What every companion tool shows | What the player actually asks |
|---|---|
| The price of any card, updated live | Who should I buy with my budget? |
| Global tier lists and meta ratings | Who fits my formation and my chemistry? |
| Full stats for thousands of cards | Where is the weak spot in my squad? |
| Price history graphs | Is this the moment to sell my striker? |
Every question on the right side is personal and combinatorial: formation times chemistry times budget times the market at this exact moment. Answering it is coaching, not lookup. No tool coached, because coaching cannot be produced editorially for millions of different squads. It has to be computed, per user, on demand.
The bet: a coach, not a database
FUT Evolution's bet is simple to state: you import your squad, and a coach does the rest. It knows your players, your budget, your weak positions. It scouts upgrades, checks chemistry, watches prices, and tells you what to do and why.
The app around the coach is deliberately boring. Kotlin Multiplatform holds the shared business logic, networking with Ktor, and the data models. iOS renders in SwiftUI with SKIE for clean Swift interop, Android renders in Compose Multiplatform, and the web companion is Next.js. The backend is Next.js API routes over PostgreSQL on Supabase, a Python FastAPI vision service on DigitalOcean, player images on S3, subscriptions through RevenueCat. All three clients and the coach share one database.
The interesting part is what the bet forced me to solve. A coach that is worth trusting needs three things no lookup app needs:
Act I: a coach that renders, not replies
The first coach was scripted: rules over ratings, prices, and positions. It worked, and it hit the wall every scripted assistant hits. Conversational requirements compound. "Compare these two" is easy to route to a template. "Compare these two, but prioritize chemistry with my current squad, within my budget, and tell me if prices are falling" is not one template, it is a combinatorial space of them, and every new dimension multiplies the routing states a human has to enumerate.
So the coach became an LLM agent, and the engineering problem became reliability. Two decisions carry it:
- The database sits beside the model, never inside it. The model is not allowed to answer from memory. Player facts, prices, and stats come only through its 11 tools, which read the same PostgreSQL database the apps use. Players named in the message are pre-fetched before the model even runs, and a plain TypeScript classifier keeps trivial messages away from the model entirely.
- The answer is interface, not paragraphs. Ask it to compare two strikers and you get a radar overlay, stat bars, and the market prices of that moment, streamed as native design-system components. The model composes the layout inside a finite, validated vocabulary; a repair layer catches its bad days before anything renders. The system owns the skeleton, the model owns the composition.
And the injected views are not passive. Every component the model streams is a live piece of the app, carrying the same actions as any native screen. Ask the coach to compare Mbappé and Dembélé and the answer itself is the interface: tap a player card and the native player sheet opens with the full model behind it, prices, alternatives, chemistry. When the coach suggests an upgrade, the suggestion ships with its own button, and tapping it applies the transfer to your squad. The model decides which views to compose; the app guarantees every one of them can act.
Act II: your squad in one screenshot
A coach that does not know your squad is a tier list with manners. But asking users to type 23 player names into a form is where onboarding goes to die. The squad already exists, rendered on their screen inside the game, so the app takes it from there: screenshot in, full squad analysis out, in 2.5 seconds.
A card in that screenshot is mostly art. Names are stylized, truncated, and overlaid on gradients, so text alone is a fragile identifier. The most reliable constant on every card is the player's face. Identification starts there and uses text only to break ties:
Face similarity alone gets 82% of players exactly right. The remaining errors are the lookalike problem: two players sharing a haircut and a squad photo style. That is where the OCR pass earns its 500 milliseconds: a face match that disagrees with the card's rating or position is demoted, and the disambiguated result reaches 95% accuracy. Chasing that last stretch, percentage point by percentage point, was the project's real lesson in production ML: the first 82% came fast, the last 13 points were the real work.
| Stage | Budget | Result |
|---|---|---|
| Face detection (InsightFace) | ~1s | every card face found |
| FAISS similarity search | ~300ms | 82% exact player match |
| OCR disambiguation | ~500ms | 95% exact player match |
| Full scan, screenshot to analysis | 2.5s | squad, formation, weak spots, upgrades |
Act III: Hypervision, teaching the app to watch FC 27
The coach knows your squad. It still cannot see the only thing that matters: how you actually play. Hypervision is the layer I am building for the FC 27 cycle: paste a video of your match and get back what a real analyst would produce, where your lines break, which flank leaks, whether your striker's runs match how you pass. No companion app does this today, because watching gameplay video is brutally hard. This is the prototype, and the honest state of it.
What already works
Ball detection came first, because every downstream event (passes, shots, turnovers) depends on it: 99.6% detection on gameplay footage.
Players came second, and detection alone is not enough: knowing that twenty-two bodies are on the pitch tells you nothing until you know which shirt each one wears. A YOLO11s model finds the players, then a kit classifier reads the torso of every box and splits the pitch into your team and theirs, with the referee separated out so he never pollutes either side. The threshold is not hardcoded; it is fitted per clip from the kit brightness of the players actually on screen, which is what lets the same code survive a change of strip.
On top of that sits the mapping layer. A pose model finds 32 pitch keypoints, used zero-shot from a public broadcast dataset because no FC 26 keypoint labels exist. RANSAC fits a homography from those keypoints, and every player's feet are projected from screen pixels into pitch metres, with a Hungarian assignment mapping positions to formation roles.
| Metric | Measured |
|---|---|
| Ball detection | 99.6% |
| Frames with a valid pitch fit (FC 26 footage) | 69 of 72 (96%) |
| Median projection error, screen to pitch | 0.38 m |
| Worst-case projection error | 1.84 m |
| Goalkeeper recall | 0.583, the current weak spot |
What blocks the product, measured
A prototype that works on curated clips is not a feature. The feature is "paste a YouTube URL of your match", and measuring on real YouTube FC 26 uploads surfaced two product-blocking facts that were invisible while working on curated footage:
- 63% of frames are not live play. Cutscenes, replays, celebrations, menus. Feed those to the pipeline and the analysis is garbage before any model runs. The fix in progress is a live-play gate in front of everything, extending the radar-presence detector that already skips faded and absent frames.
- Ball tracking holds only for the default blue ball. Players customize match balls, and the detector was trained on one. The fix is boring and necessary: training data across ball skins.
Both findings are the top of the FC 27 backlog, and both are the reason this section exists on this page. Shipping the demo would have been easy. Measuring it against real user footage is what told me the truth.
How I build products
Three acts, one method. If a recruiter takes anything from this page, it should be these four habits, because every one of them is visible above:
- Ship the thin slice end to end. The scripted coach shipped before the LLM one. Ball detection shipped before player detection. Each slice was real, used, and taught me what the next one had to be.
- Measure before believing. 82% face match became a number before it became a feature; 63% non-live frames killed a roadmap item I was excited about. The demo is never the evidence, the measurement is.
- Keep the model on a leash. The LLM never touches raw truth: data lives in the database, reached through tools, and its output passes a validator before a user sees it. Deterministic edges, creative middle.
- Publish the failures. The goalkeeper recall is weak, the blue-ball limitation is real, and they are on this page. Honest numbers are what make the good ones believable.
Stack and scope
Solo project, 2023 to present. Kotlin Multiplatform, SwiftUI, Compose Multiplatform, SKIE, Ktor, Next.js, PostgreSQL on Supabase, Python FastAPI, InsightFace, FAISS, YOLO, RevenueCat, deployed on DigitalOcean, S3, and Cloudflare. One database shared by iOS, Android, web, and the coach.