Problem Statement
The FIFA World Cup is one of the most widely viewed sporting events in the world, bringing together billions of viewers every four years. However, a large portion of this audience consists of casual fans who do not regularly watch soccer outside of major international tournaments. While these viewers are highly engaged during matches, they are often unfamiliar with the technical terminology used by commentators, making it difficult to fully understand and appreciate the game. FIFA-dex addresses this challenge by using artificial intelligence to identify and explain soccer-specific vocabulary in the context of real match footage, helping viewers expand their understanding of the sport while they watch. By transforming complex commentary into an interactive and engaging learning experience, FIFA-dex enables casual fans to follow matches with greater confidence, deepen their knowledge of the game, and enhance their overall World Cup viewing experience.
The project is organized into two primary folders. The web folder contains the frontend web application, including the user interface and video playback experience. The api folder contains the FastAPI backend, including the video-processing pipeline, AI integrations, and API endpoints used by the frontend.
Technical Approach
Users begin by uploading a World Cup match clip through our web application. The video is transmitted to a FastAPI backend hosted on Hugging Face using a multipart file upload request. The backend leverages Docling to extract and transcribe spoken commentary from the video. Rather than generating a single block of text, Docling produces a structured transcript consisting of timestamped segments. These timestamps allow our system to connect educational content directly to specific moments within the match. Once transcription is complete, the timestamped transcript is passed to IBM Granite through Watsonx.ai. Granite performs natural language analysis on the commentary and compares the transcript against our curated soccer terminology knowledge base, looking for any matches.
For each detected concept, Granite generates:
- The soccer term being discussed
- A beginner-friendly definition
- A context-aware explanation of how the term applies to the current play
- Supporting transcript evidence
- Confidence scores for concept identification
- Associated timestamps within the video
This transforms raw commentary into structured educational content that can be understood by viewers with little prior soccer knowledge. After identifying technical soccer terminology, Granite rewrites commentary into accessible language. Rather than simply defining a term, the model explains the concept using the exact play being shown on screen. This contextual learning approach enables users to connect abstract soccer vocabulary with real match situations, significantly improving retention and understanding. Each identified soccer concept is added to the user's FIFA-dex, a gamified vocabulary collection system inspired by collectible achievement mechanics.
Every entry contains:
- The soccer term
- Category classification
- Beginner explanation
- Match-specific explanation
- Supporting video moments
- Collectible token rewards
Users unlock new terms as they continue watching matches, encouraging long-term engagement and continuous learning.
The processed results are returned to the frontend, where discoveries are synchronized with video timestamps. During playback, users receive contextual notifications whenever a new soccer concept appears. Selecting a discovered term immediately jumps the user to the associated moment in the match, allowing them to review the play while reinforcing the educational explanation generated by Granite.
Why This Solution Matters
Soccer is the world's most popular and unifying sport, yet many new viewers struggle to understand the specialized language used during broadcasts. FIFA-dex lowers the barrier to entry by turning passive viewing into an active learning experience.
Instead of requiring users to search for definitions separately, the platform teaches terminology directly within the context of the match. This approach benefits a combination of casual World Cup viewers, new soccer fans, youth players learning the game, and International audiences unfamiliar with soccer terminology.
By making broadcasts more understandable and engaging, FIFA-dex helps viewers develop deeper knowledge of the sport while maintaining the excitement of watching live matches.
- Real-time processing of live matches
- Automatic detection of exact gameplay moments using computer vision
- Personalized learning recommendations
- Difficulty levels for beginner, intermediate, and advanced fans
- Multi-language support for international audiences
- Integration with live broadcasts and streaming platforms
All five developers behind FIFA-dex are first-year engineering and computer science students at Northwestern University, each bringing unique perspectives from our respective home states. Building this platform allowed us to dive deep into enterprise-grade AI integration, gain hands-on experience hosting scalable web applications via custom APIs, and grow closer as friends. We are incredibly proud of our submission, and we sincerely appreciate the judges' time and consideration.