Restoring the Visual Narrative of Sport
This is our entry for IC Hack 26, entering in the Sports and Tech for Good (IMChange) categories.
Ball Knowledge is an AI-powered accessibility framework developed for ICHack 26. Our mission is to provide individual games with the same tactical depth and emotional immersion as professional matches, using a single video feed to generate a rich, spatial audio-description.
The interface for our web app is sleek and intuitive. Using a minimalist approach, we have boiled the page down to its bare essentials as to not overwhelm the user.
- We provide a multimodal method of input: both a direct upload or a YouTube video link can be analysed.
- The method of input is simple and intuitive.
- A final layer of customisation allows the user to tailor the commentator's style to better suit their needs.
While developing Ball Knowledge, we aligned our goals with Access Sport, a charity dedicated to providing disabled and disadvantaged young people with access to the life-changing benefits of sport.
Our framework supports their "Inclusive Club Network" through three primary pillars:
Digital Equity for Local Clubs Access Sport works in the UK’s most deprived areas where high-tech broadcast equipment is non-existent. Ball Knowledge brings "Grand Slam" level technology to local park courts. This can bring professional audio commentary to a local club match or youth tournament, allowing amateur athletes to feel like pros and local clubs to build more inclusive communities.
Sensory-Friendly Spectatorship For neurodivergent fans, standard high-velocity commentary can cause sensory overload. Our platform offers a "Calm Mode" that replaces chaotic hype with a structured, descriptive narrative tailored to the user's sensory needs—aligning with Access Sport’s mission to create inclusive, welcoming environments for all.
Grassroots clubs are the lifeblood of sport, but they rely on volunteers who are often stretched thin. Providing accessibility services like live audio description usually requires dedicated personnel that local clubs simply don't have.
Ball Knowledge acts as an "Automated Volunteer." By handling descriptive commentary and match analysis in the cloud, we empower local clubs to provide a professional, accessible experience for parents and fans without requiring additional human resources. This allows coaches and volunteers to stay focused on what they do best: mentoring the next generation of athletes.
For many, following a fast-paced sport like tennis is an exercise in exclusion. Whether due to visual impairment, neurodivergence, or simply a lack of resources in local parks, standard broadcasts fail to provide an inclusive narrative. Current solutions often ignore:
Spatial Context: Where is the player positioned? Is a local youth player holding their ground or being forced wide?
Sensory Needs: Standard commentary is often a chaotic "sensory minefield" of overlapping noise that can overwhelm neurodivergent fans.
Grassroots Representation: Professional-grade analysis is usually a luxury of the elite, leaving local club matches and youth tournaments invisible.
Ball Knowledge solves this by democratizing sports technology. We use AI to "see" the court, translating raw spatial data into a narrative that restores the viewer's mental model of the game—regardless of their sensory needs or the zip code of the court.
We built Ball Knowledge to ensure that no fan or athlete is left on the sidelines. By removing the need for a sighted companion or a professional broadcast team, we provide a scalable solution for inclusive sport at every level.
Our project is rooted in three core values:
Independent Spectatorship: We provide the tools for blind and partially sighted fans to follow the "spatial story" of a match autonomously. By translating raw data into descriptive audio, we restore the dignity of an independent viewing experience.
Democratizing the Grassroots: High-quality commentary is currently a luxury reserved for the elite. We bring this "Grand Slam" experience to local park courts and youth tournaments, ensuring that disadvantaged communities have access to the same technology as professional academies.
Sensory Autonomy: We recognize that "one size fits all" commentary doesn't work for neurodivergent fans. By allowing users to choose between high-energy personas or "Calm Mode," we respect individual sensory needs and prevent cognitive overload.
A simplified rundown of the processes at work is as follows:
- The user inputs a video to the server.
- YOLO is used to track entities of interest based on their absolute position in pixels on the screen.
- In particular, we track the players, ball and boundary box.
- Planar homography is used to transform this into a top-down view in order to gauge the position of the ball relative to the playing field.
- Next, collections of frames are analysed in order to mark them with an event.
- The Physics of the Game: Detecting bounces, net-crosses, and shots.
- Player Intent: Recognizing when a player is "out of position" or "attacking the net".
- This is then used to create a JSON file that we pass to Claude, which generates a running commentary.
- ElevenLabs is then used to synthesise audio, which we can overlay on the original video.
- This is then passed back to the website so the user can watch at their own enjoyment.
While we are launching with Tennis, the framework is designed to scale across the sporting world:
Adaptive & Parasports: Tailoring the logic to track wheelchair tennis or blind football (5-a-side), providing bespoke commentary for adaptive athletes.
Coaching & Feedback: Allowing athletes, like our team members in Taekwondo, to receive automated tactical feedback to improve their performance and reach their competitive potential.