The Problem

We all have watched movies or shows which are not in the language you understand so, so we rely on subtitles to understand what's happening. Those little lines of text at the bottom of the screen are our window into a world of new stories, cultures, and ideas. But what if you could have those same helpful subtitles in real life? What if we could create a system which uses AI to listen to conversations, translate them in real-time, and display them as subtitles on a smartphone, tablet, laptop etc..?

That's exactly why we created SubsIRL - Subtitles in Real Life. Imagine walking down a street as tourist in Paris, Spain, Germany etc... and seeing live subtitles on your device screen as people speak around you. How easy it would be to understand all that without asking anyone to translate but instead get live translated subtitles in choice of you language just like we see on Screen while watching movies and shows with subtitles.

How to Use?

You can use it on any device which has browser like smartphones, laptops and desktops.

There are 2 mods,

1. Without camera feed, just subtitles

2. With Camera feed, to give better experience like we get while watching a movie/show with subtitles.

Below are screenshots of both the use:-

How SubsIRL Works?

The app uses your device's microphone to listen to nearby conversations. It breaks the spoken sentences into multiple chunks. These chunks are then sent continuously in a specific window size for transcription and translation in the given order.

Our AI workflow, powered by OpenAI's Whisper 3 Large STT Model on Groq's LPU, quickly transcribes the speech into text. Then, the LLM model (Llama 3.1 70b) on Groq's LPU quickly translates the text into your preferred language.

The translated text appears on your screen in real-time, just like subtitles in a movie!

This is a Beta version, we are working on some edge cases and trying to increase the usage limits.

This is very short explanation of the entire workflow in terms of diagram.

Thanks to the incredible lighting fast processing power of LPU Groq, all of this happens almost instantly. It's like having a super-powered interpreter in your pocket! Languages and Features Currently, SubsIRL supports seven (for now, more languages support will be added soon) languages: English, Deutsch (German), Español (Spanish), Русский (Russian), Français (French), Português (Portuguese), and Nederlands (Dutch). But we're not stopping there - we're working hard to add more languages and dialects all the time. (Follow @srx9 to be updated)

Real-World Applications:- The potential uses for SubsIRL are endless:

Travelers can understand locals in foreign countries.

Students can follow along in classes taught in different languages

Business professionals can participate more fully in international meetings.

Attending events, live speeches etcc... where the language speaken by the speaker is not known by you.

and much more...

Whether you're planning a trip abroad, trying to learn a new language, or just curious about how it works, we'd love for you to give it a spin.

Visit https://subtitlesinreal.life/ to try SubsIRL for yourself. For now its usage is limited due to limited scalability, which we are working on to increase soon.

Tech Stacks

For our WebApp (will have mobile app soon), I used Next.js (TypeScript) with Groq SDK, Whisper 3 Large Model, Llama 3.1 70B LLM, Aceternity UI and Magic UI for the UI part deployed on Vercel.

Useful Links

• Website -> https://subsirl.vercel.app/

• Website repo on GitHub -> https://github.com/SRX9/substitle-irl-groq

• Twitter to follow for updates -> https://x.com/s_r_x_9

Future Enhancement

Our next steps are to increase scalability to be able to serve huge number of users, add more languages support and improve even more accuracy in translation.

Thank You

We are committed to continuously improving SubsIRL based on feedback. We encourage users to share their experiences and suggestions to help us refine and enhance the tool.

#aifortomorrow