Back at Google I/O the search giant announced a feature called Live Caption that transcribes audio into subtitles and puts them on the screen. Now, thanks to XDA-Developers, we can see how the feature works - it analyzes any audio stream and places captions on the display. Google said in May that Live Caption will come with select phones, running Android 10, and given the Pixel 4 launch in two weeks, it is safe to assume the new handsets will be among the first devices with the feature.

Live Caption screenshots

The screenshots from an unreleased APK, obtained from a Pixel 4 XL, reveal the setup is pretty easy - you can even add a toggle in the Notification Center. Currently, it only works with English, but more languages are “coming soon”. The feature is found in the Accessibility menu, although you can also access it through the Sound panel.

You can even mask profanity in Live Caption - it comes out censored with asterisks, although it would be funny if it said “fork” or “bullshirt” like in the “The Good Place” series with Kirsten Bell.

The author at XDA shared a bunch of screenshots from YouTube, Google Photos, Amazon Prime Video, and Netflix, and it worked in all these apps. Google says the action is done offline and no audio is stored. It would be interesting to see whether it will work in Spotify and if it will manage to provide proper lyrics from modern rappers.

Due to memory and space constraints, Live Caption will be available to “select, high-end devices”. Currently, the OnePlus 7T, which is the first phone to launch on Android 10 lacks the feature, so it is obviously going to have limited reach initially.

Via